Sample Size and Power: A Hands-on Course
How many participants do you need? With both formulas and simulation — from means and proportions to non-inferiority, clusters, and analyses without a formula
Introduction
English version — machine-assisted translation (by generative AI, Anthropic Claude) of the original Japanese hands-on. 日本語版はこちら.
“How many participants do I need?” is one of the most frequently asked questions in planning a study. In R, the single line power.t.test(delta = 5, sd = 15, power = 0.8) gives an answer. But behind that one line, a lot is left out, such as:
- What exactly is 80% in “80% power”?
- What values should be entered for the difference
deltaand the standard deviationsd? - How do dropout, the allocation ratio, baseline values, and clusters change the sample size?
- If you want to show that “there is no difference”, is the same calculation valid?
- What should you do for analyses without a formula (logistic regression, survival time, mixed models, …)?
In this hands-on, using R running in your browser, you will calculate sample sizes with formulas and check them by simulation, and experience common mistakes.
Goals of this hands-on
- Explain the relationship among the significance level, power, effect size, variability, and sample size
- Calculate the required sample size for differences in means and proportions with the formulas and
power.t.test()/power.prop.test() - Judge how to estimate differences and standard deviations, and how their uncertainty affects the sample size
- Adjust the sample size for dropout, allocation ratio, baseline adjustment, clusters, and non-inferiority
- Obtain power by simulation, even for analyses without a formula
- Report the justification for the sample size in a form you can write into a study protocol
Who this is for
- People planning (or about to plan) research in clinical research, epidemiology, or healthcare data
- People with a rough understanding of what means, standard deviations, p-values, and t-tests are (introductory statistics level)
- For R, being able to “read code, modify it a little, and run it” is enough
R runs in your browser
The code cells in each chapter are executed by R inside your browser (a technology called webR). Edit the code and press “Run Code”, and the results change immediately. No installation is needed.
- Right after opening each chapter, loading R and the packages takes from a few tens of seconds to about a minute
- Simulation cells repeat thousands of tests inside your browser, so they can take from a few seconds to a dozen or so seconds
- Viewing on a PC is recommended
Almost all functions used come from the stats package that ships with R (power.t.test, power.prop.test, t.test, glm, etc.), plus replicate() for simulation.
Data used
| Data | Content | Main chapters |
|---|---|---|
| NHANES 2009–2012 | A real sample from the US National Health and Nutrition Examination Survey (about 20,000 people). Blood pressure, BMI, cholesterol, etc. | 2, 4 |
| Simulation | Data generated after deciding the “true difference” ourselves. Used to check power | All |
In this hands-on, NHANES is used in place of “data from previous studies or pilot studies” to estimate standard deviations and the like. It is summarized without survey weights, so the values are not those of the US population (weights are covered in Chapter 5 of the descriptive statistics hands-on).
Chapters
| Chapter | Content |
|---|---|
| 1 | What is power? (relationship among significance level, power, effect, variability, and sample size) |
| 2 | Difference in means: formula, power.t.test(), simulation, estimating the standard deviation |
| 3 | Difference in proportions: power.prop.test(), how the reference proportion changes the sample size, specifying by ratios |
| 4 | Deciding the effect size: clinically meaningful differences, pitfalls of pilot studies and previous studies |
| 5 | Sample size changes with design: allocation ratio, dropout, baseline adjustment, paired data, clusters |
| 6 | Non-inferiority and equivalence: margins and one-sided tests |
| 7 | When there is no formula: obtaining power by simulation (logistic regression, survival time) |
| 8 | The fallacy of post-hoc power, sample size based on precision, how to report |
| 9 | Exercises and a checklist for sample size design |
Each chapter follows the flow “first get the sample size with a function → check what is inside with the formula → check by simulation that the power really is achieved → notice the pitfalls”.
Related hands-ons: chi-squared test hands-on (Chapter 8: power for proportions and r×c tables), multiple testing hands-on (the significance level when there are multiple primary endpoints), missing data and multiple imputation hands-on (handling data from dropouts).