5  Sample size changes with design

So far, we have considered the simplest design: “assign equal numbers to two groups, everyone participates to the end, and the endpoint is measured only once”. In actual studies, the required sample size changes with the allocation ratio, dropout, baseline measurement, and clusters (allocation by site or school).

We continue with the example of difference Δ = 5 mmHg, standard deviation σ = 10 mmHg, two-sided 5%, and 80% power (64 per group).

5.1 Changing the allocation ratio

Sometimes you want to allocate more people to the new treatment (to collect more safety data, or to make it easier for participants to receive the new treatment). Allocating k people to the new treatment for every 1 to the conventional treatment, the size of the conventional group is

\[ n_0 = \left(1 + \frac{1}{k}\right) \frac{(z_{1-\alpha/2} + z_{1-\beta})^2 \sigma^2}{\Delta^2}, \qquad n_1 = k \, n_0 \]

(the formula based on the normal distribution).

  • Even at 2:1, the total is only about 1.14 times
  • At 3:1 it is about 1.33 times. The larger the ratio, the faster efficiency deteriorates

Let us check whether 2:1 (48 conventional, 96 new treatment) really gives 80% power.

It came out at about 80% (being a simulation, it fluctuates by about 1 point).

5.2 Allowing for dropout

Suppose you expect 20% of participants to drop out before assessment. How many should you increase the sample size to?

The correct one is division (64 ÷ (1 − 0.2) = 80 people). If 20% of 80 drop out, 64 remain. With multiplication (64 × 1.2 ≈ 77), 20% of 77 drop out and only 61–62 remain.

WarningIncreasing the sample size does not remove bias

Increasing the sample size to allow for dropout is to maintain power. If those who dropped out are disproportionately “people for whom it did not work” or “people who had side effects”, an analysis of only those remaining gives biased results. Increasing the sample size does not remove this bias. Decide at the planning stage how to handle data from dropouts (missing data and multiple imputation hands-on).

5.3 Measuring at baseline

Many trials measure the endpoint (blood pressure) also before treatment (at baseline). Using the baseline value in the analysis can greatly reduce the required sample size.

Letting ρ be the correlation between baseline and post-treatment values, the variance used by each analysis method is as follows.

Analysis What is compared Variance When ρ = 0.6
t-test of post-treatment values Post-treatment value σ² 1×
t-test of change Post-treatment − baseline 2(1 − ρ) σ² 0.8×
ANCOVA (regression adjusting for baseline) Post-treatment value (adjusted for baseline) (1 − ρ²) σ² 0.64×

The required sample size is proportional to the variance, so with ρ = 0.6:

With ANCOVA, 42 people per group suffice. We simulate a trial with 42 people per group 2000 times and compare the power of the three analyses.

With the same 42 people, comparing only post-treatment values gives about 64% power, change about 72%, and ANCOVA about 81%.

TipIf you can measure the baseline value, plan with ANCOVA
  • ANCOVA is more efficient than the analysis of change for any ρ (if ρ is below 0.5, the analysis of change is worse than analyzing post-treatment values alone)
  • Estimate ρ from pilot studies or existing longitudinal data. Use a conservative value (a smaller ρ)
  • Besides baseline, adjusting for variables strongly related to the endpoint (such as age) can reduce the sample size further

5.4 Paired data

For paired data, such as giving two treatments to the same person (crossover trials) or comparing the left and right eyes, use the standard deviation of within-person differences \(\sigma_d\).

If the standard deviation of the differences is 8 mmHg, 23 people (each receiving both treatments) suffice. Between-person variability cancels out, so far fewer people are needed. However, it requires the assumption that there are no carry-over or period effects.

5.5 Allocating by cluster

In cluster randomized trials, where interventions are allocated by site, ward, or school, people in the same cluster are similar to each other, so each person is not independent information. The degree of similarity is expressed by the intraclass correlation coefficient (ICC), and with m people per cluster, the required sample size is multiplied by the design effect

\[ DE = 1 + (m - 1) \times ICC \]

Even with a small ICC of 0.05, with 20 people per cluster the design effect is 1.95. The sample size almost doubles, and 7 clusters (140 people) per group are needed.

5.5.1 Even if you increase the cluster size

Increasing the number of people per cluster reduces the number of clusters needed, but the total number of people keeps increasing. What mainly determines power is the number of clusters. With too few clusters (4–5 or fewer per group), the analysis itself becomes unstable.

5.5.2 If you analyze individuals one by one

What happens if you ignore the clusters and run a t-test on individual data? We check in a world where there is truly no difference.

Ignoring clusters, about 17% are significant despite no difference (it should be 5%). In cluster trials, account for clusters in both planning and analysis (comparison of cluster means, mixed models, GEE, etc.; for mixed models see the linear mixed models hands-on).

5.6 Try it

NotePractice
  1. Allowing for 15% dropout, how many people should the 42 per group planned with ANCOVA become?
  2. If the ICC is 0.02, how many clusters per group are needed with 20 people per cluster?
  3. Simulate the power of a trial with 7 clusters per group and 20 people per cluster using the t-test on cluster means (true difference 5 mmHg). If it does not reach 80%, try adding one cluster
  1. 50 people (42 ÷ 0.85 = 49.4, rounded up)
  2. 5 clusters (design effect 1.38, 89 people per group)
  3. About 79% with 7 clusters, and over 80% with 8. With few clusters, the degrees of freedom of the t-test (2k − 2) are small and power is lower than the normal-distribution formula suggests, so adding 1–2 clusters is a rule of thumb
ImportantSummary of this chapter
  • An allocation ratio of 2:1 means about 1.14 times the total; 3:1 about 1.33 times
  • Allow for dropout by division (n ÷ (1 − dropout rate)). Increasing the sample size does not remove bias
  • If you measure baseline, planning with ANCOVA reduces the sample size by a factor of (1 − ρ²)
  • For paired data, calculate with the standard deviation of the differences
  • For cluster trials, multiply by the design effect 1 + (m − 1) ICC. The number of clusters determines power