6  Non-inferiority and equivalence

Sometimes you do not need to show that a new drug works better than the conventional one, but want to show that “it works about as well, with fewer side effects, lower cost, or easier use”. The question then is not “is there a difference?” but “is the new drug not worse beyond an acceptable range?” (a non-inferiority trial).

In this chapter too, the endpoint is the reduction in blood pressure (mmHg; larger is better), and the standard deviation is 10 mmHg.

6.1 “No significant difference” does not mean “about the same”

Suppose a trial with 30 people per group found no significant difference between the new drug and the conventional drug.

With p = 0.82, there is “no significant difference”. However, the 95% confidence interval for the difference is −4.95 to 3.96 mmHg, and we cannot rule out that the new drug is nearly 5 mmHg worse. The smaller the sample, the harder it is to get a significant difference, so if “no significant difference means equivalent” were allowed, smaller and sloppier trials would find it easier to show equivalence.

6.2 The non-inferiority margin

In a non-inferiority trial, you first decide the difference “up to which being worse is acceptable”: the non-inferiority margin M. Here, M = 3 mmHg.

The judgment is made with the confidence interval.

  • If the lower limit of the 95% confidence interval of the difference (new drug − conventional drug) is greater than −M, non-inferiority is shown
  • This is the same as rejecting the null hypothesis “the new drug is worse by M or more” at one-sided 2.5%

Trial C has an estimate of −1.5, close to trials A and B, but its confidence interval straddles −M, so it cannot show non-inferiority. This is because its confidence interval is wide (the sample is small).

6.3 Required sample size

When the true difference is 0 (the new and conventional drugs work equally well), we want the probability that the lower limit of the 95% confidence interval is above −3 to be 80%. This is the same as doing a one-sided 2.5% test on a difference between “0 and −3”.

\[ n = \frac{2 (z_{1-\alpha} + z_{1-\beta})^2 \sigma^2}{(M - |\Delta_{\text{true}}|)^2} \qquad (\alpha \text{ is one-sided } 0.025) \]

176 people per group. It is more than the superiority trial with a 5 mmHg difference (64) because the 3 mmHg margin is small. Let us check by simulation.

About 81% of trials showed non-inferiority. Viewed as a superiority test, about 5% of the same trials are significant (naturally, as there is truly no difference).

6.3.1 When the new drug is slightly worse

If you expect the new drug to be truly slightly worse (e.g., −1 mmHg), the “room” up to the margin shrinks, and the sample size rises sharply.

If the true difference is −1 mmHg, 394 people are needed, more than twice that for a difference of 0.

6.4 Equivalence trials

When you want to show not only “not worse” but also “not better” (e.g., a generic drug being the same as the brand-name drug), use an equivalence trial. Equivalence is shown if the 95% confidence interval falls entirely between −M and +M (testing each side at one-sided 2.5%: TOST).

When the true difference is 0, both boundaries must be cleared at the same time, so β is split in half between the two sides.

The formula gives 234 people per group, and the simulation gives a power of about 78%. The normal-distribution formula is slightly more optimistic than the t-distribution. In planning, add a few people or decide after checking by simulation.

NoteA 90% confidence interval is sometimes used to judge equivalence

In bioequivalence studies, it is customary to perform TOST at one-sided 5% each (= whether the 90% confidence interval falls within the range). Since the sample size changes with which is used, check regulatory guidelines and the conventions of the field.

6.5 Non-inferiority for proportions

Suppose the cure rate is expected to be 85% in both groups, with a margin of 10 percentage points.

With 201 people per group, the power is about 80%.

6.6 How to decide the margin

The margin is the most important value driving the sample size, and it is a clinical and regulatory judgment, not a statistical one.

WarningCautions when deciding the margin
  • Make it much smaller than the effect the conventional drug has against placebo (estimated from past trials). A commonly used value preserves at least 50% of that effect
  • A larger margin reduces the sample size, but even “a drug no different from placebo” could then be declared non-inferior
  • Decide the margin before looking at the data, and write it in the protocol with its rationale
  • In non-inferiority trials, ITT analysis (as randomized) is not necessarily conservative (if dropout and non-adherence make the groups more similar, non-inferiority becomes easier to show). Check non-inferiority with both ITT and per-protocol analyses

6.7 Try it

NotePractice
  1. If the margin is changed from 3 mmHg to 4 mmHg, how does the sample size of the non-inferiority trial (true difference 0, 80% power) change?
  2. If the cure rate is 70% in both groups, how many people per group does a non-inferiority trial with a 10-point margin need?
  1. 100 people per group. Widening the margin from 3 to 4 mmHg multiplies the sample size by (3/4)² ≈ 0.56. You can see how much the choice of margin drives the sample size
  2. 330 people per group. The closer the proportion is to 50%, the larger the variability, and the more people are needed
ImportantSummary of this chapter
  • “No significant difference” does not mean “about the same”. If you want to show equivalence or non-inferiority, plan for it from the outset
  • Non-inferiority is judged by “lower limit of the 95% CI of the difference > −M” (a one-sided 2.5% test)
  • The sample size is inversely proportional to (M − true difference)². Expecting even a slightly worse true difference sharply increases the sample size
  • Equivalence is judged by whether the confidence interval falls within −M to +M. Since β is split between the two sides, more people are needed than for non-inferiority
  • The margin is a clinical and regulatory judgment. Decide it before looking at the data, with its rationale