5  Interaction: is the effect the same for everyone?

So far, we have estimated a single odds ratio on the premise that “the effect of the exposure is the same for everyone”. However, the effect may differ by sex or age. This is called effect modification, or, in model terms, interaction.

Confounding and interaction are completely different things.

5.1 Diabetes and cardiovascular disease: does it differ between men and women?

In Framingham, we look at the association between diabetes at the first examination and 24-year CVD, separately for men and women.

  • Odds ratio: men 2.67, women 4.75
  • Risk ratio: men 1.65, women 2.63
  • Risk difference: men 22.6 percentage points, women 29.4 percentage points

On every scale, the impact of diabetes appears larger in women. In women, the risk of CVD without diabetes is low (18%), so the extra risk due to diabetes appears relatively large.

5.1.1 Testing with an interaction term

Whether the effect differs between men and women is checked with a model that includes an interaction term.

DIABETES:factor(SEX)2 is the interaction term, representing (the log of) how many times the odds ratio for diabetes in women is relative to that in men. exp(0.646) = 1.91 times (close to 4.75 / 2.67), p = 0.10. Because there are only about 60 people with diabetes in each sex, we cannot conclude either that there is a difference or that there is none.

5.2 A common mistake: “significant in men, not significant in women”

It is a mistake to test each subgroup separately and conclude “it was significant in men but not in women, so there is an effect only in men”. Even when the effect is exactly the same, it often happens that only one subgroup is significant if the sizes differ.

Although the effect is exactly the same in men and women, about 60% of studies come out “significant in men only”. The interaction test is significant in about 5% (the chance level).

WarningCautions for subgroup analysis
  • “Significant in one, not significant in the other” is not evidence of a difference in effect. Test and estimate the interaction (the difference in effects) directly
  • If you examine many subgroups, a difference will appear somewhere by chance (multiple comparisons). Claim only pre-planned subgroups
  • Interaction tests have low power. A non-significant result does not prove the absence of a difference either

5.3 Interaction depends on the “scale”

Whether there is an interaction or not depends on whether you look at odds ratios / risk ratios (multiplicative) or risk differences (additive).

In a simulation, consider an exposure whose risk ratio is 2 in every age group (no multiplicative interaction).

The risk ratio is 2 in both, but the risk difference is 2 percentage points in the young and 20 percentage points in the old. From a public health perspective (where should we intervene to prevent more disease?), interaction on the additive scale (risk difference) is important.

5.3.1 Measuring additive interaction

From the risk for each combination of two factors (diabetes, female), we can compute the interaction contrast (IC).

\[ IC = p_{11} - p_{10} - p_{01} + p_{00} \]

(\(p_{11}\): both present, \(p_{10}\), \(p_{01}\): only one present, \(p_{00}\): neither)

If IC is 0, there is no interaction on the additive scale (the risk difference due to diabetes is the same in men and women). In this example, standardized to everyone’s age distribution, the risk difference due to diabetes is 21 percentage points in men and 31 in women, and the IC is about 10 percentage points. Confidence intervals are obtained by the bootstrap (Chapter 3).

ImportantSummary of this chapter
  • Confounding is something to remove; effect modification is something to report
  • Whether the effect differs is tested and estimated directly with an interaction term. “Significant in one, not in the other” is not evidence
  • In subgroup analyses, claim only those planned in advance
  • Whether there is interaction depends on the scale. Show both odds ratios / risk ratios (multiplicative) and risk differences (additive)