7 Strata and order: the Mantel-Haenszel test and the trend test
The χ² tests so far looked at only two variables. In this chapter, we cover two things:
- Splitting into strata by a third variable (a confounder) and then testing them together (the Mantel-Haenszel test)
- Using the order of categories to test for a trend (the Cochran-Armitage trend test)
Both answer questions that the ordinary chisq.test() does not.
7.1 Do smokers live longer? — once more
In Chapter 1, among the Whickham women, we got the result “smokers had fewer deaths 20 years later” (χ² = 9.12, p = 0.003). Let us compute the odds ratio (with smokers in the first row).
The odds ratio is 0.68: smokers have lower odds of death. However, the age distributions of smokers and non-smokers were different.
Among those aged 65 or over, there are 193 non-smokers and 50 smokers. Because there are many older non-smokers, non-smokers have more deaths 20 years later.
7.2 Look stratum by stratum
Make a 2×2 table for each age group. This gives a 3-dimensional table.
In every age group, the proportion who died is higher among smokers. That it reverses when everything is combined is Simpson’s paradox (Chapter 1 of the descriptive statistics hands-on).
Running a χ² test in each stratum, none is significant.
Each stratum has few people, so the difference cannot be found one stratum at a time. We would like to combine the results of the 3 strata into a single test.
7.3 The Mantel-Haenszel test
The Mantel-Haenszel (MH) test computes, in each stratum, the gap between observed and expected for the cell “smoker and dead”, sums these over all strata, and then turns the result into a χ² value.
\[ \chi^2_{\mathrm{MH}} = \frac{\left(\sum_k (a_k - E[a_k])\right)^2}{\sum_k \mathrm{Var}(a_k)} \]
Because comparisons are made only within strata, it is a test with confounding by the stratifying variable (age) removed. Let us compute it by hand.
mantelhaen.test() also gives the MH χ² value and the common odds ratio adjusted for age group (the MH odds ratio).
- Unadjusted (overall) odds ratio: 0.68 (smokers have fewer deaths)
- Odds ratio adjusted for 3 age strata: 1.35 (95% CI 0.96 to 1.90, p = 0.08)
The direction has reversed.
7.3.1 Coarse strata leave confounding behind
Even within “18–44” and “45–64”, there are age differences of about 20 years. Let us use 10-year strata.
Adjusting with 6 strata gives an odds ratio of 1.52 (95% CI 1.07 to 2.16), p = 0.02. With coarse strata, age differences remain within strata (residual confounding), and the adjustment is insufficient. In the 75+ stratum, everyone died, so it carries no information for the comparison.
The MH test combines a common odds ratio on the assumption that “the odds ratio is (about) the same in every stratum”. If the direction of the relationship is opposite in different strata, combining them makes them cancel each other out, and the relationship is missed. First, line up the stratum-specific odds ratios and check. Tests of whether the odds ratios are the same across strata (homogeneity) include the Breslow-Day test and Woolf’s test below.
The odds ratios of the 3 age groups are about 1.5, 1.3, and 1.2, and the homogeneity test gives p = 0.92. It seems fine to combine them.
The MH test answers almost the same question as a logistic regression with age group entered as a category, glm(death ~ smoking + age_group, family = binomial). If you want to enter age as a continuous variable, or there are many variables to adjust for, use a regression model.
7.4 Ordered categories: the trend test
In Chapter 4, we confirmed that the χ² test ignores the order of rows and columns. When the categories are ordered, like BMI categories or age groups, and you want to know whether “the proportion rises (falls) as the category increases”, use a trend test (the Cochran-Armitage trend test).
At the first Framingham examination, look at the proportion with diabetes by BMI category (the same table as in Chapter 5).
The higher the BMI category, the higher the proportion with diabetes: 1.7% → 2.9% → 4.5% → 9.2%.
7.4.1 Decomposing the χ² value
The ordinary χ² value (3 degrees of freedom) can be split into the linear trend part (1 degree of freedom) and the remaining deviation (2 degrees of freedom).
Of the overall χ² value of 35.2, 30.7 is explained by the linear trend, and the remaining 4.4 (2 degrees of freedom, p = 0.11) is not significant. We see that the simple relationship “the higher the BMI, the higher the proportion with diabetes” explains almost everything.
7.4.2 If there is a trend, the trend test finds it more easily
When there is truly a linear trend (10% → 13% → 16% → 19%), we simulate how likely it is to be significant when studying 80 people per category.
The probability of significance (power) is about 26% for the ordinary χ² test and about 41% for the trend test. The ordinary χ² test (3 degrees of freedom) watches for every pattern of “something differs somewhere”, so its power is spread out. If you have decided in advance to look for a trend, the 1-degree-of-freedom trend test finds differences more easily with the same number of people.
- By default,
prop.trend.test()gives the categories equally spaced scores 1, 2, 3, …. When the category widths differ (e.g., use BMI medians), you can specify them withscore =, but decide the scores before seeing the results - Running both the χ² test and the trend test and reporting whichever has the smaller p-value is multiple testing (Chapter 8)
- A significant trend test does not mean the relationship is linear (it can be significant even for a U-shape). Always line up the proportions and look
- Splitting into strata by a confounder and combining the within-stratum comparisons is the Mantel-Haenszel test. It also gives an adjusted odds ratio
- Coarse strata leave residual confounding. If the stratum-specific odds ratios differ greatly, report them by stratum without combining
- For ordered categories, use the trend test (1 degree of freedom). The χ² value can be decomposed into “trend” and “the rest”
- Decide which test to use and how to assign scores before looking at the data