2  Where does the χ² distribution come from?

In Chapter 1, we obtained a p-value by comparing the χ² value of 9.12 with “the χ² distribution with 1 degree of freedom”. In this chapter, we build a world with truly no relationship inside the computer and check how the χ² value is distributed.

2.1 Building a world with no relationship: shuffling labels

If smoking and death are unrelated, it makes no difference if we shuffle the labels of “who died” independently of smoking. A shuffled table has the same row totals (582 smokers, 732 non-smokers) and the same column totals (369 deaths) as the original, and is a table whose contents alone are determined by chance.

Try running it several times (if you delete set.seed(1), you get a different result each time). The χ² value is usually a small value between 0 and a few.

We repeat this 10,000 times. In a 2×2 table, once the marginal totals are fixed, the whole table is determined by the single cell “smoker and dead”, so we count only that number to compute quickly.

The distribution of χ² values from shuffling (grey) overlaps well with the χ² distribution with 1 degree of freedom (blue).

Out of 10,000, 0.40% (40 times) were at or above the actual value (9.12). This agrees with the p-value of 0.25% computed from the χ² distribution in that both are “much smaller than 1%” (with more shuffles, they get even closer). The χ² distribution can be thought of as an approximation of “what happens if we shuffle”.

Notechisq.test() can do the same thing

With simulate.p.value = TRUE, the p-value is computed from random tables with fixed marginal totals (r2dtable()). This can be used when expected frequencies are small and the χ² approximation is questionable (Chapter 5).

2.2 The χ² test for a 2×2 table = the z-test for a difference between two proportions

The χ² test for a 2×2 table is the same as testing “whether the proportion who died is the same in smokers and non-smokers”. Let us compute the z value, i.e. the difference in proportions divided by its standard error.

z² = 9.12, matching the χ² value. z follows a standard normal distribution (if there is no relationship), and the square of a standard normal variable follows the χ² distribution with 1 degree of freedom.

prop.test() is a test of a difference in proportions, and its χ² value and p-value are the same as those of chisq.test(). In addition, it gives the 95% confidence interval for the difference in proportions (difference in proportion dead 7.5 percentage points, 95% CI 2.7 to 12.4 percentage points).

2.3 Degrees of freedom: the number of cells that can be chosen freely

In general, the sum of the squares of k mutually independent standard normal values follows the χ² distribution with k degrees of freedom. The χ² value sums (O − E)²/E over as many terms as there are cells, but because the marginal totals are fixed, not all cells can move freely.

  • 2×2 table: with the row and column totals fixed, deciding one cell determines the other 3 → 1 degree of freedom
  • Table with r rows × c columns: the last row and the last column are determined by subtraction from the totals → (r − 1)(c − 1) degrees of freedom

2.3.1 Checking by simulation

Among NHANES adults, using the marginal totals of race (5 categories) × self-rated health (5 levels), we randomly generate tables with no relationship. The degrees of freedom should be (5 − 1)(5 − 1) = 16.

The χ² values of tables with no relationship have a distribution with a mean of about 16, overlapping the χ² distribution with 16 degrees of freedom. Judge the size of a χ² value relative to its degrees of freedom. A χ² value of 9 is very large with 1 degree of freedom (p = 0.003), but rather small with 16 degrees of freedom.

2.4 When the approximation breaks down

The χ² distribution is an approximation for when the counts are sufficiently large. In tables with few people, the χ² value can take only discrete values and deviates from the χ² distribution. Let us simulate a situation in which only 20 people are selected from Whickham (10 smokers, 10 non-smokers, 6 deaths).

The χ² value takes only 4 values — 0, 0.95, 3.81, and 8.57 — and exceeds the 5% point of the χ² distribution (3.84) only about 1% of the time (it should be 5%). In this case, the p-value computed from the χ² distribution deviates from reality. How to deal with this is covered in Chapter 5.

ImportantSummary of this chapter
  • The χ² distribution is an approximation of “how the χ² value is distributed if there is no relationship (if the labels are shuffled)”
  • The χ² test for a 2×2 table = the z-test for a difference between two proportions (z² = χ²). prop.test() also gives the confidence interval of the difference
  • The degrees of freedom are the number of cells that can be chosen freely with the marginal totals fixed = (r − 1)(c − 1). Judge the χ² value relative to its degrees of freedom
  • With few people, the χ² value becomes discrete and the approximation deviates