1  Computing expected frequencies and the χ² value by hand

TipYou can run R in the browser in this chapter

Code cells can be edited and re-run by pressing “Run Code” (R is running inside your browser). The first time, loading the packages takes several tens of seconds.

First, we use chisq.test() naively, and then rebuild the numbers in its output one by one by hand.

1.1 Data: smoking and vital status 20 years later

Smoking in 1972–74 among 1314 women in Whickham, UK, and whether they had died 20 years later.

Of the 582 smokers, 139 had died; of the 732 non-smokers, 230 had died.

1.2 First, a naive test

We get X-squared = 9.1209, df = 1, p-value = 0.002527 (the meaning of correct = FALSE is explained in Chapter 5). Let us check where these three numbers — the χ² value, the degrees of freedom, and the p-value — come from.

1.3 Build the table “if there were no relationship”: expected frequencies

The χ² test considers what the table should have looked like if we assume “smoking and death are unrelated (independent)”, and compares it with the actual table.

Overall, 369 of 1314 people, i.e. 28.1%, died. If smoking and death were unrelated, 28.1% should have died among both smokers and non-smokers.

  • 582 smokers × 28.1% = 163.4 deaths, the remaining 418.6 alive
  • 732 non-smokers × 28.1% = 205.6 deaths, the remaining 526.4 alive

These are called expected frequencies. In general, they can be computed as

\[ E_{ij} = \frac{(\text{total of row } i) \times (\text{total of column } j)}{\text{grand total}} \]

In R, use outer() to make a “multiplication table” of the row totals and column totals.

The result of chisq.test() contains the same thing.

NoteExpected frequencies keep the marginal totals fixed

In the table of expected frequencies too, the row totals (582, 732) and column totals (369, 945) are the same as in the original table. Think of it as “the table with the marginal totals unchanged, but with independent contents”. This property leads to the degrees of freedom in Chapter 2.

1.4 Summarising the gap between observed and expected in one number: the χ² value

Look at the differences between the actual table (observed frequencies \(O\)) and the expected frequencies \(E\).

We see that deaths among smokers are 24.4 fewer than if there were no relationship, and deaths among non-smokers are 24.4 more. In a 2×2 table, the size of the gap is the same in all 4 cells (because the marginal totals are the same, once one cell deviates the others are determined).

Squaring the gap in each cell, dividing it by the expected frequency, and summing gives Pearson’s χ² value.

\[ \chi^2 = \sum_{\text{all cells}} \frac{(O - E)^2}{E} \]

This matches X-squared = 9.1209 from chisq.test().

1.4.1 Why divide by the expected frequency?

The same “gap of 24 people” means something completely different in a cell with an expected frequency of 20 and in a cell with 2000. For counted data (frequencies), the larger the expected frequency, the larger the chance variation as well (in the Poisson distribution, variance ≈ mean). So we divide the squared gap by the expected frequency to put it on the scale of “how large the gap is compared with chance variation”.

A gap of 24 people in a cell with expected frequency 20 is “more than 5 times the chance variation (about 4.5 people)”, whereas in a cell with 2000 it is only “about half of roughly 45 people”.

1.5 The p-value: is this χ² value a size that arises by chance?

Under “no relationship”, the χ² value approximately follows a χ² distribution with 1 degree of freedom (why this is so, and what degrees of freedom are, is Chapter 2). The probability of getting 9.12 or more is the p-value.

If there is no relationship, the χ² value exceeds 3.84 only 5% of the time. 9.12 is far beyond that.

1.6 Do not enter proportions (%)

The χ² value depends on the size of the “frequencies”. If you copy the proportions (%) from a table in a paper into chisq.test(), you get a completely different p-value.

If you enter proportions, the whole thing is treated as a table of “200 people”, and p = 0.24 (the correct value is 0.0025). Always enter a table of numbers of people (frequencies) into the χ² test.

WarningWhat this test says and does not say

All the χ² test showed is that “smoking and death 20 years later do not seem to be independent”. Here it was significant in the direction of fewer deaths among smokers, but this was because the smokers were younger (confounding). In Chapter 7, we confirm that the interpretation changes when we stratify by age and test again (see also Chapter 1 of the descriptive statistics hands-on).

1.7 Practice

NoteTry it

The following table shows the numbers of people who improved with a fictitious new treatment (A) and a conventional treatment (B). Compute the expected frequencies and the χ² value yourself, and check whether they match chisq.test(correct = FALSE).

The expected frequencies are 37.5 and 22.5 in every row (the case in which 75 improved ÷ 120 people = 62.5% improve in either group). The χ² value is 8.0, p = 0.0047.

ImportantSummary of this chapter
  • Expected frequency = row total × column total ÷ grand total. The table “with the marginal totals unchanged, if the contents were independent”
  • χ² value = Σ (observed − expected)² / expected. Dividing by the expected frequency puts it on the scale of chance variation
  • The p-value is the probability, under the χ² distribution, of a χ² value at least this large
  • Enter a table of numbers of people (frequencies). Entering proportions gives a completely different p-value