Mastering the χ² Test: A Hands-on Course
From expected frequencies and degrees of freedom to small tables, paired data, stratification, and power — with real data and simulation
Introduction
English version — machine-assisted translation (by generative AI, Anthropic Claude) of the original Japanese hands-on. 日本語版はこちら.
The χ² (chi-squared) test is one of the most commonly used tests, from Table 1 of a paper to the primary analysis of a clinical study. In R, one line — chisq.test(table(x, y)) — gives you a p-value. But behind that one line, a great deal is left out:
- What is being compared with what, and why does that value follow a “χ² distribution”?
- How are the degrees of freedom determined?
- How far is “if an expected frequency is below 5, use Fisher” correct?
- Can it be used for data in which the same person was measured twice?
- When the p-value is small, where in the table, and by how much, is there a difference?
In this hands-on, using real epidemiological data and R running in your browser, you will compute the χ² test by hand, check it by simulation, and experience common misuses.
Goals of this hands-on
- Compute by hand the expected frequencies, χ² value, degrees of freedom, and p-value, and be able to explain the output of
chisq.test() - Confirm by simulation that the χ² distribution is an approximation, and judge when the approximation breaks down
- Be able to choose, according to the shape of the data, among goodness-of-fit, independence, McNemar, Mantel-Haenszel, and trend tests
- Be able to report results not only with p-values but also with residuals, effect sizes, and confidence intervals
- Take home a checklist for choosing a test
Who this is for
- People involved (or about to be involved) in analysing clinical research, epidemiological, or healthcare data
- People who roughly understand proportions, cross-tables, and what a p-value means (roughly the level of Japan’s Statistics Certificate Grade 3)
- For R, it is enough if you can “read code, modify it a little, and run it”
R runs in your browser
The code cells in each chapter are executed by R inside your browser (a technology called webR). Edit the code and press “Run Code” and the result changes immediately. No installation is needed.
- Right after opening each chapter, loading R and the packages takes from several tens of seconds to about a minute
- If you move the tab to the background while loading, processing slows down, so please wait until it is displayed
- Viewing on a PC is recommended
Almost all the functions used come from the stats package that ships with R (chisq.test, fisher.test, mcnemar.test, mantelhaen.test, prop.trend.test, power.prop.test, etc.).
Data used
| Data | Contents | Main chapters |
|---|---|---|
| Whickham | 1314 women in the UK. Smoking in 1972–74 and vital status 20 years later | 1, 2, 7 |
| Framingham (teaching) | A US heart study. 4434 people examined up to 3 times. Blood pressure, smoking, BMI, diabetes, etc. | 3, 6, 7 |
| NHANES 2009–2012 | The actual sample of a US health and nutrition survey (about 20,000 people). Education, self-rated health, race, etc. | 4, 8 |
Part of the Framingham teaching dataset has been modified for anonymisation. It cannot be cited as research findings. The NHANES sample is a survey that oversampled certain racial groups and age groups. In this hands-on, we tabulate without weights in order to learn how the tests work, so the results are not values for the US population (for weights, see Chapter 5 of the descriptive statistics hands-on).
Chapter overview
| Chapter | Contents |
|---|---|
| 1 | Computing expected frequencies and the χ² value by hand |
| 2 | Where the χ² distribution comes from (simulation, degrees of freedom, relation to the test of a difference in proportions) |
| 3 | Goodness-of-fit tests (Mendel’s peas, the last digit of blood pressure, estimated parameters and degrees of freedom) |
| 4 | r×c tables: where is the difference (residuals), how large is it (Cramér’s V), p-values in large samples |
| 5 | When counts are small (the expected-frequency-of-5 rule, Yates’ correction, Fisher’s exact test, type I error) |
| 6 | Non-independent data (McNemar’s test, data with multiple rows per person) |
| 7 | Strata and order (Mantel-Haenszel test, trend test) |
| 8 | Power, multiple comparisons, and how to report |
| 9 | Exercises and a checklist for choosing a test |
Each chapter follows the flow “first use chisq.test() naively → predict and interpret the result → check what is inside → notice the pitfalls”.