Mastering the χ² Test: A Hands-on Course

From expected frequencies and degrees of freedom to small tables, paired data, stratification, and power — with real data and simulation

Published

October 17, 2026

Introduction

English version — machine-assisted translation (by generative AI, Anthropic Claude) of the original Japanese hands-on. 日本語版はこちら.

The χ² (chi-squared) test is one of the most commonly used tests, from Table 1 of a paper to the primary analysis of a clinical study. In R, one line — chisq.test(table(x, y)) — gives you a p-value. But behind that one line, a great deal is left out:

  • What is being compared with what, and why does that value follow a “χ² distribution”?
  • How are the degrees of freedom determined?
  • How far is “if an expected frequency is below 5, use Fisher” correct?
  • Can it be used for data in which the same person was measured twice?
  • When the p-value is small, where in the table, and by how much, is there a difference?

In this hands-on, using real epidemiological data and R running in your browser, you will compute the χ² test by hand, check it by simulation, and experience common misuses.

Goals of this hands-on

  1. Compute by hand the expected frequencies, χ² value, degrees of freedom, and p-value, and be able to explain the output of chisq.test()
  2. Confirm by simulation that the χ² distribution is an approximation, and judge when the approximation breaks down
  3. Be able to choose, according to the shape of the data, among goodness-of-fit, independence, McNemar, Mantel-Haenszel, and trend tests
  4. Be able to report results not only with p-values but also with residuals, effect sizes, and confidence intervals
  5. Take home a checklist for choosing a test

Who this is for

  • People involved (or about to be involved) in analysing clinical research, epidemiological, or healthcare data
  • People who roughly understand proportions, cross-tables, and what a p-value means (roughly the level of Japan’s Statistics Certificate Grade 3)
  • For R, it is enough if you can “read code, modify it a little, and run it”

R runs in your browser

The code cells in each chapter are executed by R inside your browser (a technology called webR). Edit the code and press “Run Code” and the result changes immediately. No installation is needed.

Note
  • Right after opening each chapter, loading R and the packages takes from several tens of seconds to about a minute
  • If you move the tab to the background while loading, processing slows down, so please wait until it is displayed
  • Viewing on a PC is recommended

Almost all the functions used come from the stats package that ships with R (chisq.test, fisher.test, mcnemar.test, mantelhaen.test, prop.trend.test, power.prop.test, etc.).

Data used

Data Contents Main chapters
Whickham 1314 women in the UK. Smoking in 1972–74 and vital status 20 years later 1, 2, 7
Framingham (teaching) A US heart study. 4434 people examined up to 3 times. Blood pressure, smoking, BMI, diabetes, etc. 3, 6, 7
NHANES 2009–2012 The actual sample of a US health and nutrition survey (about 20,000 people). Education, self-rated health, race, etc. 4, 8
WarningNotes on the data

Part of the Framingham teaching dataset has been modified for anonymisation. It cannot be cited as research findings. The NHANES sample is a survey that oversampled certain racial groups and age groups. In this hands-on, we tabulate without weights in order to learn how the tests work, so the results are not values for the US population (for weights, see Chapter 5 of the descriptive statistics hands-on).

Chapter overview

Chapter Contents
1 Computing expected frequencies and the χ² value by hand
2 Where the χ² distribution comes from (simulation, degrees of freedom, relation to the test of a difference in proportions)
3 Goodness-of-fit tests (Mendel’s peas, the last digit of blood pressure, estimated parameters and degrees of freedom)
4 r×c tables: where is the difference (residuals), how large is it (Cramér’s V), p-values in large samples
5 When counts are small (the expected-frequency-of-5 rule, Yates’ correction, Fisher’s exact test, type I error)
6 Non-independent data (McNemar’s test, data with multiple rows per person)
7 Strata and order (Mantel-Haenszel test, trend test)
8 Power, multiple comparisons, and how to report
9 Exercises and a checklist for choosing a test

Each chapter follows the flow “first use chisq.test() naively → predict and interpret the result → check what is inside → notice the pitfalls”.