2 Bland-Altman plots
The method that Bland and Altman published in the Lancet in 1986 is now the standard for evaluating agreement. The idea is simple: just plot each individual’s difference against the mean of the two measurements.
2.1 The 1st and 2nd blood pressure readings
Blood pressure is recorded in 2 mmHg steps, so the points would overlap as they are. They are drawn slightly shifted with geom_jitter().
There are three things to read.
- Mean difference (bias): +1.1 mmHg. The 1st reading tends to be slightly higher
- 95% limits of agreement: mean difference ± 1.96 × SD of the differences = −10.3 to +12.5 mmHg. “For most (95%) people, the difference between the two measurements falls within this range”
- Trend: whether the difference changes with the magnitude of the values (sloping upwards, widening)
That the 1st blood pressure reading tends to be high is known to be due to nervousness or habituation (a phenomenon close to the so-called white-coat effect), and NHANES’s official blood pressure (BPSysAve) also uses the mean of the 2nd and 3rd readings.
2.1.1 Why is the horizontal axis the “mean”?
If you use the conventional measurement (or one of the two values) on the horizontal axis, measurement error alone creates a spurious relationship between the difference and the horizontal axis (if one value happens to be high, the difference is also large). Using the mean of both on the horizontal axis makes this spurious relationship almost disappear (if the two measurements have the same variability).
Even though we merely measured with the same monitor in succession, putting the 1st reading on the horizontal axis makes it look as if “the higher the value, the larger the difference”.
2.2 Evaluating Devices A, B, and C
Let us look at the new monitors (simulation) from Chapter 1 with Bland-Altman plots.
- Device A: the difference slopes downwards. There is proportional error (the larger the value, the lower it reads), which cannot be represented by a single “mean difference”
- Device B: the differences lie horizontally around +10. Constant systematic error
- Device C: no systematic error, but the limits of agreement are wide at ±30 mmHg
2.2.1 Checking for proportional error
Regress the difference on the mean and look at the slope.
A slope of around −0.2 means “when the mean is 10 mmHg higher, the difference is 2 mmHg smaller”. When there is proportional error, one approach is to show the limits of agreement as lines that vary with the value: regression line ± 1.96 × SD of the residuals.
2.3 The limits of agreement also have error
The limits of agreement are values estimated from a sample. With few people, the limits of agreement themselves fluctuate a lot. Using the 1st and 2nd blood pressure readings, let us repeat 5 times a study that evaluates only 30 people.
In studies of 30 people, the limits of agreement vary by as much as 2–3 mmHg from study to study. The 95% confidence interval of a limit of agreement is approximately limit of agreement ± 1.96 × √(3 × SD² / n).
The sample size of an agreement study is determined based on the width of the confidence intervals of the limits of agreement (e.g., 100 or more people is often recommended).
2.4 Judge “agreement” against criteria decided in advance
A Bland-Altman plot only shows the size of the discrepancies; it does not tell you whether the methods “agree”. Decide the clinically acceptable discrepancy before looking at the data, and compare with it.
For blood pressure monitors, the international standard (the joint AAMI/ESH/ISO standard) requires, for the difference from the reference method (auscultation), roughly a mean difference within ±5 mmHg and an SD of the differences within 8 mmHg.
The actual standard specifies in detail how subjects are selected (range of blood pressure, arm circumference), the number and training of observers doing the auscultation, the order of measurements, and so on. Here we show only the idea.
2.5 When the spread grows with the value
In measurements where the larger the value, the larger the error, such as blood concentrations, the differences fan out like a trumpet. In this case, take logarithms and evaluate ratios.
Back-transforming the log differences gives limits of agreement for the ratio “new / conventional”. In this example, the new measurement falls within about 0.82 to 1.20 times the conventional one.
- Bland-Altman plot: mean of the two on the horizontal axis, difference on the vertical axis. Draw the mean difference (systematic error) and the 95% limits of agreement
- Check by eye for a slope (proportional error) or widening (error proportional to the value → logs and ratios)
- The limits of agreement also have error. With few people, also show their confidence intervals
- Judge “agreement” by comparing with a clinically acceptable range decided in advance