3 Pitfalls of single imputation
To avoid losing people in complete-case analysis, missing values are sometimes filled in with some value (imputed) before analysis. Methods that fill in a single value are called single imputation. They are convenient, but each has pitfalls.
In this chapter we make 40% of blood pressure values in full missing, with a mechanism in which younger people are more likely to be missing (MAR).
3.1 Filling in the mean
Fill in missing blood pressure with the mean of those observed.
3.2 Filling in regression predictions
Fit a regression predicting blood pressure from age, BMI, and sex to those observed, and fill in its predicted values.
3.3 Filling in predictions plus error
Fill in the regression predictions plus a random error with the residual standard deviation (stochastic regression imputation).
3.4 Comparing
| Method | Mean | Standard deviation | Correlation with age |
|---|---|---|---|
| Mean imputation | Same as complete case (biased under MAR) | Too small (the same value repeated) | Too weak |
| Regression imputation | Small bias | Too small (all on the regression line) | Too strong |
| Stochastic regression imputation | Small bias | About right | About right |
- Mean imputation distorts the distribution and weakens relationships under any missing-data mechanism. Do not use it
- Regression imputation puts every imputed value on the line, making relationships look stronger than they really are
- Stochastic regression imputation reproduces the distribution and relationships almost correctly. So is this enough?
3.5 What even stochastic regression imputation lacks
We repeat 500 times an experiment in which 500 people are drawn from the population, 40% of blood pressure values are made missing, and the mean blood pressure and its 95% confidence interval are computed with each method. If the proportion of confidence intervals that contain the true mean (coverage) is 95%, the confidence intervals are being computed correctly.
- Complete case and mean imputation are biased by about 3.5 mmHg in the mean, with coverage of 7% and under 1%
- Regression imputation and stochastic regression imputation have almost no bias. But their coverage is 75% and 79%, falling short of 95%
This is because the “computed standard error” (about 0.66–0.80) is too small compared with “the actual spread of the estimates” (about 1.1–1.2).
If imputed values are treated just like values that were really measured, then
- the fact that the imputed values are uncertain (the same person might have received a different value), and
- the fact that the regression coefficients used for imputation are themselves uncertain values estimated from the sample
are ignored, and the data appear to contain more information than they actually do.
- Mean imputation: distorts the distribution and weakens relationships. Do not use it
- Regression imputation: makes relationships look stronger and spread look smaller
- Stochastic regression imputation: little bias in the estimates, but standard errors are too small, confidence intervals too narrow (p-values too small)
Reflecting this “uncertainty of imputation” in the analysis is what multiple imputation does (Chapter 4).
3.6 Other single-imputation methods
- LOCF (last observation carried forward): in longitudinal studies, the last value of someone who dropped out is used for all later time points. It assumes that “values do not change after dropout”, which rarely holds. Standard errors are also too small
- Missing-indicator method: fill missing explanatory variables with 0 or similar and add a variable for “missing or not” to the model. Apart from baseline covariates in randomized trials, it is known to be biased
- Filling in worst or best values: meaningful as a sensitivity analysis, but unsuitable for the primary analysis
3.7 Try it
- If the proportion missing is changed from 40% to 20%, how does the coverage of stochastic regression imputation change? (Change
0.4to0.2in the experiment above) - Using the scatter plot, explain why the correlation between age and blood pressure is too strong with regression imputation
- About 88%. The less missingness, the closer it gets to 95%, but it does not reach it (the uncertainty of imputation is still ignored)
- All imputed values lie on the regression line (the predictions from age, BMI, and sex), without the “scatter around the line” that real blood pressures have. As a result, age and blood pressure look more lined up than they really are
- Mean imputation distorts distributions and relationships
- Regression imputation makes relationships look stronger and spread smaller
- Stochastic regression imputation has little bias, but because it ignores the uncertainty of imputation, its standard errors are too small
- Do not treat singly imputed values as if they were measured values