4 Multiple imputation and Rubin’s rules
In Chapter 3 we saw that stochastic regression imputation has little bias in its estimates but, because it ignores the uncertainty of imputation, its standard errors are too small. Multiple imputation (MI) measures this uncertainty by imputing in many different ways and seeing how much the results change.
4.1 The three steps of multiple imputation
- Impute: create m “complete datasets” in which missing values are filled in by a method that includes randomness
- Analyse: analyse each of the m datasets in the usual way (mean, regression, …)
- Pool: combine the m results with Rubin’s rules
In this chapter we write these three steps by hand, without a package.
4.2 Step 1: impute (properly)
The difference from stochastic regression imputation is that the regression coefficients themselves are also varied a little each time. The regression coefficients are estimates from the sample, so a different sample would have given different values. We represent that uncertainty by redrawing them at random according to the coefficient estimates and their standard errors (a draw from a Bayesian linear regression; the same idea as mice’s norm method).
4.3 Step 2: analyse
In each of the 5 datasets, compute the mean blood pressure and its standard error.
With each imputation the mean differs slightly, from 121.3 to 121.7. This variability is the uncertainty of imputation.
4.4 Step 3: pool with Rubin’s rules
From the m estimates \(\hat{Q}_1, \dots, \hat{Q}_m\) and their variances (squared standard errors) \(U_1, \dots, U_m\):
| Quantity | Formula | Meaning |
|---|---|---|
| Pooled estimate | \(\bar{Q} = \frac{1}{m}\sum \hat{Q}_i\) | Mean of the m estimates |
| Within-imputation variance | \(W = \frac{1}{m}\sum U_i\) | Uncertainty within one dataset (mean of the usual squared standard errors) |
| Between-imputation variance | \(B = \frac{1}{m-1}\sum (\hat{Q}_i - \bar{Q})^2\) | How much the results change with imputation |
| Total variance | \(T = W + \left(1 + \frac{1}{m}\right) B\) | Variance of the pooled estimate |
- The pooled mean is 121.5 mmHg, with a standard error of 0.44
- This is larger than the standard error of a single dataset (about 0.40), because the between-imputation variability B was added
- The confidence interval uses the t distribution (160 degrees of freedom). The degrees of freedom are smaller when m is small and B is large relative to W
The fraction of missing information (λ) is a rough measure of “the proportion of information lost due to missingness”. Here it came out as 0.16, but with m = 5 the estimate of B is very unstable, and this value is not very trustworthy (with m = 50 in Practice 2, it becomes 0.47). How large m should be is covered in Chapter 7.
- The estimate is simply averaged
- The standard error adds “the uncertainty of imputation (B)” on top of “the uncertainty within one dataset (W)”
- \((1 + 1/m)\) is a correction for the number of imputations m being finite
4.5 Checking coverage
Using the same experiment as in Chapter 3 (500 people from the population, 40% of blood pressure missing, 500 repetitions), we check the coverage of confidence intervals from multiple imputation (m = 10). We also compare with improper multiple imputation that does not redraw the regression coefficients (simply repeating stochastic regression imputation 10 times).
| Method | Coverage |
|---|---|
| Complete case (Chapter 3) | 7% |
| Stochastic regression imputation (Chapter 3) | 79% |
| Improper multiple imputation (coefficients not redrawn) | about 90% |
| Proper multiple imputation | about 96% |
With proper multiple imputation, the computed standard error (1.12) almost matches the actual spread of the estimates (1.12), and coverage is around 95%. Without redrawing the coefficients, “the uncertainty of the regression coefficients used for imputation” is left out, so the standard errors are still too small.
- Missingness is MAR (the missing-data mechanism can be explained by the variables in the imputation model)
- The imputation model is appropriate: it contains the variables and relationships used in the analysis (Chapter 6) and reflects the uncertainty of the parameters (proper imputation)
- Pooling is done with Rubin’s rules (not only averaging the estimates but adding B to the variance)
4.6 Try it
- Using
imps, pool the regression coefficient for age (lm(BPSysAve ~ Age + BMI + Gender)) with Rubin’s rules - If m is increased from 5 to 50, how do the pooled standard error and degrees of freedom change?
- The age coefficient is about 0.52, with a standard error of about 0.025. It differs a little from the estimate in
fullbefore missingness (0.454), but this is fluctuation due to the particular combination of people who happened to be missing in the single datasetfull(averaged over 500 repetitions, it is unbiased) - The standard error rose from 0.44 to 0.56, and the fraction of missing information from 0.16 to 0.47. With m = 5, the 5 estimates happened to be close to one another, so B was underestimated. When m is small, the standard error itself fluctuates. How to choose m is covered in Chapter 7
- Multiple imputation has three steps: impute → analyse → pool
- When imputing, not only add error to the predictions but also reflect the uncertainty of the regression coefficients (proper imputation)
- Rubin’s rules: the estimate is the mean; the variance is \(T = W + (1 + 1/m) B\)
- Done correctly, confidence-interval coverage is around 95%