2  Complete-case analysis

R’s lm() and glm() silently drop rows with a missing value in any of the variables used, and then analyse. This is called complete-case analysis (listwise deletion). It is the easiest approach, but when is it biased, and when is it not?

2.1 Rows silently disappear

Using adults (which contains actual missing values), we regress systolic blood pressure on age, BMI, sex, income, and cholesterol.

Of 10990 people, 8662 were used in the analysis. The only sign is a small line at the end of summary(fit) saying “2328 observations deleted due to missingness”; there is no warning. Make it a habit to always check and report the number of people used in the analysis.

2.2 Means are easily biased

As seen in Chapter 1, the mean of only those observed is biased under MAR and MNAR. Here we repeat 500 times an experiment in which 2000 people are drawn from the same population and 30% of blood pressure values are made missing, to see how far off things are on average (new_sample() is a function that creates a new sample by the same mechanism as full).

The mean blood pressure is about 2.6 mmHg too high under MAR and about 5.9 mmHg too low under MNAR.

2.3 Regression coefficients are sometimes unbiased

Regression coefficients, however, behave differently.

  • MAR (missing by age): the coefficients for age, BMI, and sex are almost the same as the true values
  • MNAR (missing by blood pressure): all coefficients shrink (the age coefficient goes from 0.46 to 0.37)

Under MAR, whether a value is missing is determined by age, and age is in the model. Among people of the same age, the distribution of blood pressure is the same for those observed and those missing, so the regression coefficients are not biased.

The mean blood pressure lumps everyone together without regard to age, so it is biased when young people drop out.

2.4 When an explanatory variable is missing

This time, instead of blood pressure, we make BMI (an explanatory variable) 30% missing.

  • If missingness in BMI is determined by age (an explanatory variable), the coefficients are not biased
  • If missingness in BMI is determined by blood pressure (the outcome), the coefficients are biased
ImportantWhen complete-case regression coefficients are unbiased

If whether a value is missing is determined only by the model’s explanatory variables and does not depend on the outcome, the regression coefficients from complete-case analysis are unbiased (provided the model is correct). This is a broader condition than MCAR, and it does not coincide with MAR either.

Missingness determined by Mean Regression coefficients
Nothing (MCAR) Unbiased Unbiased
The model’s explanatory variables Biased Unbiased
The outcome (even via other observed variables) Biased Biased

Conversely, if missingness is determined by a variable related to the outcome (e.g. home blood pressure) and that variable is not in the model, complete-case analysis is biased even under MAR. With multiple imputation, such variables can be used by putting them only in the imputation model (Chapter 6).

2.5 Even without bias, precision drops

Even when there is no bias under MCAR, the sample shrinks, so the standard errors become larger.

Losing 30% makes the standard error of the age coefficient about 1.2 times larger (≈ \(\sqrt{1/0.7}\)). The more variables have missing values, the more people are lost, because anyone missing at least one is dropped (21% in adults).

2.6 When complete-case analysis is acceptable

TipRules of thumb
  • Missingness is small (e.g. under 5%), and the reasons for missingness suggest little bias
  • Missingness is thought not to depend on the outcome (determined only by explanatory variables)
  • In a randomized trial, baseline covariates are missing (they are often missing for reasons unrelated to the outcome)

In every case, report the numbers used and dropped and the characteristics of those dropped, and add a sensitivity analysis comparing with results from multiple imputation or similar.

2.7 Try it

NotePractice
  1. In adults, check the number of people used by lm(BPSysAve ~ Age + Gender, data = adults). How does the number change when you use fewer explanatory variables?
  2. When missingness in blood pressure is determined by sex (45% for men, 15% for women), what happens to the mean blood pressure and the regression coefficients?
  1. 10164 people. Only those missing blood pressure are dropped, so this is more than when five explanatory variables are used (8662)
  2. The mean blood pressure is slightly lower (because men, with higher blood pressure, are more likely to be missing). The regression coefficients are almost unbiased, because sex is in the model
ImportantChapter summary
  • lm() silently drops rows with missing values. Check and report the number used in the analysis
  • Means (overall summaries) are biased unless the data are MCAR
  • Regression coefficients are unbiased if missingness is determined only by explanatory variables, and biased if it is determined by the outcome
  • Even without bias, the smaller sample reduces precision