4 Describe missingness — Whose data are missing?
In large datasets, almost every variable has missing values. Most analysis software silently excludes anyone with a missing value in any of the variables entered in the model (complete-case analysis). As a result, you may be analysing a different population from the one you first had in mind.
In this chapter we mainly use the 11778 NHANES participants aged 20 or over (adults, already loaded).
4.1 Count first: missingness by variable
More than half of testosterone (Testosterone) and current smoking (SmokeNow) are missing. Consider not only the number of missing values but also why they are missing.
4.2 Structural missingness: never measured, never asked
- Testosterone was measured only in 2011–12 (100% missing in 2009–10)
- The depression questions were asked only of those aged 18 or over
The “mean” or “proportion” of these two variables are values only for people from a specific survey year or age range. Writing them up as values for the whole sample would be wrong.
4.2.1 Questions not asked because of the answer to a previous question
Especially easy to miss are cases where a question is skipped depending on the answer to a previous question.
“Do you currently smoke?” (SmokeNow) was asked only of people who had smoked 100 or more cigarettes. All 6536 people who had never smoked are NA. The true meaning of this NA is not “missing” but “does not smoke”.
In Framingham too, HDL and LDL cholesterol were measured only at the third examination.
4.3 Who disappears in complete-case analysis?
Suppose a study decided to examine “the relationship of blood pressure with socio-economic status and lifestyle” using a regression model. These are the variables entered in the model.
Of the 11778 people, how many have all of these variables (i.e., are actually used in the model)? Look at the variable with the most missing values in the table above and make a prediction.
Only 4138 people (35%) remain. And when we compare the characteristics of those who remain …
100% of the complete cases are people with a smoking history. “People who have never smoked” have NA for SmokeNow, so they were all excluded. The proportion of women also falls from 51% to 41% (because women are less likely to have a smoking history).
The results of this model are not about “US adults” but about “US adults with a smoking history”. And no error or warning is issued.
Let us convert the structural missing values into their true values and count again.
9202 people (78%) remain, and their characteristics are now close to the whole.
4.4 Describe how missingness arises
Are the remaining missing values occurring at random? Compare the characteristics of people with and without missing values.
People who did not report income are more often Hispanic or less educated. If you drop people with missing income, you end up analysing a population with fewer socio-economically disadvantaged people.
People with missing blood pressure are often also missing cholesterol, BMI, and the self-rated health question. In NHANES, the home interview is followed by physical measurements, blood sampling, and additional questions at a mobile examination centre. People who did not come to the examination centre are missing all of these together. Missingness is often not scattered variable by variable but occurs in blocks that follow the survey procedure.
- MCAR (missing completely at random): missingness occurs with the same probability for everyone. Excluding does not introduce bias (but this is rare in practice)
- MAR (missing at random — explainable by observed variables): e.g., Hispanic people are less likely to report income. Can be corrected by methods such as multiple imputation
- MNAR (missing not at random — related to the missing value itself): e.g., people with lower income are less likely to report income. Cannot be verified from the data alone
The material for thinking about which type applies is the comparison of people with and without missing values.
4.5 Report: how many were excluded, and why
Show, step by step, the number of people used in the analysis and the reasons for exclusion (papers often present this as a flow diagram).
- Count the number and proportion missing for each variable, and consider why they are missing (not measured, not asked, not answered)
- Structural missingness (survey year, age, skip patterns) should be converted to the “true value”, or the target population should be stated explicitly
- Check who was excluded in complete-case analysis by comparing people with and without missing values
- Report the number of exclusions and the reasons step by step