4  Random-intercept models

TipYou can run R in your browser in this chapter

Code cells can be edited and re-run with “Run Code” (R is running inside your browser). The first run takes several tens of seconds to load the packages. Feel free to change numbers and see how the results change.

4.1 The model equation

For student \(i\) in school \(j\),

\[ \text{score}_{ij} = \underbrace{\beta_0 + \beta_1 \cdot \text{hours}_{ij}}_{\text{fixed effects (common to all)}} + \underbrace{u_j}_{\text{school offset}} + \underbrace{\varepsilon_{ij}}_{\text{individual variation}} \]

\[ u_j \sim N(0, \tau^2), \qquad \varepsilon_{ij} \sim N(0, \sigma^2) \]

The only difference from simple regression is that one term, \(u_j\), has been added.

  • \(u_j\): how far the intercept of school \(j\) is from the overall intercept \(\beta_0\)
  • \(\tau\) (tau): the standard deviation of the school offsets. The size of the variation between schools
  • \(\sigma\): the variation between individuals within a school

So the line for school \(j\) has “intercept \(\beta_0 + u_j\) and slope \(\beta_1\)”. It is a model with as many parallel lines as there are schools.

NoteThis is the key point

Rather than estimating each \(u_j\) freely for each school (= no pooling), we assume that “\(u_j\) comes from a normal distribution with mean 0 and standard deviation \(\tau\)”. What we estimate is not 20 values of \(u_j\) but a single value, \(\tau\). This assumption is what produces shrinkage.

4.2 How to write it in R

lmer(score ~ hours + (1 | school), data = d)
Part Meaning
score ~ hours Fixed effects. Written the same way as in lm(). The intercept is included automatically
(1 | school) Random effect. “The intercept (1) varies by school”

The left side of | is “what varies”, and the right side is “what it varies by (the group)”.

4.3 Fit the model and read the output

Let’s read it from the top.

4.3.1 Random effects

 Groups   Name        Variance Std.Dev.
 school   (Intercept)   ...      ...      ← τ (variation between schools)
 Residual               ...      ...      ← σ (individual variation)

The data were generated with true values \(\tau = 8\) and \(\sigma = 6\). Check whether the estimates are close.

4.3.2 Fixed effects

  • (Intercept): \(\hat\beta_0\). The score for an average school when study time is 0
  • hours: \(\hat\beta_1\). The increase in score per hour of study time (true value 3)

The p-values in the Pr(>|t|) column are computed by the lmerTest package, which approximates the degrees of freedom using the Satterthwaite method.

4.4 Extracting the values for each school

  • fixef(): fixed effects (common to all)
  • ranef(): random effects (offset for each school, \(\hat u_j\))
  • coef(): the sum of the two (intercept and slope for each school)
Tip

\(\hat u_j\) is not “estimated” as a parameter; it is a value “predicted” from the estimated \(\tau, \sigma\) and the data (it is called the BLUP: best linear unbiased predictor). This is the shrinkage from Chapter Chapter 3.

4.5 Check with a plot

We get many parallel lines. This is the random-intercept model.

4.6 Intraclass correlation coefficient (ICC)

The proportion of the total variation that is explained by differences between schools is called the intraclass correlation coefficient (ICC).

\[ \text{ICC} = \frac{\tau^2}{\tau^2 + \sigma^2} \]

  • ICC ≈ 0: there is almost no difference between schools → almost the same as ordinary regression
  • Large ICC: students in the same school are very similar → the mixed model matters

The ICC can also be interpreted as “the correlation between the scores of two students chosen from the same school”.

4.7 Compare with lm

Here hours is a variable that varies within schools, so once the mixed model removes the school offsets, the remaining error becomes smaller, and the standard error actually gets smaller (the estimate is more precise). On the other hand, for a variable determined at the school level, like the “tutoring program” in Chapter Chapter 1, lm underestimated the standard error. Either way, modeling the structure correctly is what matters.

4.8 Practice: create your own data and check

CautionTry it
  1. What happens if you set sd = 5 (τ) to 0? (Hint: you may get a boundary (singular) fit warning → Chapter Chapter 7)
  2. What happens to the estimate of τ if you reduce n_school to 5?
ImportantChapter summary
  • Random-intercept model: \(y_{ij} = \beta_0 + \beta_1 x_{ij} + u_j + \varepsilon_{ij}\), \(u_j \sim N(0,\tau^2)\)
  • In R: lmer(y ~ x + (1 | group))
  • Read the output in two parts: Random effects (τ, σ) and Fixed effects (β)
  • ICC = τ² / (τ² + σ²) tells you how large the group effect is