4 Random-intercept models
Code cells can be edited and re-run with “Run Code” (R is running inside your browser). The first run takes several tens of seconds to load the packages. Feel free to change numbers and see how the results change.
4.1 The model equation
For student \(i\) in school \(j\),
\[ \text{score}_{ij} = \underbrace{\beta_0 + \beta_1 \cdot \text{hours}_{ij}}_{\text{fixed effects (common to all)}} + \underbrace{u_j}_{\text{school offset}} + \underbrace{\varepsilon_{ij}}_{\text{individual variation}} \]
\[ u_j \sim N(0, \tau^2), \qquad \varepsilon_{ij} \sim N(0, \sigma^2) \]
The only difference from simple regression is that one term, \(u_j\), has been added.
- \(u_j\): how far the intercept of school \(j\) is from the overall intercept \(\beta_0\)
- \(\tau\) (tau): the standard deviation of the school offsets. The size of the variation between schools
- \(\sigma\): the variation between individuals within a school
So the line for school \(j\) has “intercept \(\beta_0 + u_j\) and slope \(\beta_1\)”. It is a model with as many parallel lines as there are schools.
Rather than estimating each \(u_j\) freely for each school (= no pooling), we assume that “\(u_j\) comes from a normal distribution with mean 0 and standard deviation \(\tau\)”. What we estimate is not 20 values of \(u_j\) but a single value, \(\tau\). This assumption is what produces shrinkage.
4.2 How to write it in R
lmer(score ~ hours + (1 | school), data = d)| Part | Meaning |
|---|---|
score ~ hours |
Fixed effects. Written the same way as in lm(). The intercept is included automatically |
(1 | school) |
Random effect. “The intercept (1) varies by school” |
The left side of | is “what varies”, and the right side is “what it varies by (the group)”.
4.3 Fit the model and read the output
Let’s read it from the top.
4.3.1 Random effects
Groups Name Variance Std.Dev.
school (Intercept) ... ... ← τ (variation between schools)
Residual ... ... ← σ (individual variation)
The data were generated with true values \(\tau = 8\) and \(\sigma = 6\). Check whether the estimates are close.
4.3.2 Fixed effects
(Intercept): \(\hat\beta_0\). The score for an average school when study time is 0hours: \(\hat\beta_1\). The increase in score per hour of study time (true value 3)
The p-values in the Pr(>|t|) column are computed by the lmerTest package, which approximates the degrees of freedom using the Satterthwaite method.
4.4 Extracting the values for each school
fixef(): fixed effects (common to all)ranef(): random effects (offset for each school, \(\hat u_j\))coef(): the sum of the two (intercept and slope for each school)
\(\hat u_j\) is not “estimated” as a parameter; it is a value “predicted” from the estimated \(\tau, \sigma\) and the data (it is called the BLUP: best linear unbiased predictor). This is the shrinkage from Chapter Chapter 3.
4.5 Check with a plot
We get many parallel lines. This is the random-intercept model.
4.6 Intraclass correlation coefficient (ICC)
The proportion of the total variation that is explained by differences between schools is called the intraclass correlation coefficient (ICC).
\[ \text{ICC} = \frac{\tau^2}{\tau^2 + \sigma^2} \]
- ICC ≈ 0: there is almost no difference between schools → almost the same as ordinary regression
- Large ICC: students in the same school are very similar → the mixed model matters
The ICC can also be interpreted as “the correlation between the scores of two students chosen from the same school”.
4.7 Compare with lm
Here hours is a variable that varies within schools, so once the mixed model removes the school offsets, the remaining error becomes smaller, and the standard error actually gets smaller (the estimate is more precise). On the other hand, for a variable determined at the school level, like the “tutoring program” in Chapter Chapter 1, lm underestimated the standard error. Either way, modeling the structure correctly is what matters.
4.8 Practice: create your own data and check
- What happens if you set
sd = 5(τ) to0? (Hint: you may get aboundary (singular) fitwarning → Chapter Chapter 7) - What happens to the estimate of τ if you reduce
n_schoolto5?
- Random-intercept model: \(y_{ij} = \beta_0 + \beta_1 x_{ij} + u_j + \varepsilon_{ij}\), \(u_j \sim N(0,\tau^2)\)
- In R:
lmer(y ~ x + (1 | group)) - Read the output in two parts: Random effects (τ, σ) and Fixed effects (β)
- ICC = τ² / (τ² + σ²) tells you how large the group effect is