2 Review of simple regression
Code cells can be edited and re-run with “Run Code” (R is running inside your browser). The first run takes several tens of seconds to load the packages. Feel free to change numbers and see how the results change.
A linear mixed model is “an extension of ordinary regression (the linear model)”. First, let’s check simple regression, which is the foundation.
2.1 The simple regression model
\[ \text{score}_i = \beta_0 + \beta_1 \cdot \text{hours}_i + \varepsilon_i, \qquad \varepsilon_i \sim N(0, \sigma^2) \]
The symbols mean the following.
| Symbol | Read as | Meaning |
|---|---|---|
| \(i\) | Student index (1, 2, …, about 300) | |
| \(\beta_0\) | beta zero | Intercept: mean score when study time is 0 |
| \(\beta_1\) | beta one | Slope: increase in score when study time increases by 1 hour |
| \(\varepsilon_i\) | epsilon | Error: individual variation not explained by the line |
| \(\sigma\) | sigma | Standard deviation of the error (size of the variation) |
\(\varepsilon_i \sim N(0, \sigma^2)\) means “the errors follow a normal distribution with mean 0 and variance \(\sigma^2\), independently for each student”.
2.2 Fitting it in R
What to look at in the output:
- The
Estimatecolumn underCoefficients:(Intercept)= \(\hat\beta_0\) (estimated intercept)hours= \(\hat\beta_1\) (estimated slope)
Std. Error: the standard error of the estimate (uncertainty of the estimate)Pr(>|t|): the p-valueResidual standard error: \(\hat\sigma\) (estimated standard deviation of the error)
\(\beta_1\) is the “true value (unknown)”, and \(\hat\beta_1\) is the “estimate computed from the data”. This data was generated with a true slope of 3, so check whether \(\hat\beta_1\) is close to 3.
2.3 Check with a plot
2.4 Looking at the residuals…
Let’s line up the residuals (observed − predicted) by school. If the model were adequate, the residuals should scatter around 0 in every school.
In some schools the residuals are positive overall, and in others negative overall. In other words, the “school offset” is mixed into the error \(\varepsilon_i\), and the assumption that “the errors are independent” is broken.
- Simple regression is \(y = \beta_0 + \beta_1 x + \varepsilon\), with independent normal \(\varepsilon\)
- In this data, an “offset for each school” remains in the residuals
- In the next chapter, we compare three ways of handling this “school offset”