2  Review of simple regression

TipYou can run R in your browser in this chapter

Code cells can be edited and re-run with “Run Code” (R is running inside your browser). The first run takes several tens of seconds to load the packages. Feel free to change numbers and see how the results change.

A linear mixed model is “an extension of ordinary regression (the linear model)”. First, let’s check simple regression, which is the foundation.

2.1 The simple regression model

\[ \text{score}_i = \beta_0 + \beta_1 \cdot \text{hours}_i + \varepsilon_i, \qquad \varepsilon_i \sim N(0, \sigma^2) \]

The symbols mean the following.

Symbol Read as Meaning
\(i\) Student index (1, 2, …, about 300)
\(\beta_0\) beta zero Intercept: mean score when study time is 0
\(\beta_1\) beta one Slope: increase in score when study time increases by 1 hour
\(\varepsilon_i\) epsilon Error: individual variation not explained by the line
\(\sigma\) sigma Standard deviation of the error (size of the variation)

\(\varepsilon_i \sim N(0, \sigma^2)\) means “the errors follow a normal distribution with mean 0 and variance \(\sigma^2\), independently for each student”.

2.2 Fitting it in R

What to look at in the output:

  • The Estimate column under Coefficients:
    • (Intercept) = \(\hat\beta_0\) (estimated intercept)
    • hours = \(\hat\beta_1\) (estimated slope)
  • Std. Error: the standard error of the estimate (uncertainty of the estimate)
  • Pr(>|t|): the p-value
  • Residual standard error: \(\hat\sigma\) (estimated standard deviation of the error)
TipAbout the hat symbol \(\hat{}\)

\(\beta_1\) is the “true value (unknown)”, and \(\hat\beta_1\) is the “estimate computed from the data”. This data was generated with a true slope of 3, so check whether \(\hat\beta_1\) is close to 3.

2.3 Check with a plot

2.4 Looking at the residuals…

Let’s line up the residuals (observed − predicted) by school. If the model were adequate, the residuals should scatter around 0 in every school.

In some schools the residuals are positive overall, and in others negative overall. In other words, the “school offset” is mixed into the error \(\varepsilon_i\), and the assumption that “the errors are independent” is broken.

ImportantChapter summary
  • Simple regression is \(y = \beta_0 + \beta_1 x + \varepsilon\), with independent normal \(\varepsilon\)
  • In this data, an “offset for each school” remains in the residuals
  • In the next chapter, we compare three ways of handling this “school offset”