---
webr:
packages: [dplyr, ggplot2]
resources: [R/setup.R, py/setup.py, data/penguins.csv, data/nhanes_adults.csv, fonts/NotoSansJP-Regular.ttf, fonts/NotoSansJP-Bold.ttf]
cell-options:
autorun: true
pyodide:
packages: [pandas, matplotlib, seaborn, scipy, statsmodels]
cell-options:
autorun: true
---
{{< include ./_extensions/r-wasm/live/_knitr.qmd >}}
# Drawing regression lines and equations {#sec-regression}
```{webr}
#| include: false
source("R/setup.R")
pb <- penguins |> filter(!is.na(bill_length_mm), !is.na(bill_depth_mm))
sp_cols <- c(Adelie = "#E69F00", Chinstrap = "#56B4E9", Gentoo = "#009E73")
```
```{pyodide}
#| include: false
exec(open("py/setup.py").read())
from scipy import stats
import statsmodels.formula.api as smf
pb = penguins.dropna(subset=["bill_length_mm", "bill_depth_mm"]).copy()
sp_cols = {"Adelie": "#E69F00", "Chinstrap": "#56B4E9", "Gentoo": "#009E73"}
```
We make a figure that draws a **regression line** on a scatter plot and writes in the **equation (y = ax + b), the coefficient of determination R² and the p-value**.
We use the relationship between bill **length** and bill **depth**.
## Regression line and confidence band
::: {.panel-tabset group="lang"}
## R (ggplot2)
```{webr}
ggplot(pb, aes(bill_length_mm, bill_depth_mm)) +
geom_point(alpha = 0.5) +
geom_smooth(method = "lm", formula = y ~ x) + # straight line (least squares) and 95% confidence band
labs(x = "Bill length (mm)", y = "Bill depth (mm)")
```
If you specify nothing, `geom_smooth()` draws **a curve such as local regression (loess)**. When you want a straight line, always specify `method = "lm"`.
## Python (seaborn / matplotlib)
```{pyodide}
fig, ax = plt.subplots()
sns.regplot(data=pb, x="bill_length_mm", y="bill_depth_mm", scatter_kws={"alpha": 0.5, "s": 15},
line_kws={"color": "C1"}, seed=1, ax=ax)
ax.set(xlabel="Bill length (mm)", ylabel="Bill depth (mm)")
plt.show()
```
By default, the band of seaborn's `regplot()` is **a 95% confidence interval computed by bootstrapping (random numbers)**. Make it reproducible with `seed=`.
:::
It looks as though longer bills are shallower (sloping down to the right). Let's compute the equation of this line and write it on the figure.
## Computing the equation, R² and p-value
::: {.panel-tabset group="lang"}
## R (ggplot2)
```{webr}
fit <- lm(bill_depth_mm ~ bill_length_mm, data = pb)
s <- summary(fit)
b0 <- coef(fit)[1]; b1 <- coef(fit)[2]
c(intercept = b0, slope = b1, R2 = s$r.squared, p = coef(s)[2, 4])
```
## Python (seaborn / matplotlib)
```{pyodide}
r = stats.linregress(pb["bill_length_mm"], pb["bill_depth_mm"])
print(f"intercept {r.intercept:.4f} slope {r.slope:.5f} R2 {r.rvalue**2:.4f} p {r.pvalue:.2e}")
```
`scipy.stats.linregress()` returns the slope, intercept, correlation coefficient and p-value of a simple linear regression with one variable. For multiple variables, use `statsmodels` (see below).
:::
The result is **y = −0.085x + 20.89, R² = 0.055, p = 1.1 × 10⁻⁵**.
### Turning the equation into a string: watch the sign
When the intercept is negative, a naive `paste` gives "y = 0.2x + -5.3". So we write a function that switches between "+" and "−" depending on the sign. While we're at it, we replace the hyphen (-) with the mathematical minus sign (−, U+2212).
::: {.panel-tabset group="lang"}
## R (ggplot2)
```{webr}
eq_label <- function(fit, digits = 3) {
b <- coef(fit); r2 <- summary(fit)$r.squared
sgn <- ifelse(b[1] < 0, "−", "+") # U+2212 (mathematical minus sign)
lab <- sprintf("y = %.*fx %s %.*f\nR² = %.3f", digits, b[2], sgn, digits, abs(b[1]), r2)
sub("-", "−", lab, fixed = TRUE) # also use the mathematical minus when the slope is negative
}
cat(eq_label(fit))
```
## Python (seaborn / matplotlib)
```{pyodide}
def eq_label(slope, intercept, r2, digits=3):
sgn = "−" if intercept < 0 else "+"
lab = f"y = {slope:.{digits}f}x {sgn} {abs(intercept):.{digits}f}\nR² = {r2:.3f}"
return lab.replace("-", "−", 1) # also use the mathematical minus when the slope is negative
print(eq_label(r.slope, r.intercept, r.rvalue**2))
```
:::
## Writing the equation on the figure: "data coordinates" vs "panel coordinates"
There are two ways of thinking about specifying the position of text.
- **Data coordinates**: specified by axis values, such as x = 55 mm, y = 21 mm. When the data change, the text may stick out or overlap the points
- **Panel coordinates**: specified as a fraction of the plot frame, such as "5% in from the top left of the panel". It can be placed in the same spot for any data
For annotations like equations, placing them in **panel coordinates** is convenient.
::: {.panel-tabset group="lang"}
## R (ggplot2)
```{webr}
ggplot(pb, aes(bill_length_mm, bill_depth_mm)) +
geom_point(alpha = 0.5) +
geom_smooth(method = "lm", formula = y ~ x) +
annotate("text", x = -Inf, y = Inf, label = eq_label(fit), # -Inf, Inf = the top-left corner
hjust = -0.1, vjust = 1.2, size = 4.2) + # shift slightly inward from the corner
labs(x = "Bill length (mm)", y = "Bill depth (mm)")
```
In ggplot2, specifying **`-Inf` / `Inf`** as a position gives the edge of the panel. Use `hjust` and `vjust` (the anchor point of the text; 0 is left/bottom, 1 is right/top) to pull it inward.
## Python (seaborn / matplotlib)
```{pyodide}
fig, ax = plt.subplots()
sns.regplot(data=pb, x="bill_length_mm", y="bill_depth_mm", scatter_kws={"alpha": 0.5, "s": 15}, seed=1, ax=ax)
ax.text(0.03, 0.97, eq_label(r.slope, r.intercept, r.rvalue**2),
transform=ax.transAxes, # "panel coordinates": (0, 0) is bottom left, (1, 1) is top right
ha="left", va="top", fontsize=11,
bbox=dict(boxstyle="round", fc="white", ec="0.7"))
ax.set(xlabel="Bill length (mm)", ylabel="Bill depth (mm)")
plt.show()
```
In matplotlib, specifying `transform=ax.transAxes` makes the position **panel coordinates** (0 to 1). `ha` and `va` are the anchor point of the text. Coordinate transforms are covered in detail in Chapter 9.
:::
## Formatting math: superscripts and italics
To make the "2" in R² a superscript or to italicise variables, use math formatting.
::: {.panel-tabset group="lang"}
## R (ggplot2)
```{webr}
r2 <- summary(fit)$r.squared; pval <- coef(summary(fit))[2, 4]
ex <- floor(log10(pval)); man <- pval / 10^ex # split the p-value into "mantissa × 10 to the exponent"
lab <- sprintf("italic(R)^2 == %.3f*\",\"~~italic(p) == %.1f %%*%% 10^%d", # plotmath syntax
r2, man, ex)
lab
ggplot(pb, aes(bill_length_mm, bill_depth_mm)) +
geom_point(alpha = 0.5) +
geom_smooth(method = "lm", formula = y ~ x) +
annotate("text", x = -Inf, y = Inf, label = lab, parse = TRUE, hjust = -0.1, vjust = 1.5, size = 4.2) +
labs(x = "Bill length (mm)", y = "Bill depth (mm)")
```
With `parse = TRUE`, the label is interpreted as R's **plotmath** (its syntax for math). `^` makes a superscript, `italic()` italics, `==` an equals sign, `%*%` ×, `~` a space, and `*` concatenates.
## Python (seaborn / matplotlib)
```{pyodide}
fig, ax = plt.subplots()
sns.regplot(data=pb, x="bill_length_mm", y="bill_depth_mm", scatter_kws={"alpha": 0.5, "s": 15}, seed=1, ax=ax)
ex = int(np.floor(np.log10(r.pvalue))) # split the p-value into "mantissa × 10 to the exponent"
man = r.pvalue / 10**ex
ax.text(0.03, 0.97, rf"$R^2 = {r.rvalue**2:.3f}$, $p = {man:.1f} \times 10^{{{ex}}}$",
transform=ax.transAxes, ha="left", va="top", fontsize=12)
ax.set(xlabel="Bill length (mm)", ylabel="Bill depth (mm)")
plt.show()
```
In matplotlib, enclosing text in `$...$` makes it **mathtext** (a LaTeX-like syntax for math). `^` makes a superscript, `\times` gives ×, and `{}` delimits a group.
Japanese text can't be used inside `$...$`, so write any Japanese outside the `$`.
:::
## Drawing a line and equation per group
Now let's colour by species.
::: {.panel-tabset group="lang"}
## R (ggplot2)
```{webr}
eqs <- pb |> group_by(species) |>
group_modify(~ {
f <- lm(bill_depth_mm ~ bill_length_mm, data = .x)
tibble(slope = coef(f)[2], intercept = coef(f)[1], R2 = summary(f)$r.squared,
label = sprintf("%s: y = %.2fx + %.2f, R² = %.2f", .y$species, coef(f)[2], coef(f)[1], summary(f)$r.squared))
}) |> ungroup() |>
mutate(row = row_number())
eqs
ggplot(pb, aes(bill_length_mm, bill_depth_mm, colour = species)) +
geom_point(alpha = 0.5) +
geom_smooth(method = "lm", formula = y ~ x, se = FALSE) +
geom_smooth(aes(group = 1), method = "lm", formula = y ~ x, se = FALSE, colour = "black", linetype = 2) + # overall line
geom_text(data = eqs, aes(x = -Inf, y = Inf, label = label, vjust = 1.3 + 1.5 * (row - 1)),
hjust = -0.05, size = 3.6, show.legend = FALSE) +
scale_colour_manual(values = sp_cols) +
scale_y_continuous(expand = expansion(mult = c(0.05, 0.35))) + # add 35% headroom at the top for the equations
labs(x = "Bill length (mm)", y = "Bill depth (mm)", colour = "Species", caption = "Black dashed line: regression line for all 3 species combined")
```
To **offset the equations one line at a time**, `vjust` is increased with the row number (`vjust` is an offset in units of text height). To keep the equations off the points, `expansion()` adds headroom at the top of the vertical axis.
## Python (seaborn / matplotlib)
```{pyodide}
fig, ax = plt.subplots(figsize=(7.5, 4.5))
for i, (sp, d) in enumerate(pb.groupby("species")):
rr = stats.linregress(d["bill_length_mm"], d["bill_depth_mm"])
sns.regplot(data=d, x="bill_length_mm", y="bill_depth_mm", ci=None, color=sp_cols[sp],
scatter_kws={"alpha": 0.5, "s": 15}, label=sp, ax=ax)
ax.text(0.02, 0.97 - 0.07 * i, f"{sp}: y = {rr.slope:.2f}x + {rr.intercept:.2f}, R² = {rr.rvalue**2:.2f}",
transform=ax.transAxes, color=sp_cols[sp], ha="left", va="top", fontsize=10)
sns.regplot(data=pb, x="bill_length_mm", y="bill_depth_mm", ci=None, scatter=False,
line_kws={"color": "black", "ls": "--"}, ax=ax)
lo, hi = pb["bill_depth_mm"].min(), pb["bill_depth_mm"].max()
ax.set_ylim(lo - 0.05 * (hi - lo), hi + 0.35 * (hi - lo)) # add headroom at the top for the equations
ax.set(xlabel="Bill length (mm)", ylabel="Bill depth (mm)")
ax.legend(loc="lower right", title="Species")
plt.show()
```
:::
## Simpson's paradox
The line for all 3 species combined (black dashed) **slopes down** (slope −0.085), yet **within every species it slopes up** (slopes 0.18–0.22).
- Within a species, individuals with longer bills have deeper bills
- But Gentoo is a species with "long, shallow" bills, so when the 3 species are combined it looks like a downward slope
**If you draw a regression line without separating the groups, even the direction of the relationship can be reversed** ([descriptive statistics hands-on](https://lab.nghmst.com/study/en/descriptive_hands_on/) Chapter 1, [confounding hands-on](https://lab.nghmst.com/study/en/confounding_hands_on/)).
If you include species in the model, you can estimate the slope within species.
::: {.panel-tabset group="lang"}
## R (ggplot2)
```{webr}
coef(lm(bill_depth_mm ~ bill_length_mm + species, data = pb))
```
## Python (seaborn / matplotlib)
```{pyodide}
smf.ols("bill_depth_mm ~ bill_length_mm + species", data=pb).fit().params
```
:::
The species-adjusted slope is **0.20**, close to the per-species slopes.
## Confidence bands and prediction bands
The band of `geom_smooth()` and `regplot()` is the **95% confidence interval of the regression line (the mean)**. A **prediction interval**, which shows "where the value of one new penguin is likely to fall", is much wider.
::: {.panel-tabset group="lang"}
## R (ggplot2)
```{webr}
ad <- pb |> filter(species == "Adelie")
f_ad <- lm(bill_depth_mm ~ bill_length_mm, data = ad)
grid <- data.frame(bill_length_mm = seq(min(ad$bill_length_mm), max(ad$bill_length_mm), length.out = 100))
ci <- cbind(grid, predict(f_ad, grid, interval = "confidence"))
pi <- cbind(grid, predict(f_ad, grid, interval = "prediction"))
ggplot(ad, aes(bill_length_mm, bill_depth_mm)) +
geom_ribbon(data = pi, aes(y = fit, ymin = lwr, ymax = upr), fill = "grey85") +
geom_ribbon(data = ci, aes(y = fit, ymin = lwr, ymax = upr), fill = "steelblue", alpha = 0.4) +
geom_line(data = ci, aes(y = fit), colour = "steelblue", linewidth = 1) +
geom_point(alpha = 0.6) +
labs(x = "Bill length (mm)", y = "Bill depth (mm)",
title = "Adelie: blue = 95% CI of the line, grey = 95% prediction interval")
```
## Python (seaborn / matplotlib)
```{pyodide}
ad = pb[pb["species"] == "Adelie"]
m = smf.ols("bill_depth_mm ~ bill_length_mm", data=ad).fit()
grid = pd.DataFrame({"bill_length_mm": np.linspace(ad["bill_length_mm"].min(), ad["bill_length_mm"].max(), 100)})
pr = m.get_prediction(grid).summary_frame(alpha=0.05) # mean_ci_* is the confidence interval, obs_ci_* the prediction interval
fig, ax = plt.subplots()
ax.fill_between(grid["bill_length_mm"], pr["obs_ci_lower"], pr["obs_ci_upper"], color="0.85", label="95% prediction interval")
ax.fill_between(grid["bill_length_mm"], pr["mean_ci_lower"], pr["mean_ci_upper"], color="steelblue", alpha=0.4, label="95% confidence interval")
ax.plot(grid["bill_length_mm"], pr["mean"], color="steelblue", lw=2)
ax.scatter(ad["bill_length_mm"], ad["bill_depth_mm"], s=12, alpha=0.6, color="black")
ax.set(xlabel="Bill length (mm)", ylabel="Bill depth (mm)", title="Adelie")
ax.legend(loc="lower right")
plt.show()
```
:::
::: {.callout-important}
## Chapter summary
- Straight lines: `geom_smooth(method = "lm")` in ggplot2 (a curve if not specified), `regplot()` in seaborn (the band is bootstrapped)
- **Compute the equation yourself and turn it into a string**. Watch the sign of a negative intercept
- Place annotations in **panel coordinates** (ggplot2: `-Inf`/`Inf` with `hjust`/`vjust`; matplotlib: `transform=ax.transAxes`)
- Superscripts and italics: R's plotmath (`parse = TRUE`), matplotlib's mathtext (`$...$`)
- Per-group lines and the overall line **can point in opposite directions** (Simpson's paradox)
- The band is the **confidence interval of the line**. The range of individual values is the **prediction interval**
:::