9  Internals: statistical transformations, axis ranges, coordinate transforms, drawing objects

This chapter explains, from the internal mechanisms, the places in earlier chapters where you may have wondered “why is it written this way?”. Once you understand this, you will know what to fix when things don’t turn out as you want.

9.1 ggplot2’s statistical transformations: what is computed before drawing

geom_histogram(), geom_boxplot() and geom_smooth() don’t draw the data as is; they draw the result of summarising it with a statistical transformation (stat). With layer_data(), you can extract the values that are actually drawn.

9.1.1 after_stat(): using computed values

Values computed by a statistical transformation (count, density, prop, etc.) can be used in mappings with after_stat().

9.2 The axis-range trap: ylim() vs coord_cartesian()

Suppose you want to narrow the vertical axis of the box plot to 3000–5000 g. ggplot2 has two ways to do this, and they give different results.

ylim() (the same as scale_y_continuous(limits = )) discards the data outside the range first and then applies the statistical transformation. As a result, Gentoo’s median changed from 5000 to 4700. coord_cartesian() only zooms in on the already computed figure, so the median doesn’t change. The warning “Removed n rows containing non-finite values” is a sign that data were discarded. To narrow the range, use coord_cartesian().

NoteIn matplotlib and seaborn

matplotlib’s ax.set_ylim() only changes the display range after drawing, so it is the same as coord_cartesian(). The same mistake happens when you filter the data before drawing (e.g. drawing df[df.y < 5000]).

9.3 matplotlib coordinate transforms: where to put text

Every drawing in matplotlib converts the values you specify into screen positions through a coordinate transform (transform). There are 4 main ones.

Coordinates Specify with (0, 0) and (1, 1) When to use
Data ax.transData (default) The axis values themselves Points, lines, annotations tied to data
Panel ax.transAxes Bottom left / top right of the panel Equations, panel labels, legend positions
Whole figure fig.transFigure Bottom left / top right of the figure Headings and notes for the whole figure
Blended ax.get_xaxis_transform() x in data, y in panel Keeping the label of a vertical line always at the top of the panel

ggplot2 has no mechanism for specifying coordinate transforms directly, but combining x = -Inf/Inf (the panel edges) with hjust/vjust gets you close to panel coordinates (Chapter 6).

9.4 matplotlib drawing objects: changing things after drawing

A matplotlib figure is a tree of objects (Artists): Figure → Axes → lines, points, rectangles, text. Even for figures drawn with seaborn, you can explore this tree and change things one by one afterwards.

TipStacking order (zorder) and overflow (clip_on)
  • zorder=: larger values are drawn in front. Put grid lines behind the points and annotations at the very front
  • clip_on=False: draw beyond the edge of the panel (for symbols and labels placed outside the panel)

9.5 Building a legend yourself

Sometimes you want an element in the legend that isn’t in the data (such as “black dashed line: overall regression line”).

In ggplot2, you create a legend by putting a constant in aes() (deliberately using the style introduced as a mistake in Chapter 2), and set the appearance with scale_xxx_manual(). guides(... override.aes = ) changes only the legend’s keys, while the points in the plot stay faint.

A matplotlib legend is a list of keys (handles) and texts (labels). With Line2D and the like, you can freely create keys that aren’t in the plot.

9.6 seaborn.objects: using a “grammar” in Python

seaborn 0.12 added seaborn.objects, which lets you write plots with the same grammar of graphics approach as ggplot2.

ggplot2 seaborn.objects
ggplot(data, aes(x, y, colour)) so.Plot(data, x=, y=, color=)
+ geom_point() .add(so.Dot())
+ geom_smooth(method = "lm") .add(so.Line(), so.PolyFit(1))
position_jitter / position_dodge so.Jitter() / so.Dodge()
stat_summary() so.Agg(), so.Est()
facet_wrap(~ v) .facet(col=)
labs() .label()

It doesn’t yet have as many features as ggplot2, but the advantage is that you can write both languages with the same way of thinking. If you want to use R’s grammar as is in Python, there is also plotnine, a near-direct port of ggplot2 (it is not available in the browser Python on this page).

9.7 Turning it into a function for reuse

If you make figures of the same form many times, turn them into a function. In ggplot2, an easy-to-understand approach is to take column names as strings and use them with .data[[ ]].

If matplotlib functions take ax= and return ax, they can be plugged into any layout (the same design as seaborn’s axes-level functions).

ImportantChapter summary
  • Many ggplot2 geoms draw the result of a statistical transformation. Inspect it with layer_data() and use it with after_stat()
  • ylim() discards data before computing. Narrow the range with coord_cartesian() (matplotlib’s set_ylim() only affects the display)
  • In matplotlib, choose among coordinate transforms: data, panel (transAxes), whole figure, blended
  • A matplotlib figure is a tree of Artists. seaborn’s output can also be changed piece by piece afterwards. A legend is a list of handles and labels
  • seaborn.objects is a grammar for Python. When writing functions, use .data[[ ]] in R; in Python, take ax= and return it