You already know: transformers, the residual stream and attention heads (D5); linear algebra, projections and PCA (D2); training small networks from scratch (D4); experimental controls and ablations (Research workflow).
Why this matters for Atlas. Evaluations say whether a model fails; interpretability asks why, by studying its internal activations. Three tools cover most practical work. Probes test whether information is linearly readable from a layer. Activation patching tests whether a component causally matters for a behaviour by transplanting its activation between runs. Sparse autoencoders (SAEs) unpack dense activations into many interpretable features, working around superposition (models packing more features than they have dimensions). This lesson builds each in NumPy on small networks where the ground truth is known, including the controls that keep conclusions honest.
1. Probes, and the control that makes them meaningful
The question. Does a layer "know" some property of the input?
Start from nothing. A probe is a simple classifier (usually linear) trained to predict a property from a layer's activations. High probe accuracy shows the information is linearly decodable there. It does not show the model uses it: a wide enough layer re-encodes much of its input whether trained or not. So every probe needs a control: the same probe on an untrained (random-weight) network, or on a control task with random labels (Hewitt & Liang). The interesting signal is what the trained model represents beyond the control.