You already know: the chain rule and backpropagation (Calculus); Jacobians and vector–Jacobian products (Jacobians and Taylor); layers, activations, losses and initialization (Neural network fundamentals).
Why this matters for Atlas. PyTorch's autograd feels like magic until something goes wrong. Building a small version yourself (about 60 lines) shows exactly what happens: the forward pass records a graph, and the backward pass walks it in reverse, accumulating gradients. That understanding makes the debugging habits obvious: check gradients numerically, overfit a single batch, read loss curves, and know the dozen bugs that cause most failed runs. This lesson builds the engine, trains a network with it, and then walks through the debugging checklist, ending with the "write a training loop and attention from memory" exercise interviewers love.
1. A tiny autograd engine
The question. How does loss.backward() know what to do?
Start from nothing. Every operation creates a node that remembers its inputs and how to compute its local gradient. During backward(), we order the graph so each node comes after everything that depends on it (reverse topological order), seed the output's gradient with 1, and let each node push gradient to its inputs using the chain rule. Gradients accumulate (+=), because a value used in several places receives gradient from each use; that's also why real frameworks need you to zero gradients between steps.