You already know: convolutions and CNNs at the concept level (Major architectures); autograd and gradient checking (Autograd from scratch); normalization and initialization (Neural network fundamentals).
Why this matters for Atlas. Writing a convolution's forward and backward pass once makes everything about CNNs concrete: why weight sharing gives translation equivariance and few parameters, how receptive fields grow, what the backward pass costs. And the most influential idea in deep learning architecture, the residual connection, is easiest to understand on a deep CNN: without it, very deep networks train worse than shallow ones. This lesson builds conv and pooling layers in NumPy with verified gradients, trains a small CNN, measures the degradation problem and its fix, and surveys modern CNN design.
1. Convolution forward and backward
The question. What exactly does a conv layer compute, and what are its gradients?
Start from nothing. A conv layer slides each of C_out kernels (size C_in × k × k) over the input and computes a dot product at every position (plus a bias). Output size for input H, kernel k, padding p, stride s: ⌊(H + 2p − k)/s⌋ + 1. The backward pass follows from the chain rule: the kernel gradient is the correlation of the input with the output gradient; the input gradient is the output gradient "convolved back" through the flipped kernel (each input pixel receives gradient from every output that used it). Implementations unroll patches into a matrix (im2col) so convolution becomes one big matrix multiply, which is what GPUs do well.