Domain 4: Deep Learning & Neural Networks
Neural networks from scratch: backprop and autograd, debugging training, CNNs and ResNets, the major architectures, and engineering with PyTorch.
Unlock all 6 lessons with Unrolled Pro
Full production code, quantitative capacity math, failure recovery playbooks, and interview coaching.
Lessons
6 lessonsNeural Network Fundamentals — Layers That Learn Their Own Features
Every LLM is a very deep neural network. Its building blocks — linear layers, activations, normalization, residual connections, careful initialization — are all on this page.
Autograd From Scratch, and Debugging Training — Gradient Checks, Loss Curves and the Usual Bugs
PyTorch's autograd feels like magic until something goes wrong.
Major Architectures — Matching the Network to the Shape of the Data
An MLP treats its input as a flat list and ignores structure. But images are grids (nearby pixels matter), events are sequences (order matters), knowledge is a graph (connections matter).
CNNs From Scratch and ResNets — Convolution, Pooling, Residuals and Modern CNN Design
Writing a convolution's forward *and backward* pass once makes everything about CNNs concrete: why weight sharing gives translation equivariance and few parameters, how receptive fields grow, what the backward pass costs.
PyTorch Engineering — From Tensors to Fast, Correct Training Loops
The difference between a model that trains in a day and one that takes a week — or silently trains wrong — is usually engineering: unnecessary copies, a broken autograd graph, a data loader starving the GPU, memory blown on activations, or a checkpoint that can't be loaded safely.
Bridge: From Learning Numbers to Learning Language
Atlas can train deep networks efficiently — but its input is still numbers we prepared. Students type text.