VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D4: Deep Learning · CNNs From Scratch and ResNets — Convolution, Pooling, Residuals and Modern CNN Design
Phase 3/Domain 4/Lesson 4

CNNs From Scratch and ResNets — Convolution, Pooling, Residuals and Modern CNN Design

15 min read·Intermediate· Unrolled Pro
a 3×3 kernel sliding over an image with its output feature map; the backward pass routing gradients from each output back to the input patch and kernel; a plain deep stack where the gradient signal fades layer by layer next to a residual stack where the identity path carries it through; and a ResNet bottleneck block.
Diagram— a 3×3 kernel sliding over an image with its output feature map; the backward pass routing gradients from each output back to the input patch and kernel; a plain deep stack where the gradient signal fades layer by layer next to a residual stack where the identity path carries it through; and a ResNet bottleneck block.

You already know: convolutions and CNNs at the concept level (Major architectures); autograd and gradient checking (Autograd from scratch); normalization and initialization (Neural network fundamentals).

Why this matters for Atlas. Writing a convolution's forward and backward pass once makes everything about CNNs concrete: why weight sharing gives translation equivariance and few parameters, how receptive fields grow, what the backward pass costs. And the most influential idea in deep learning architecture, the residual connection, is easiest to understand on a deep CNN: without it, very deep networks train worse than shallow ones. This lesson builds conv and pooling layers in NumPy with verified gradients, trains a small CNN, measures the degradation problem and its fix, and surveys modern CNN design.


1. Convolution forward and backward

The question. What exactly does a conv layer compute, and what are its gradients?

Start from nothing. A conv layer slides each of C_out kernels (size C_in × k × k) over the input and computes a dot product at every position (plus a bias). Output size for input H, kernel k, padding p, stride s: ⌊(H + 2p − k)/s⌋ + 1. The backward pass follows from the chain rule: the kernel gradient is the correlation of the input with the output gradient; the input gradient is the output gradient "convolved back" through the flipped kernel (each input pixel receives gradient from every output that used it). Implementations unroll patches into a matrix (im2col) so convolution becomes one big matrix multiply, which is what GPUs do well.

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "CNNs From Scratch and ResNets — Convolution, Pooling, Residuals and Modern CNN Design"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee