VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
DSA for AI/ML · Module 15: The ML coding round

Lesson 15.10 of 12 in this module Unrolled Pro

Batch normalisation forward pass

Implement batch normalisation's forward pass for training and inference: normalise each feature with the batch mean and variance, scale and shift with learned γ and β, and keep running statistics for inference. From loops over features with a one-pass E[x²] − E[x]² variance that loses all precision when values are large, to vectorised two-pass statistics over the right axes (rows for dense layers; batch, height and width for convolutions), the momentum update of running statistics, and the train-versus-inference switch. Then small batches (layer and group norm), folding batch norm into the previous layer for inference, and statistics across devices.

about 10 min 3 Python examples 2 quick checks

Problem. Implement batchnorm_forward(x, gamma, beta, running_mean, running_var, momentum, eps, training):

text
training:   μ = mean of x over the batch,   σ² = biased variance over the batch
            x̂ = (x − μ) / √(σ² + ε)
            y = γ · x̂ + β
            running_mean ← (1 − momentum) · running_mean + momentum · μ
            running_var  ← (1 − momentum) · running_var  + momentum · σ²_unbiased
inference:  x̂ = (x − running_mean) / √(running_var + ε),   y = γ · x̂ + β

for dense inputs x (N × D, statistics per feature over N) and convolutional inputs (N × C × H × W, statistics per channel over N, H and W).

The formula is short. What's tested: reducing over the right axes, computing the variance in a numerically safe way, updating running statistics only in training, using them (not batch statistics) at inference, and knowing why this breaks with tiny batches.

ML context. Batch norm is in most convolutional networks; its sibling layer norm (Change A) is in every transformer. The classic production bug is evaluating a model in training mode: predictions then depend on what else is in the batch. Another is a variance computed as E[x²] − E[x]², which goes negative or loses all digits for features with a large mean.


1. Step 1: Clarify

That's the preview. Modules 00–01 are free; this lesson is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Batch normalisation forward pass"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee