VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D2: Math, Stats & Optimization · Jacobians, Matrix Calculus and Taylor Series — Gradients of Whole Layers and Local Approximations
Phase 1/Domain 2/Lesson 6

Jacobians, Matrix Calculus and Taylor Series — Gradients of Whole Layers and Local Approximations

15 min read·Foundational· Unrolled Pro
a curve with its tangent line (first-order Taylor) and osculating parabola (second-order) at a point, showing where each approximation holds; beside it, a layer y = Wx drawn as a box with its Jacobian matrix ∂y/∂x = W, and the backward pass multiplying vector-Jacobian products right to left.
Diagram— a curve with its tangent line (first-order Taylor) and osculating parabola (second-order) at a point, showing where each approximation holds; beside it, a layer y = Wx drawn as a box with its Jacobian matrix ∂y/∂x = W, and the backward pass multiplying vector-Jacobian products right to left.

You already know: derivatives, partial derivatives, gradients and the chain rule (Calculus); matrices as transformations and eigenvalues (Linear algebra, Linear systems).

Why this matters for Atlas. Backpropagation is the chain rule applied to functions whose inputs and outputs are vectors, so each step multiplies by a Jacobian matrix. Frameworks never form these matrices; they compute vector–Jacobian products, which is why backprop costs about the same as the forward pass. Taylor series explain why gradient descent works (locally a function is linear), how far a step can go (curvature), and where second-order methods and learning-rate limits come from. This lesson makes both concrete and checks them numerically.


1. The Jacobian: derivatives of vector functions

The question. What is the derivative of a function that outputs a vector?

Start from nothing. For f: ℝⁿ → ℝᵐ, the Jacobian J is the m×n matrix of partial derivatives Jᵢⱼ = ∂fᵢ/∂xⱼ: row i is the gradient of output i. It's the best linear approximation of f near x: f(x + δ) ≈ f(x) + Jδ. For a linear layer y = Wx, J = W. For an elementwise function like ReLU or tanh, J is diagonal. For softmax, J = diag(p) − ppᵀ. Checking a Jacobian with finite differences is the standard way to verify hand-written derivatives.

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Jacobians, Matrix Calculus and Taylor Series — Gradients of Whole Layers and Local Approximations"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee