VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D2: Math, Stats & Optimization · SVD and PCA — Every Matrix Is a Rotation, a Stretch and a Rotation
Phase 1/Domain 2/Lesson 4

SVD and PCA — Every Matrix Is a Rotation, a Stretch and a Rotation

15 min read·Foundational· Unrolled Pro
a unit circle mapped by a matrix into an ellipse, decomposed into Vᵀ (rotate), Σ (stretch along axes by singular values) and U (rotate); beside it, a scree plot of singular values falling off quickly, with "keep top k" marked, and a cloud of points with its two principal axes.
Diagram— a unit circle mapped by a matrix into an ellipse, decomposed into Vᵀ (rotate), Σ (stretch along axes by singular values) and U (rotate); beside it, a scree plot of singular values falling off quickly, with "keep top k" marked, and a cloud of points with its two principal axes.

You already know: eigenvectors of symmetric matrices, rank and conditioning (Linear systems and eigenvectors); covariance (Probability).

Why this matters for Atlas. The singular value decomposition factors any matrix into rotations and a stretch, and sorts its "directions of action" by importance. Truncating it gives the best low-rank approximation of a matrix, the mathematical basis of PCA (dimensionality reduction), LoRA (low-rank fine-tuning updates, D6), embedding compression, recommendation by matrix factorization, and many interpretability analyses. This lesson builds SVD and PCA and measures what truncation keeps.


1. The decomposition

The question. What does a matrix do to space, in the simplest possible terms?

Start from nothing. Any m×n matrix A factors as A = UΣVᵀ, where V (n×n) and U (m×m) have orthonormal columns (rotations/reflections) and Σ is diagonal with non-negative singular values σ₁ ≥ σ₂ ≥ … ≥ 0. Applying A means: rotate the input into V's basis, stretch axis i by σᵢ, rotate into U's basis. The number of non-zero singular values is the rank; σ₁/σ_min is the condition number; and the singular values are the square roots of the eigenvalues of AᵀA, linking SVD to the previous lesson.

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "SVD and PCA — Every Matrix Is a Rotation, a Stretch and a Rotation"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee