VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
DSA for AI/ML · Module 15: The ML coding round

Lesson 15.11 of 12 in this module Unrolled Pro

Linear regression: closed form and gradient descent

Fit y ≈ Xw + b by least squares two ways: in closed form and by gradient descent. From building XᵀX with Python loops and inverting it (O(n · d²) interpreted, and numerically fragile when features are nearly collinear), to a bias column, a least-squares solver based on QR or SVD instead of an explicit inverse, ridge regularisation, and vectorised gradient descent with a step size from the data's scale. Then data too large for memory (accumulate XᵀX in chunks, or mini-batch SGD), very many features, and outliers.

about 10 min 3 Python examples 2 quick checks

Problem. Given X (n × d) and targets y (n), find weights w and bias b minimising the mean squared error

text
MSE(w, b) = mean( (X w + b − y)² )

Implement (1) the closed form and (2) gradient descent, and say when to use which.

Setting the gradient to zero gives the normal equations XᵀX w = Xᵀy (with a column of ones appended for the bias). The textbook answer is w = (XᵀX)⁻¹ Xᵀy. The interview answer is: never form that inverse. Solve the least-squares problem with a factorisation (QR or SVD), which stays accurate when features are nearly collinear, and switch to gradient descent when d or n is too large to factorise.

ML context. Linear regression is the baseline regressor, the last layer of many networks, the solver inside linear probes on frozen embeddings, and the building block of ridge, kernel and Bayesian regression. Its two solution styles, a direct solve vs iterative optimisation, are the same trade-off you face for every model: exact and O(d³), or approximate and O(n · d) per step.


1. Step 1: Clarify

That's the preview. Modules 00–01 are free; this lesson is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Linear regression: closed form and gradient descent"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee