VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D2: Math, Stats & Optimization · Convexity and Lagrange Multipliers — When Local Means Global, and Optimizing Under Constraints
Phase 1/Domain 2/Lesson 10

Convexity and Lagrange Multipliers — When Local Means Global, and Optimizing Under Constraints

15 min read·Foundational· Unrolled Pro
a convex bowl with a single minimum next to a non-convex landscape with several local minima; beside it, contour lines of an objective touching a constraint line at the point where the two gradients are parallel, labelled ∇f = λ∇g.
Diagram— a convex bowl with a single minimum next to a non-convex landscape with several local minima; beside it, contour lines of an objective touching a constraint line at the point where the two gradients are parallel, labelled ∇f = λ∇g.

You already know: gradient descent, curvature and learning rates (Optimization, Jacobians and Taylor); maximum likelihood (Probability); joint distributions and the Gaussian (previous lesson).

Why this matters for Atlas. Convexity is the property that makes optimization trustworthy: for a convex function every local minimum is the global minimum, so linear and logistic regression, SVMs and many classical models have guaranteed solutions. Neural networks aren't convex, which is why their training needs the heuristics of D4. Lagrange multipliers turn constrained problems into unconstrained ones, and they show up in surprising places: the softmax and maximum-entropy distributions, the SVM's support vectors, and the KL-constrained objectives behind RLHF and DPO (D6).


1. Convex sets and functions

The question. When can you trust that the minimum you found is the minimum?

Start from nothing. A function is convex if the line segment between any two points on its graph lies on or above the graph: f(tx + (1−t)y) ≤ t f(x) + (1−t) f(y). Equivalently (for twice-differentiable f) its Hessian is positive semi-definite everywhere: curvature is never negative in any direction. Consequences: any local minimum is global, gradient descent with a small enough step converges to it, and sums of convex functions (plus regularizers like ‖w‖²) stay convex. Squared error of a linear model, logistic loss, and hinge loss are convex in the weights; a neural network's loss isn't, because composing nonlinear layers bends the landscape.

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Convexity and Lagrange Multipliers — When Local Means Global, and Optimizing Under Constraints"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee