VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D3: Classical ML · Gaussian Mixtures and EM, t-SNE and UMAP — Soft Clusters and Honest Pictures of High-Dimensional Data
Phase 2/Domain 3/Lesson 5

Gaussian Mixtures and EM, t-SNE and UMAP — Soft Clusters and Honest Pictures of High-Dimensional Data

10 min read·Intermediate· Unrolled Pro
EM alternating between the E-step (each point coloured by its responsibility for each Gaussian) and the M-step (each Gaussian's mean and covariance ellipse re-estimated), converging over iterations; beside it, the same high-dimensional clusters shown with PCA (global geometry preserved, clusters overlapping) and t-SNE (clusters separated, distances between clusters meaningless).
Diagram— EM alternating between the E-step (each point coloured by its responsibility for each Gaussian) and the M-step (each Gaussian's mean and covariance ellipse re-estimated), converging over iterations; beside it, the same high-dimensional clusters shown with PCA (global geometry preserved, clusters overlapping) and t-SNE (clusters separated, distances between clusters meaningless).

You already know: k-means, hierarchical clustering and PCA (Unsupervised learning); the multivariate Gaussian and maximum likelihood (Joint distributions, Probability); SVD (SVD and PCA).

Why this matters for Atlas. k-means gives hard assignments and assumes round clusters of similar size. A Gaussian mixture model (GMM) gives each point a probability of belonging to each cluster, allows elongated clusters of different sizes, and provides a density you can use for anomaly detection. It's fitted by expectation–maximization (EM), an algorithm pattern that reappears in many latent-variable models. And when you need to look at high-dimensional data, t-SNE and UMAP produce the familiar cluster maps, which are useful for exploration but are easy to over-read.


1. Gaussian mixtures and EM

The question. How do you fit clusters when you don't know which point belongs to which?

Start from nothing. A GMM says each point comes from one of K Gaussians: pick component k with probability πₖ, then draw x ~ N(μₖ, Σₖ). If we knew the assignments, fitting would be easy (per-cluster means and covariances); if we knew the parameters, assigning would be easy (Bayes' rule). EM alternates: the E-step computes each point's responsibilities γᵢₖ = p(component k | xᵢ); the M-step re-estimates πₖ, μₖ, Σₖ as responsibility-weighted averages. Each iteration never decreases the likelihood, so EM converges, but only to a local optimum: run several initializations and keep the best likelihood. k-means is the limit of EM with tiny, equal, round covariances and hard assignments.

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Gaussian Mixtures and EM, t-SNE and UMAP — Soft Clusters and Honest Pictures of High-Dimensional Data"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee