VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D3: Classical ML · Three Classic Classifiers — k-Nearest Neighbours, Naive Bayes and Support Vector Machines
Phase 2/Domain 3/Lesson 2

Three Classic Classifiers — k-Nearest Neighbours, Naive Bayes and Support Vector Machines

10 min read·Intermediate· Unrolled Pro
three decision boundaries on the same 2-D data: kNN's jagged boundary following local neighbourhoods, Naive Bayes' smooth boundary from per-class distributions, and an SVM's maximum-margin boundary with the support vectors circled; below, a kernel lifting non-separable 1-D points onto a parabola where a line separates them.
Diagram— three decision boundaries on the same 2-D data: kNN's jagged boundary following local neighbourhoods, Naive Bayes' smooth boundary from per-class distributions, and an SVM's maximum-margin boundary with the support vectors circled; below, a kernel lifting non-separable 1-D points onto a parabola where a line separates them.

You already know: linear and logistic regression, overfitting and regularization (Supervised learning); Bayes' rule (Probability); Lagrange multipliers and convexity (Convexity and Lagrange); distances and dot products (Linear algebra).

Why this matters for Atlas. These three methods are built on three different ideas: similarity (kNN: you're like your neighbours), probability (Naive Bayes: which class most likely generated these features?) and geometry (SVM: the widest possible gap between classes). Each idea survives in modern systems: nearest-neighbour search powers RAG and recommendations, Naive Bayes-style counting powers fast text filters, and margins and kernels underlie contrastive learning and attention. This lesson builds each from scratch, shows where it shines and fails, and compares them on the same problems.


1. k-nearest neighbours: predict like your neighbours

The question. What's the simplest classifier that can learn any boundary?

Start from nothing. Store the training set. To classify a new point, find its k nearest training points (by some distance) and take a majority vote (or average, for regression). There's no training step; all the cost is at prediction time, which is why large-scale kNN relies on approximate nearest-neighbour indexes (D8). k controls the bias–variance trade-off: k = 1 memorizes noise, large k smooths over real structure. Features must be on comparable scales, and in high dimensions distances concentrate (everything is about equally far away), the curse of dimensionality, unless the data lives on a lower-dimensional structure.

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Three Classic Classifiers — k-Nearest Neighbours, Naive Bayes and Support Vector Machines"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee