VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D3: Classical ML · Imbalanced Data, Hyperparameter Search and SHAP — Making Tabular Models Work and Explaining Them
Phase 2/Domain 3/Lesson 9

Imbalanced Data, Hyperparameter Search and SHAP — Making Tabular Models Work and Explaining Them

10 min read·Intermediate· Unrolled Pro
a precision–recall curve with the operating threshold chosen by cost; a grid search vs random search over two hyperparameters where only one matters, showing random search tries more distinct values of the important one; a SHAP waterfall for one student, from the base rate to the prediction, with features pushing up and down.
Diagram— a precision–recall curve with the operating threshold chosen by cost; a grid search vs random search over two hyperparameters where only one matters, showing random search tries more distinct values of the important one; a SHAP waterfall for one student, from the base rate to the prediction, with features pushing up and down.

You already know: precision, recall, PR-AUC, calibration and thresholds (ML evaluation); boosted trees and early stopping (Boosting); cross-validation (ML evaluation).

Why this matters for Atlas. Rare positives (churn, fraud, failing students) break naive training and evaluation; hyperparameters decide whether a model is mediocre or excellent, and the search strategy decides how much compute that costs; and explanations decide whether people trust and correctly use a model. Each has a principled approach and a set of traps. This lesson covers resampling vs reweighting vs thresholding, random and Bayesian search with proper validation, and Shapley-value explanations computed from first principles.


1. Imbalanced data: fix the decision, not just the data

The question. With only a few percent positives, how should you train and evaluate?

Start from nothing. Imbalance mainly hurts in three ways: accuracy becomes meaningless (predicting "no" scores 96%), a default 0.5 threshold rarely matches the business trade-off, and small minority classes give noisy estimates. Options: class weights (scale the loss of minority examples), resampling (oversample minorities, e.g. SMOTE, or undersample the majority), and threshold tuning on a calibrated model. For probabilistic models, the cleanest approach is usually: train on the natural distribution (or reweight), calibrate, then choose the threshold from costs (Fraud case study, D22). Resampling changes the predicted probabilities, so if you resample, recalibrate. Evaluate with PR-AUC, recall at a fixed precision, or expected cost, with stratified splits.

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Imbalanced Data, Hyperparameter Search and SHAP — Making Tabular Models Work and Explaining Them"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee