You already know: decision trees and bagging/random forests (Supervised learning); gradient descent (Optimization); loss functions and overfitting (Supervised learning, ML evaluation).
Why this matters for Atlas. Random forests average many independent deep trees to reduce variance. Boosting does the opposite: it builds small trees sequentially, each one correcting the errors of the ensemble so far, reducing bias. Framed as gradient descent in function space, it works with any differentiable loss, which is why one library handles regression, classification, ranking and survival models. This lesson builds AdaBoost and gradient boosting from scratch, explains what XGBoost, LightGBM and CatBoost add, and finishes with stacking.
1. AdaBoost: reweight the mistakes
The question. Can many weak classifiers make a strong one?
Start from nothing. Start with equal weights on all training examples. Fit a weak learner (a decision stump: one split on one feature), compute its weighted error ε, give it a vote α = ½ ln((1 − ε)/ε), then increase the weights of misclassified examples so the next stump focuses on them. The final classifier is the sign of the weighted vote. AdaBoost turns stumps barely better than chance into a strong classifier; it's equivalent to minimizing an exponential loss, which also makes it sensitive to label noise (mislabelled points get ever larger weights).