Domain 6: LLM Post-training, Fine-tuning & Alignment
Post-training LLMs: supervised fine-tuning, LoRA and QLoRA, preference optimization (DPO), reinforcement learning from basics to PPO and GRPO, synthetic data.
Unlock all 10 lessons with Unrolled Pro
Full production code, quantitative capacity math, failure recovery playbooks, and interview coaching.
Lessons
10 lessonsSupervised Fine-Tuning — Teaching a Base Model to Answer
Supervised fine-tuning (SFT) is the step that turns a pretrained base model into something that answers questions.
Parameter-Efficient Fine-Tuning — LoRA, QLoRA and Adapters
Full fine-tuning updates every weight, so it needs memory for weights, gradients and optimizer state for all 8 billion parameters, and it produces a full new copy of the model.
Preference Optimization — Reward Models, RLHF and DPO
SFT teaches the model to imitate demonstrations, so it can never be better than the demonstrations. And for many qualities — clarity, tone, safety, "is this actually helpful?
Reinforcement Learning I — Bandits, Exploration, MDPs and the Bellman Equation
Supervised learning is told the right answer; reinforcement learning only gets rewards for what it tried, so it must explore to discover good actions and exploit what it knows, and in multi-step problems it must assign credit for a reward to the decisions that led to it.
Reinforcement Learning II — Monte Carlo, TD Learning, Q-Learning and DQN
Value iteration needs the model of the world.
Reinforcement Learning III — Policy Gradients, Baselines, Actor-Critic, GAE and PPO From Scratch
Value-based methods (Q-learning) pick actions by maximizing estimated values, which is awkward for huge or continuous action spaces like "the next token".
Reinforcement Learning for LLMs — Policy Gradients, PPO and GRPO
SFT imitates and DPO learns from fixed comparisons.
RL in Practice — Environments, Verifiers, Reward Hacking, Reading RL Curves, and a GRPO Capstone
Reinforcement learning optimizes whatever the reward says, not what you meant. With verifiable rewards (maths answers, unit tests, environment states), that's powerful: no human labels, unlimited practice.
Synthetic Data — Distillation, Self-Instruct and Rejection Sampling
Most post-training data in modern models is synthetic: generated by models, then filtered.
Bridge: From Changing the Weights to Changing the Input
Atlas is now an assistant: fine-tuned, preference-tuned and trained with verifiable rewards. But every new feature request can't mean another training run.