VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D6: Post-training & Alignment
Phase 4: LLM Architecture & Engineering/DOMAIN 6

Domain 6: LLM Post-training, Fine-tuning & Alignment

AdvancedUnrolled Pro

Post-training LLMs: supervised fine-tuning, LoRA and QLoRA, preference optimization (DPO), reinforcement learning from basics to PPO and GRPO, synthetic data.

Estimated study time: about 120 hours
Access: Unrolled Pro
Unrolled Pro Domain

Unlock all 10 lessons with Unrolled Pro

Full production code, quantitative capacity math, failure recovery playbooks, and interview coaching.

Get Pro from ₹167/mo (~$2/mo)14-day refund guarantee

Lessons

10 lessons
01

Supervised Fine-Tuning — Teaching a Base Model to Answer

Pro 25m

Supervised fine-tuning (SFT) is the step that turns a pretrained base model into something that answers questions.

✓ Runnable code✓ Exercise with solutionStart Lesson
02

Parameter-Efficient Fine-Tuning — LoRA, QLoRA and Adapters

Pro 20m

Full fine-tuning updates every weight, so it needs memory for weights, gradients and optimizer state for all 8 billion parameters, and it produces a full new copy of the model.

✓ Runnable code✓ Exercise with solutionStart Lesson
03

Preference Optimization — Reward Models, RLHF and DPO

Pro 20m

SFT teaches the model to imitate demonstrations, so it can never be better than the demonstrations. And for many qualities — clarity, tone, safety, "is this actually helpful?

✓ Runnable code✓ Exercise with solutionStart Lesson
04

Reinforcement Learning I — Bandits, Exploration, MDPs and the Bellman Equation

FREE 10m

Supervised learning is told the right answer; reinforcement learning only gets rewards for what it tried, so it must explore to discover good actions and exploit what it knows, and in multi-step problems it must assign credit for a reward to the decisions that led to it.

✓ Runnable code✓ Exercise with solutionStart Lesson
05

Reinforcement Learning II — Monte Carlo, TD Learning, Q-Learning and DQN

FREE 15m

Value iteration needs the model of the world.

✓ Runnable code✓ Exercise with solutionStart Lesson
06

Reinforcement Learning III — Policy Gradients, Baselines, Actor-Critic, GAE and PPO From Scratch

Pro 15m

Value-based methods (Q-learning) pick actions by maximizing estimated values, which is awkward for huge or continuous action spaces like "the next token".

✓ Runnable code✓ Exercise with solutionStart Lesson
07

Reinforcement Learning for LLMs — Policy Gradients, PPO and GRPO

Pro 20m

SFT imitates and DPO learns from fixed comparisons.

✓ Runnable code✓ Exercise with solutionStart Lesson
08

RL in Practice — Environments, Verifiers, Reward Hacking, Reading RL Curves, and a GRPO Capstone

Pro 15m

Reinforcement learning optimizes whatever the reward says, not what you meant. With verifiable rewards (maths answers, unit tests, environment states), that's powerful: no human labels, unlimited practice.

✓ Runnable code✓ Exercise with solutionStart Lesson
09

Synthetic Data — Distillation, Self-Instruct and Rejection Sampling

Pro 20m

Most post-training data in modern models is synthetic: generated by models, then filtered.

✓ Runnable code✓ Exercise with solutionStart Lesson
10

Bridge: From Changing the Weights to Changing the Input

Pro 10m

Atlas is now an assistant: fine-tuned, preference-tuned and trained with verifiable rewards. But every new feature request can't mean another training run.

✓ Runnable codeStart Lesson
Domain 6: Post-training & Alignment · Unrolled