VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
Build Atlas

Part 5 of 9 · project Unrolled Pro

Make it helpful

Teach TinyGPT to answer questions with a chat template and a loss mask, then compare full fine-tuning with LoRA on unseen questions.

Before you start

This page builds the code. The ideas behind it are taught in the Library lessons listed at the end; read them first if a step uses something you haven't met. Inspect and copy the complete code in the Atlas Codebase Browser and run each step as you read.

TinyGPT continues text. Ask it a question and it writes something that looks like more lesson. To make it answer, Atlas shows it conversations in a fixed format and trains it only on the assistant's half. This is supervised fine-tuning (SFT), the first step of turning any base model into a chat model. Then the same training runs a second time with LoRA, which updates 2.5% of the weights. On this tiny dataset, the smaller update wins, and the reason is the most useful idea in this part.

a conversation is rendered as user marker, question, assistant marker, answer and end marker; the loss mask is 0 over the prompt and 1 over the answer and end marker. Beside it, a frozen weight matrix W with a small trainable detour B times A.
Diagram— a conversation is rendered as user marker, question, assistant marker, answer and end marker; the loss mask is 0 over the prompt and 1 over the answer and end marker. Beside it, a frozen weight matrix W with a small trainable detour B times A.

Step 1 · A chat template

A chat model sees one string. Special markers say who is speaking and where the answer ends:

Everything up to <|assistant|> is the prompt. At inference time Atlas renders the same prompt and lets the model write until <|end|>. The template must be identical in training and inference; a single changed newline is enough to degrade a fine-tuned model.

That's the preview. Build Atlas Parts 1 and 2 are free; the full walkthrough of Part 5 is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Build Atlas, Part 5: Make it helpful"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee

Go deeper in the Library

The lessons that teach the ideas this part uses, in the order to read them.