Build Atlas · the core course
Code along
Every written part has a runnable checkpoint: a script, tests and the real numbers it produces on a laptop CPU. Part 4 trains a 1M-parameter GPT on the lessons themselves; Part 7 answers questions with citations.
Your progress building Atlas
0 of 9 parts built
Everything later is arrays of numbers. Get fluent with them first.
Build it · 5 steps · 10 min · freeTurn a folder of Markdown lessons into clean sections, question–answer pairs and batches of integers.Before neural networks: what "learning" means, and how not to fool yourself measuring it.
Build it · 5 steps · 10 min · freeA first model that learns: TF-IDF features, softmax regression in NumPy, and an evaluation that can't cheat.Derive backprop, then let PyTorch do it for you, knowing exactly what it does.
Build it · 5 steps · 10 min · ProBuild backpropagation yourself, prove it against finite differences, then write the training loop every later part reuses.Tokens, attention and next-token prediction, then the real thing, end to end.
Build it · 4 steps · 10 min · ProTrain a byte-level BPE tokenizer and a 940K-parameter GPT on the lessons, on a laptop CPU, and read its loss curve honestly.A base model predicts text. Post-training turns it into an assistant.
Build it · 4 steps · 10 min · ProTeach TinyGPT to answer questions with a chat template and a loss mask, then compare full fine-tuning with LoRA on unseen questions.Most of an AI product is what the model sees, and proof that it works.
Build it · 5 steps · 10 min · ProPrompts as code with untrusted text escaped, one interface for every model, and an eval harness that names the cases a change breaks.Models forget and make things up. Retrieval gives them the facts, and evaluation keeps them honest.
Build it · 5 steps · 10 min · ProChunk the lessons by structure, search them with BM25 and dense vectors, fuse the rankings, and measure retrieval apart from answers.From answering to doing: tool use, planning, and guardrails.
Build it · 4 steps · 10 min · ProGive Atlas tools, memory and a loop, then put a gateway between the model and the tools so that a fooled model still can't do damage.Make it fast and affordable for thousands of users.
Build it · 5 steps · 10 min · ProAdd a KV cache to TinyGPT, measure what batching buys, simulate static against continuous batching on real traffic, and turn throughput into a cost per token.Foundations
Dip into these whenever Build Atlas assumes something you haven't met yet.
Electives
Each branches off Build Atlas at the part it builds on.