VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D5: LLM Architecture
Phase 4: LLM Architecture & Engineering/DOMAIN 5

Domain 5: LLM Architecture & Engineering

AdvancedUnrolled Pro

How LLMs work: tokenization, attention, RoPE and grouped-query attention, the transformer block, architecture families, pretraining and building a GPT.

Estimated study time: about 140 hours
Access: Unrolled Pro
Unrolled Pro Domain

Unlock all 7 lessons with Unrolled Pro

Full production code, quantitative capacity math, failure recovery playbooks, and interview coaching.

Get Pro from ₹167/mo (~$2/mo)14-day refund guarantee

Lessons

7 lessons
01

NLP Foundations — From Text to Tokens to Predictions

Pro 15m

Every LLM pipeline begins the same way: text → tokens → token IDs → embedding vectors, and every LLM is trained on the same task: predict the next token.

✓ Runnable code✓ Exercise with solutionStart Lesson
02

The Transformer I — Attention, Positions and Heads from First Principles

FREE 25m

Every modern language model, from a 1M-parameter TinyGPT to the frontier, is the same stack: embeddings, then N identical blocks of attention and MLP, then a projection back to the vocabulary.

✓ Runnable code✓ Exercise with solutionStart Lesson
03

The Transformer II — The Block, the Output, the Arithmetic and Production Practice

FREE 30m

Lesson I built attention.

✓ Runnable code✓ Exercise with solutionStart Lesson
04

LLM Architecture Families — Trade-offs, Not Model Names

Pro 15m

Model names change every month; the design choices underneath don't. Each family makes a different trade between understanding vs generating, parameters vs compute per token, and context length vs memory.

✓ Runnable code✓ Exercise with solutionStart Lesson
05

LLM Pretraining — Data, Compute and Scaling Laws

Pro 15m

Pretraining is where a model gets almost all of its knowledge and ability — and almost all of its cost. Three questions dominate: What data? (quality beats quantity, and leakage of test sets ruins evaluations.

✓ Runnable code✓ Exercise with solutionStart Lesson
06

Build an LLM from Scratch

Pro 25m

Every idea in this domain has been shown in pieces. This lesson puts them together into one script that trains a working GPT: data → tokenizer → model → training → checkpoint → sampling → evaluation.

✓ Runnable code✓ Exercise with solutionStart Lesson
07

Bridge: From a Model That Writes to a Model That Helps

Pro 10m

Atlas now has a language model that writes fluent text. Ask it a question, though, and it carries on writing instead of answering.

✓ Runnable codeStart Lesson
Domain 5: LLM Architecture · Unrolled