VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D12: Inference Serving & vLLM
Phase 6: GPU & LLM Inference Engineering/DOMAIN 12

Domain 12: LLM Inference Engineering

Staff+ TierUnrolled Pro

LLM inference engineering: prefill and decode, KV cache sizing and paging, batching, quantization, serving frameworks, capacity planning and cost per token.

Estimated study time: about 120 hours
Access: Unrolled Pro
Unrolled Pro Domain

Unlock all 17 lessons with Unrolled Pro

Full production code, quantitative capacity math, failure recovery playbooks, and interview coaching.

Get Pro from ₹167/mo (~$2/mo)14-day refund guarantee

Lessons

17 lessons
01

Inference Fundamentals — Prefill, Decode and the Roofline

Pro 35m

Every LLM product's speed, cost and capacity come from one fact: a response is produced in two very different phases. Prefill processes the whole prompt at once and is limited by arithmetic.

✓ Runnable code✓ Exercise with solutionStart Lesson
02

KV Cache I — What It Stores, Exact Sizing, and Paging

Pro 35m

After the weights, the KV cache is the largest consumer of GPU memory, and unlike the weights it grows with every token of every active request.

✓ Runnable code✓ Exercise with solutionStart Lesson
03

KV Cache II — Prefix Sharing, Preemption, Quantization, Capacity and Operations

Pro 35m

Lesson I made the cache fit.

✓ Runnable code✓ Exercise with solutionStart Lesson
04

KV Cache III — Long Contexts, Attention Sinks and Eviction

Pro 20m

Paging removes wasted KV memory; it cannot shrink what a long context genuinely needs.

✓ Runnable code✓ Exercise with solutionStart Lesson
05

Batching & Scheduling — Continuous Batching, Chunked Prefill and Fairness

Pro 30m

Batching is how one GPU serves many students; scheduling decides *which* requests share each step.

✓ Runnable code✓ Exercise with solutionStart Lesson
06

Quantization — Fewer Bits per Number, Measured Honestly

Pro 30m

Quantization stores weights (and sometimes activations and the KV cache) in fewer bits: 8, 4, even fewer, instead of 16.

✓ Runnable code✓ Exercise with solutionStart Lesson
07

Advanced Inference Optimization — Speculation, Fused Attention, Parallelism and MoE

Pro 25m

Once batching and quantization are in place, the next gains come from changing how much useful work each expensive operation does. Speculative decoding produces several tokens per pass of the big model.

✓ Runnable code✓ Exercise with solutionStart Lesson
08

Inference Frameworks — Inside vLLM, SGLang, TensorRT-LLM and Friends

Pro 30m

You rarely build an inference engine; you choose, configure and operate one.

✓ Runnable code✓ Exercise with solutionStart Lesson
09

Measurement & Capacity — Benchmarks That Don't Lie, and Breaking One GPU on Purpose

Pro 30m

Capacity planning answers one question: how many GPUs do we need to serve this traffic within these latency targets, at what cost?

✓ Runnable code✓ Exercise with solutionStart Lesson
10

Observability for LLM Serving — Golden Signals, Prometheus and SLOs

Pro 20m

Load tests tell you what the system can do; observability tells you what it is doing right now, for real users, and warns you before they notice.

✓ Runnable code✓ Exercise with solutionStart Lesson
11

Disaggregated Serving — Splitting Prefill and Decode Across Machines

Pro 25m

On a shared GPU, prefill and decode fight: a prefill step delays every decoding sequence (TPOT spikes), and decode-heavy batches delay new prompts (TTFT grows). Every scheduler knob trades one against the other.

✓ Runnable code✓ Exercise with solutionStart Lesson
12

Tiered KV Cache — HBM, CPU Memory and SSD for Long Contexts and Reuse

Pro 15m

GPU memory is the most expensive and smallest place to keep a KV cache, so a GPU-only prefix cache forgets quickly.

✓ Runnable code✓ Exercise with solutionStart Lesson
13

Inference Economics — What a Token Costs, and Which Levers Move It

Pro 25m

Inference is where AI products spend most of their money once they have users.

✓ Runnable code✓ Exercise with solutionStart Lesson
14

Autoscaling LLM Servers — Queue-Depth Signals, KEDA and Cold Starts

Pro 20m

Traffic swings several-fold over a day and spikes within minutes. Too little capacity and students queue; too much and the GPU bill doubles for nothing.

✓ Runnable code✓ Exercise with solutionStart Lesson
15

Routing and Token Budgets — Model Cascades, Rate Limits and Load Shedding

Pro 20m

Autoscaling controls supply. This lesson controls demand: which model each request goes to, how many tokens each tenant may consume, how long each response may run, and who waits when the fleet is full.

✓ Runnable code✓ Exercise with solutionStart Lesson
16

The Inference Reading List — The Papers Behind Every Serving Engine

Pro 15m

Engines change every few months; the ideas underneath them change slowly.

✓ Exercise with solutionStart Lesson
17

Bridge: From Serving Systems to the Silicon Underneath

Pro 10m

Atlas's serving stack now batches, pages, quantizes, speculates and caches across tiers.

✓ Runnable codeStart Lesson
Domain 12: Inference Serving & vLLM · Unrolled