You already know: prefill builds the KV cache and decode reads it every step (Inference fundamentals); multi-head, multi-query and grouped-query attention (D5 Transformer architecture); virtual memory, pages and fragmentation, including the PagedAttention preview (D1 Operating systems); prompt caching from the API side (D7 Context engineering).
Why this matters for Atlas. After the weights, the KV cache is the largest consumer of GPU memory, and unlike the weights it grows with every token of every active request. It decides how many requests fit in a batch (throughput), how long a context you can serve, and how much a repeated system prompt costs. Serving engines spend most of their cleverness managing it: allocating it in pages, sharing it across requests with a common prefix, evicting or offloading it under pressure. This lesson explains what the cache holds, computes its size exactly for real architectures, and builds the paged allocator every modern engine uses. Lesson II (KV Cache II) covers sharing it across requests, what happens when it runs out, quantizing it, planning capacity and operating it.
Where you'll use this. Answering "how many concurrent users can this GPU serve at 8K context?" before buying hardware. Understanding why a model with grouped-query attention serves several times more users than one without. Reading a serving engine's startup log ("# GPU blocks: 25,000") and knowing what it means for your traffic.
1. What is cached, and why it's worth it
The question. Why store keys and values at all, instead of recomputing them?
Start from nothing. At decode step t, attention needs the keys and values of all t earlier tokens in every layer. Without a cache you'd recompute them from scratch each step: generating n tokens would cost about n²/2 token-forwards instead of n. With a cache, each token's K and V are computed once (in prefill or in its own decode step) and read back afterwards. Queries are never cached: each is used once. The trade: compute saved for memory spent.