You already know: KV sizes, paging, prefix caching and preemption by swap or recompute (KV cache engineering); the memory hierarchy and paging to disk (D1 Operating systems); radix-tree prefix caches and cache-aware routing (Inference frameworks); moving KV between machines (Disaggregated serving).
Why this matters for Atlas. GPU memory is the most expensive and smallest place to keep a KV cache, so a GPU-only prefix cache forgets quickly. Yet the reuse opportunity is enormous: multi-turn conversations, agents that call the model dozens of times on a growing context, shared documents many students ask about. A tiered KV cache keeps hot KV in GPU HBM, warm KV in CPU DRAM, and cold KV on local SSD (or a shared store across machines), and decides per request whether loading cached KV is faster than recomputing it. This is the memory-hierarchy thread at its most direct, and it's what turns "every return visit repays the whole prefill" into "only the new message is computed".
1. The hierarchy and its numbers
The question. What does each tier offer, and how much KV can it hold?
Start from nothing. Each step down is larger, cheaper and slower: