You already know: TTFT, TPOT and goodput (Inference fundamentals); scheduler knobs, chunked prefill and preemption (Batching & scheduling); engine configuration (Inference frameworks); percentiles, intervals and honest comparisons (D10 Evaluation foundations); system-level monitoring (D11 Model monitoring).
Why this matters for Atlas. Capacity planning answers one question: how many GPUs do we need to serve this traffic within these latency targets, at what cost? Every number in that answer comes from a benchmark, and LLM benchmarks are easy to get wrong in ways that overstate capacity by 2–5×: closed-loop load generators that never queue, unrealistic prompt lengths, averages instead of tails, and metrics like "tokens per second" with no latency attached. This lesson builds a correct load test, finds the knee of the latency curve, deliberately breaks one GPU to locate its failure cliffs, computes utilization metrics (MFU, MBU), and turns the results into a capacity plan.
1. Open-loop load: the coordinated omission trap
The question. Why did the load test say 40 requests per second when production fails at 12?
Start from nothing. A closed-loop load generator has N virtual users, each sending a request, waiting for the response, then sending the next. When the server slows down, the generator automatically slows down too: it never builds the queue real traffic would build, and the latency it measures leaves out the time requests would have spent waiting. This is coordinated omission. Real users arrive independently of how busy you are: an open-loop generator sends requests on a schedule (e.g. Poisson arrivals at a target rate) regardless of whether earlier ones finished, and measures latency from the scheduled send time.