You already know: every mechanism these engines use: prefill/decode and sampling (Inference fundamentals), paged KV and prefix caching (KV cache engineering), continuous batching and chunked prefill (Batching & scheduling), quantized formats (Quantization), speculation, fused attention and tensor parallelism (Advanced inference optimization).
Why this matters for Atlas. You rarely build an inference engine; you choose, configure and operate one. The open-source engines share the same core ideas, and they differ in emphasis: memory management, prefix sharing, compiled kernels, hardware support, ease of operation. Knowing the anatomy lets you read their docs and benchmarks critically, set the few configuration knobs that decide capacity, and choose by your workload rather than by someone else's chart. This lesson opens up the engine, builds SGLang's radix-tree prefix cache, sizes a vLLM deployment, and lays out how to choose.
1. Anatomy of an inference engine
The question. What's inside vllm serve or sglang.launch_server?
Start from nothing. Every modern LLM engine has the same parts: