You already know: throughput, goodput and capacity planning (Measurement & capacity); cost per successful task (D10 LLM evaluation); every lever this domain built: batching, KV memory, quantization, speculation, prefix and tiered caching, disaggregation.
Why this matters for Atlas. Inference is where AI products spend most of their money once they have users. The cost of a token isn't a property of the model; it's GPU price divided by useful throughput, and throughput depends on everything you've built in this domain, plus one thing people forget: how busy the GPUs actually are. This lesson derives cost per million tokens from first principles, explains why input and output tokens are priced differently, compares self-hosting with APIs honestly, and ranks the levers by how much they move the bill.
1. Cost per token from first principles
The question. A GPU rents for $2.50 an hour. What does a million output tokens cost?
Start from nothing. Cost per token = (cost per GPU-hour × GPUs per replica) ÷ (useful tokens per hour per replica). Useful tokens per hour depend on: