You already know: cost per token and per task (Inference economics); queue depth, saturation and shedding (Observability, Measurement & capacity); autoscaling and cold starts (Autoscaling); routing in agent systems (D9 Agent architectures); token-bucket rate limiting (D1 Networking).
Why this matters for Atlas. Autoscaling controls supply. This lesson controls demand: which model each request goes to, how many tokens each tenant may consume, how long each response may run, and who waits when the fleet is full. These gateway decisions often save more than any kernel optimisation, and they are what keep one runaway client from degrading everyone else.
1. One large model for everything is expensive
The question. If the large model answers everything well, why not send everything to it?
Start from nothing. In many products most requests are simple: classify, extract, reformat, define, short factual lookups. With illustrative prices of about $5 per million tokens for a large model and $0.30 for a small one, sending a simple request to the large model costs about 17 times more for no quality gain, and it is slower: a small model has lower inter-token latency and shorter queues.