VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D12: Inference Serving & vLLM · Inference Economics — What a Token Costs, and Which Levers Move It
Phase 6/Domain 12/Lesson 13

Inference Economics — What a Token Costs, and Which Levers Move It

25 min read·Staff+ Tier· Unrolled Pro
Inference Economics — What a Token Costs, and Which Levers Move It

You already know: throughput, goodput and capacity planning (Measurement & capacity); cost per successful task (D10 LLM evaluation); every lever this domain built: batching, KV memory, quantization, speculation, prefix and tiered caching, disaggregation.

Why this matters for Atlas. Inference is where AI products spend most of their money once they have users. The cost of a token isn't a property of the model; it's GPU price divided by useful throughput, and throughput depends on everything you've built in this domain, plus one thing people forget: how busy the GPUs actually are. This lesson derives cost per million tokens from first principles, explains why input and output tokens are priced differently, compares self-hosting with APIs honestly, and ranks the levers by how much they move the bill.


1. Cost per token from first principles

The question. A GPU rents for $2.50 an hour. What does a million output tokens cost?

Start from nothing. Cost per token = (cost per GPU-hour × GPUs per replica) ÷ (useful tokens per hour per replica). Useful tokens per hour depend on:

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Inference Economics — What a Token Costs, and Which Levers Move It"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee
Build Atlas · Part 9 of 9: Serve it fastUse this lesson in the project: Add a KV cache to TinyGPT, measure what batching buys, simulate static against continuous batching on real traffic, and turn throughput into a cost per token.