VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
Easy10 XP Today's questioncachingcost

How much does an LLM response cache save?

An FAQ assistant gets 1,000 requests per second and a cache answers 30% of them. How many model calls per second remain?

calls per second
Stuck? Show the solution

Answer: 700 calls per second

Only cache misses reach the model: 1,000 × (1 − 0.3) = 700 calls per second.

Learn the idea behind itContext Engineering — Deciding What the Model Sees