You already know: the system design method, LLM serving basics (prefill and decode, batching, KV cache) and the enterprise RAG case. This case looks similar on the surface (an LLM plus context) but the constraints are almost opposite: tiny answers, enormous request rates, and a latency budget measured in hundreds of milliseconds.
How to read this lesson. No code, on purpose: an interview is about requirements, numbers, components and trade-offs. Numbers marked assumed are assumptions you state out loud; they are chosen to be realistic, not quoted from any product.
1. The prompt, and the questions that change the design
The trap is to treat this as "a chatbot for code". It is not. A suggestion that arrives after the developer has typed the next three characters is worthless, so latency is the product. A staff candidate also spots the hidden cost problem (every pause in typing is a request) and the hidden trust problems (customer source code, secrets, and licensed code).
Assumptions you state: