You already know: LLM serving, the code completion case (latency as the product) and the support agent (actions under policy). Voice combines both: conversation-speed latency and real-world actions, plus two new problems: knowing when the user has finished speaking, and keeping an always-listening device private.
How to read this lesson. No code, on purpose. Numbers marked assumed are assumptions you state out loud; prices are illustrative.
1. The prompt, and the questions that change the design
The mid-level answer is "speech-to-text, then an LLM, then text-to-speech". A staff candidate sees four harder problems: turn-taking (when to respond, what to do when interrupted), the device–cloud split (what runs locally for speed, cost and privacy), misheard actions (sending a message to the wrong person), and evaluation across accents, noise and devices.
Assumptions you state: