Capstone: A Production Support Assistant — and Choosing Between Prompting, RAG and Fine-Tuning
15 min read·Advanced· Unrolled Pro
Diagram— a decision tree for prompt vs RAG vs fine-tune (does the model need knowledge it lacks? does it change often? is the problem behaviour/format rather than knowledge?), and the support assistant's pipeline: ticket → PII redaction → intent router → (account actions via tools with confirmation | knowledge answers via permission-filtered RAG with citations | escalate) → answer with citations → evaluation set and online metrics (resolution rate, escalation precision, cost per ticket).
You already know: prompting and structured outputs (D7), RAG with hybrid retrieval and citations (D8), tool use and agent guardrails (D9), fine-tuning (D6), evaluation (D7 prompt evaluation, D10), agent security (D19).
Why this matters for Atlas. This capstone pulls the AI-engineering path into one production system. The first decision is architectural: prompt, RAG or fine-tune? Most teams reach for fine-tuning too early; most knowledge problems are retrieval problems, and most behaviour problems are prompting problems. Then comes the real work: routing, grounded answers, safe actions, escalation, an evaluation set that mirrors real tickets, and the cost and latency budget. Each step is small; doing all of them well is what makes the assistant trustworthy.
1. Prompt, RAG or fine-tune?
The question. The assistant gives outdated refund rules and sometimes answers in the wrong format. Which tool fixes which problem?
Start from nothing. Diagnose the failure first:
That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson
Unlock "Capstone: A Production Support Assistant — and Choosing Between Prompting, RAG and Fine-Tuning"
The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.
Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied