VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D9: AI Agents & Tool Use · Capstone: A Production Support Assistant — and Choosing Between Prompting, RAG and Fine-Tuning
Phase 5/Domain 9/Lesson 9

Capstone: A Production Support Assistant — and Choosing Between Prompting, RAG and Fine-Tuning

15 min read·Advanced· Unrolled Pro
a decision tree for prompt vs RAG vs fine-tune (does the model need knowledge it lacks? does it change often? is the problem behaviour/format rather than knowledge?), and the support assistant's pipeline: ticket → PII redaction → intent router → (account actions via tools with confirmation | knowledge answers via permission-filtered RAG with citations | escalate) → answer with citations → evaluation set and online metrics (resolution rate, escalation precision, cost per ticket).
Diagram— a decision tree for prompt vs RAG vs fine-tune (does the model need knowledge it lacks? does it change often? is the problem behaviour/format rather than knowledge?), and the support assistant's pipeline: ticket → PII redaction → intent router → (account actions via tools with confirmation | knowledge answers via permission-filtered RAG with citations | escalate) → answer with citations → evaluation set and online metrics (resolution rate, escalation precision, cost per ticket).

You already know: prompting and structured outputs (D7), RAG with hybrid retrieval and citations (D8), tool use and agent guardrails (D9), fine-tuning (D6), evaluation (D7 prompt evaluation, D10), agent security (D19).

Why this matters for Atlas. This capstone pulls the AI-engineering path into one production system. The first decision is architectural: prompt, RAG or fine-tune? Most teams reach for fine-tuning too early; most knowledge problems are retrieval problems, and most behaviour problems are prompting problems. Then comes the real work: routing, grounded answers, safe actions, escalation, an evaluation set that mirrors real tickets, and the cost and latency budget. Each step is small; doing all of them well is what makes the assistant trustworthy.


1. Prompt, RAG or fine-tune?

The question. The assistant gives outdated refund rules and sometimes answers in the wrong format. Which tool fixes which problem?

Start from nothing. Diagnose the failure first:

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "Capstone: A Production Support Assistant — and Choosing Between Prompting, RAG and Fine-Tuning"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee
Build Atlas · Part 8 of 9: Let it actUse this lesson in the project: Give Atlas tools, memory and a loop, then put a gateway between the model and the tools so that a fooled model still can't do damage.