VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
u_Unrolled
D8: RAG & Knowledge Systems · RAG Evaluation — Measuring Retrieval and Answers Separately
Phase 5/Domain 8/Lesson 7

RAG Evaluation — Measuring Retrieval and Answers Separately

20 min read·Advanced· Unrolled Pro
RAG Evaluation — Measuring Retrieval and Answers Separately

You already know: eval-set sizing, paired comparisons and LLM judges (D7 Prompt evaluation); confidence intervals and the bootstrap (D2 Statistics); every component of a RAG pipeline (this domain).

Why this matters for Atlas. A wrong RAG answer has two possible causes: retrieval never found the right passage, or the model had it and still got the answer wrong. They need completely different fixes, and an end-to-end "accuracy" number can't tell them apart. RAG evaluation therefore measures each stage on its own: retrieval with ranking metrics against labelled relevant documents, and generation with groundedness (is every claim supported by the sources?) and correctness. This lesson builds the metrics, a failure-attribution report, and the statistics to decide whether a change really helped.


1. Retrieval metrics: did we find it, and how high?

The question. Given the documents that answer each question, how do you score a ranked list?

Start from nothing. First you need relevance labels (qrels): for each test question, which documents answer it, optionally graded (2 = answers it, 1 = partly helpful). Then four standard metrics:

That's the preview. The full lesson, with its code and exercises, is part of Unrolled Pro.
Staff AI & Pro Tier Lesson

Unlock "RAG Evaluation — Measuring Retrieval and Answers Separately"

The foundations are free forever. Unrolled Pro unlocks the complete lesson, production architectures, and 55 battle-tested ML system design case studies.

Everything included with Pro:
All 234+ Deep-Dive Lessons across transformer internals, inference, RAG & agents
55 Staff-Level ML & System Design Case Studies with mathematical capacity estimates
AI Engineer Sheet & Exercises with worked solutions, self-checks, and code
100 Landmark Research Papers guided walkthroughs & progress tracking
14-Day Money-Back Guarantee — 100% full refund if you aren't satisfied
View Plans & Unlock All Lessons

From ₹167/mo (~$2/mo) · Cancel anytime · Instant activation

100% Risk-Free: 14-day no-questions-asked money-back guarantee
Build Atlas · Part 7 of 9: Ground it in your documentsUse this lesson in the project: Chunk the lessons by structure, search them with BM25 and dense vectors, fuse the rankings, and measure retrieval apart from answers.