You already know: BM25 and its blind spots (Retrieval fundamentals); dense retrieval and contrastive training (Embeddings); structural chunks with heading paths (Document ingestion); ANN indexes and filtering (Vector databases); chains with checks (D7 Advanced prompting).
Why this matters for Atlas. "Basic RAG" — embed the question, take the top 5 chunks, generate — is a demo, not a product. Production RAG is a pipeline of small, measurable improvements, each fixing a specific failure: hybrid search for codes and paraphrases, reranking for precision, query rewriting for vague questions, contextual chunks for orphaned facts, multi-hop retrieval for questions that span documents, diversity for redundant results, and citation checking for trust. This lesson builds each one and measures what it fixes and what it doesn't.
1. Hybrid search: keywords and meaning together
The question. BM25 misses paraphrases; dense retrieval misses exact codes. Can one system get both?
Start from nothing. Run both retrievers and fuse their result lists. Their scores live on different scales (BM25 is unbounded, cosine sits between −1 and 1), so the simplest robust fusion ignores scores and uses ranks. Reciprocal Rank Fusion (Cormack et al., 2009) scores each document as Σ 1/(k + rank) over the lists it appears in, with k ≈ 60. To make the dense side realistic, its "model" is trained on general text that never contains this help centre's error codes, just as a real embedding model was trained before your product existed: