Core
Evaluating RAG Systems
The central rule of RAG evaluation is to score retrieval and generation separately, because they fail for different reasons and you cannot fix what you cannot isolate. Retrieval is graded against a golden set with recall@k, precision@k, MRR, and NDCG; generation is graded for faithfulness and answer relevance, usually with an LLM judge. Recall@k is the ceiling on everything downstream.
a free account unlocks the core curriculum tier · no card
Related course lessons
Related concepts
Practise this in questions
RAG & Agent System DesignRetrieval metrics are green, the correct passage is provably in the context, and the answer is still wrong. Where do you look? →hard · PremiumSystem Design & Production EngineeringOrchestrate a multi-agent system over a 500-page merger agreement to produce a redlined risk summary that never contradicts the source. →expert · PremiumMachine Learning & Data ScienceCatch hallucinated facts in LLM meeting summaries against the transcript, on a tight human-review budget. →hard · PremiumLLM & GenAI FundamentalsAfter fine-tuning an LLM on company data, do you still need RAG? How would you verify? →hard · PremiumRAG & Agent System DesignA chunking change shipped last week. Aggregate recall barely moved, and support says answers got worse. Find the regression. →hard · PremiumRAG & Agent System DesignYou added a cross-encoder reranker. p95 latency roughly doubled and answer quality did not move. Explain what happened and fix it. →hard · Premium