12.9.5 — Evaluating RAG

Three proposals reached the "Ask Cartwise" team in the same week. The data team wanted to switch to a bigger embedding model. A consultant recommended doubling the chunk size. And an engineer had a new generation prompt that "felt much more accurate". Each was plausible, each had a cost, and they pulled in different directions. Without measurement, the decision would go to whoever argued best.

The general method is the eval of 12.8.6. RAG adds one essential twist: a wrong answer has two possible causes, and they need completely different fixes. Either retrieval failed, and the right passage never reached the model, or generation failed, and the model had the right passage and still answered badly. Measuring only the final answer tells you that something is wrong and nothing about which half to work on. So RAG is evaluated in two layers, retrieval and generation, measured separately on the same set of questions.

1. The golden set

A RAG evaluation set is a list of real questions, each labelled with:

• the chunks or documents that contain the answer, for measuring retrieval;
• the facts a correct answer must contain, and anything it must not say, for measuring generation;
• its category, such as returns, delivery, membership, or "answer not in the documents".

CASE = {
    "question": "I opened my earphones 9 days after delivery. Can I return them?",
    "relevant_chunks": ["returns-2", "returns-3"],
    "must_include": ["only if faulty", "10-day window for electronics"],
    "must_not": ["30 days", "any reason"],
    "category": "returns",
}

Build it the same way as any eval: start from real customer questions, add every failure found, and add deliberate edge cases, especially questions whose answer is not in the documents, where the correct behaviour is to decline (12.9.4). Labelling which chunks are relevant takes effort; a practical shortcut is to have a model propose candidate relevant chunks for each question and a person confirm or correct them. Questions can also be generated from the documents by a model, which is a fast way to cover every section, but generated questions tend to reuse the document's own wording, which makes retrieval look easier than it is with real customers' phrasing. Use them to fill gaps, not as the core. A set of 100 to 300 questions is enough to make most decisions, with the noise level estimated as in 12.8.6.

2. Measuring retrieval

Retrieval metrics ask: for each question, did the relevant chunks come back, and how high? Here are four questions, each with one relevant chunk, and the rank at which retrieval returned it among the top 5:

Question	Rank of the relevant chunk
Q1	1
Q2	3
Q3	not in the top 5
Q4	2

Recall at k, written recall@k, is the fraction of relevant chunks that appear in the top k. Here, recall@5 is 3/4 = 0.75: three of the four relevant chunks were found. Recall@1 is 1/4. This is the most important retrieval number, because nothing downstream can recover a chunk that was never retrieved: a perfect generator given the wrong passages still produces a wrong or declined answer. Measure it at the number of candidates you pass to the reranker and at the number you finally send to the model, since the first tells you whether retrieval found it and the second whether reranking kept it.

Precision at k is the fraction of the top k that are relevant. With one relevant chunk per question, it is at most 1/k, so it is most useful when questions have several relevant chunks.

Mean reciprocal rank, MRR, rewards finding the relevant chunk near the top. For each question, take one divided by the rank of the first relevant result (0 if it was not found), and average: \frac{1}{4}\left(1 + \frac{1}{3} + 0 + \frac{1}{2}\right) \approx 0.458. A system that always finds the right chunk at rank 1 scores 1.0; one that finds it at rank 2 every time scores 0.5.

nDCG, normalised discounted cumulative gain, handles questions with several relevant chunks of different importance, and rewards putting the most important first. Each result gets a relevance grade (say 3 for the chunk that fully answers the question, down to 0 for irrelevant), and gains are added up with a discount that grows with the rank:

\text{DCG@}k = \sum_{i=1}^{k} \frac{2^{\text{rel}_i} - 1}{\log_2(i + 1)}

Read it as "for each position i, two to the power of its relevance, minus one, divided by the log base 2 of i plus one, all added up". Dividing by the log means that relevance at rank 1 counts fully, at rank 2 about 63 percent, at rank 3 half, and so on. To make scores comparable between questions, divide by the best possible DCG, obtained by sorting the same results perfectly; that ratio is nDCG, between 0 and 1.

For a question whose top 5 results have grades (3, 0, 2, 1, 0): \text{DCG} = \frac{7}{1} + 0 + \frac{3}{2} + \frac{1}{\log_2 5} + 0 \approx 7 + 1.5 + 0.431 = 8.931. The ideal order (3, 2, 1, 0, 0) gives 7 + \frac{3}{1.585} + \frac{1}{2} \approx 9.393. So nDCG@5 is 8.931 / 9.393 \approx 0.951: nearly perfect, losing a little because the second-best chunk was at rank 3 instead of 2. Had the best chunk been at rank 2 instead of 1, with grades (0, 3, 2, 1, 0), nDCG would fall to about 0.676, which shows how heavily the metric rewards getting the top position right.

import math

def recall_at_k(ranked_ids, relevant, k):
    return len(set(ranked_ids[:k]) & set(relevant)) / len(relevant)

def reciprocal_rank(ranked_ids, relevant):
    for i, cid in enumerate(ranked_ids, start=1):
        if cid in relevant:
            return 1 / i
    return 0.0

def ndcg_at_k(grades, k):
    dcg = lambda g: sum((2**r - 1) / math.log2(i + 2) for i, r in enumerate(g[:k]))
    ideal = dcg(sorted(grades, reverse=True))
    return dcg(grades) / ideal if ideal else 0.0

print(round(ndcg_at_k([3, 0, 2, 1, 0], 5), 3))   # → 0.951

These three functions are all you need to run retrieval evaluation over the golden set; in the ndcg_at_k helper, i + 2 is the "\log_2(i + 1)" of the formula with positions counted from 0.

3. Measuring generation

Generation metrics ask: given the retrieved passages, was the answer good? They are usually graded by a model acting as a judge, with the care described in 12.8.6, against the specific criteria in each case.

• Faithfulness: is every claim in the answer supported by the retrieved passages? This is the hallucination measure, checked claim by claim exactly as in the audit of 12.9.4. It is judged against what was retrieved, not against the truth, which is what makes it diagnostic: an answer can be faithful to wrong passages, which points at retrieval.
• Answer relevance: does the answer actually address the question, rather than a nearby one, or wander?
• Correctness: does it contain the required facts (must_include) and avoid the forbidden ones (must_not)? This needs the reference facts in the golden set.
• Citation accuracy: does each citation point to a passage that supports the sentence it follows?
• Appropriate declines: for questions whose answer is not in the documents, did it decline rather than invent?

Open-source frameworks such as RAGAS package judge prompts for these metrics. They are a convenient start; calibrate their judges against your own human labels before trusting the numbers.

4. Reading the two layers together

The point of measuring both layers is that their combination tells you where the problem is:

Retrieval found it?	Answer correct?	Diagnosis
yes	yes	working
yes	no	generation problem: prompt, model, too many chunks
no	no	retrieval problem: chunking, search, rewriting, reranking
no	yes	lucky, or the model answered from memory: check faithfulness

The last row is easy to miss and matters: an answer can be "correct" without the right passage, because the model happened to know a common fact. That is a fragile success, the very behaviour that produced the wrong 30-day answer in 12.9.1, and a low faithfulness score exposes it.

The debugging order

When a specific answer is wrong, walk through the pipeline in order, from the data outwards, because the most common causes come first and most teams jump straight to the prompt:

1. Is the answer in the documents at all? If not, no pipeline change can help: the content must be added.
2. Was it extracted correctly? Look at the indexed text for that passage (12.9.2). A destroyed table looks like a retrieval failure.
3. Was it retrieved among the candidates? If not: hybrid search, query rewriting, chunk size, contextual retrieval (12.9.3).
4. Did it survive reranking into the final set? If not: retrieve more candidates, a better reranker, or keep more chunks.
5. Was it in the context but not used, or misused? Then it is a generation problem: fewer, better-ordered chunks, a clearer instruction, a stronger model.
6. Did the sources conflict? A data problem: the golden record and recency rules of 12.9.2.

5. Using the numbers to decide

Back to the three proposals. The team ran each against the golden set of 240 questions. The results below are illustrative of the kind of table such a comparison produces:

Change	Recall@50	Recall@5 (after rerank)	Faithfulness	Correct
Current system	0.91	0.82	0.94	0.80
Bigger embedding model	0.93	0.83	0.94	0.81
Chunks twice as large	0.90	0.76	0.92	0.75
New generation prompt	0.91	0.82	0.97	0.84
Prompt + query rewriting	0.95	0.88	0.97	0.89

Reading it: the bigger embedding model helped retrieval a little, within the noise for 240 questions, at a real cost in re-indexing and serving, so it was not worth it yet. Larger chunks hurt both recall after reranking and correctness. The new prompt really did improve faithfulness and correctness, so the engineer's feeling was right this time, but now it was a measurement. And the experiment nobody had proposed, adding query rewriting for follow-up questions, produced the biggest gain of all, because the per-category breakdown showed that most remaining failures were second and third messages in conversations. A good evaluation does not just settle arguments; it points at the fix nobody suggested.

6. Evaluating in production

The golden set is a snapshot. In production, three signals show whether RAG quality is holding:

• Retrieval health: the share of questions where the best rerank score falls below your relevance threshold, meaning nothing good was found. A rise usually means new kinds of questions, or documents that changed or vanished.
• Sampled faithfulness and correctness, judged daily on a random sample of real conversations.
• Behavioural signals: how often customers ask for a human after an answer, rephrase the same question, or contact support again about the same order.

New failures found this way go straight into the golden set, closing the loop of 12.1.4 once more.

What the interviewer will push on

"How do you evaluate a RAG system?" They want the two layers: retrieval metrics (recall@k first, then MRR or nDCG) against labelled relevant chunks, and generation metrics (faithfulness, relevance, correctness, citations, declines) graded by a calibrated judge, on a golden set of real questions with a category breakdown.

"The answers are wrong. Where do you start?" The debugging order from the data outwards: is it in the corpus, extracted correctly, retrieved, kept after reranking, used correctly, or conflicting. Jumping to prompt changes is the weak answer.

"What is nDCG?" Graded relevance, discounted by the log of the rank, normalised by the ideal ordering; it rewards putting the most useful results first.

"Why is a correct answer with low faithfulness a problem?" The model answered from memory rather than the sources, which works for common facts and fails for anything specific to the business.

Recall

• Evaluate RAG in two layers on the same golden set: retrieval and generation. A wrong answer is either a passage never retrieved or a passage misused, and the fixes differ.
• The golden set: real questions, relevant chunk IDs, required and forbidden facts, categories, and deliberate "not in the documents" cases. Generated questions fill gaps but flatter retrieval.
• Retrieval:
  • recall@k matters most, because nothing later can recover a chunk that was never found.
  • precision@k.
  • MRR. The worked example came to 0.458.
  • nDCG, which uses graded relevance, discounts lower ranks by a logarithm and is normalised. The worked example came to 0.951, or 0.676 with the best chunk at rank 2.
• Generation: faithfulness to the retrieved passages, answer relevance, correctness against reference facts, citation accuracy, and appropriate declines, judged by a calibrated model.
• Combine the layers to diagnose; debug from the data outwards; decide changes by measured tables, not argument; in production watch low-relevance retrievals, sampled faithfulness and behavioural signals.

Self-test: Compute MRR if Q3's relevant chunk had been at rank 4. Why is recall measured both before and after reranking? An answer is correct but faithfulness is low: what has probably happened, and why does it matter?

Next: 12.9.6 Advanced RAG Patterns — the pipeline so far answers questions one passage can answer; the last RAG page covers conversations, questions that join facts across many documents, retrieval inside agents, images and tables, and when long context is the better choice.