How do you measure whether RAG retrieval is actually improving?

1-minute summary

• Evaluate retrieval separately from answer generation. Otherwise, one score cannot tell you what broke.
• Use Recall@k for coverage, MRR for the first useful result, and nDCG@k when relevance has levels and order matters.
• Keep a versioned query-and-relevance set, compare every retrieval change against a baseline, and inspect results by query segment.

Who this is for

• Teams tuning chunking, embeddings, filters, hybrid search, rerankers, or top-k in a RAG system
• Engineers who need a repeatable release gate instead of judging a few answers by eye
• Product teams that must distinguish search failures from prompt or model failures

Conclusion

Start with two evaluation layers:

1. Retrieval evaluation: Did the system find the right evidence and rank it high enough?
2. Answer evaluation: Given the retrieved evidence, did the model answer correctly, completely, and without unsupported claims?

For retrieval, a practical minimum is Recall@k, MRR, and nDCG@k. No metric works alone. Recall can improve while noisy chunks crowd the context window. MRR can look strong even when multi-document questions miss half their evidence. nDCG captures graded relevance and ordering, but it still depends on reliable labels.

Use the metrics to compare a candidate configuration with a fixed baseline. Do not treat a universal score such as “0.8 is good” as a release standard.

Explanation

Retrieval and generation fail differently

A wrong answer can come from at least three places:

• the index does not contain usable evidence
• the retriever fails to return or rank that evidence
• the model receives good evidence but misuses it

One end-to-end judge score hides these causes. Run the same question with retrieved context and with an oracle context that contains the known evidence. If the oracle run succeeds and the normal run fails, focus on retrieval. If both fail, inspect the prompt, model, or answer grader.

What the core retrieval metrics tell you

Recall@k measures the fraction of known relevant documents or chunks found in the top k. Use it when missing evidence makes an answer incomplete.

MRR (Mean Reciprocal Rank) averages 1 / rank for the first relevant result. Use it when the first strong result matters most, such as single-answer question answering.

nDCG@k (Normalized Discounted Cumulative Gain) rewards highly relevant results near the top and compares the ranking with an ideal ordering. Use it when relevance is graded, for example:

• 3: directly answers the question
• 2: useful supporting evidence
• 1: related but insufficient
• 0: irrelevant

Also track Precision@k or the irrelevant-chunk rate when context noise is costly. A retriever can reach high recall by returning too much.

Choose the evaluation unit before computing scores

Decide whether a relevance label applies to a source document, section, or chunk. Changing chunk boundaries while keeping old chunk IDs can invalidate the test set. Stable source IDs plus quoted evidence spans make labels easier to migrate after re-indexing.

Measure at the stages users depend on:

• candidate retrieval, such as top 50 before reranking
• post-reranker results, such as top 10
• final context, such as the 4 chunks sent to the model

This shows whether a reranker improves ordering or whether context assembly drops good evidence later.

Practical Guide

Step 1: define query segments and failure costs

Group queries before averaging them:

• direct fact lookup
• multi-document synthesis
• exact identifiers, error codes, and product names
• paraphrased or vague questions
• no-answer questions
• permission-sensitive questions

A single average can hide a complete failure in a small but high-risk segment.

Step 2: build a labeled evaluation set

Start with 30–100 representative questions. Include real queries, known failures, and a smaller set of adversarial cases.

Store enough data to reproduce each judgment:

{
  "query_id": "billing-017",
  "query": "When can an annual plan be refunded?",
  "segment": "policy_lookup",
  "relevance": {
    "refund-policy#annual-plans": 3,
    "billing-faq#cancellations": 2
  },
  "must_not_retrieve": ["tenant-b-private-policy"]
}

Have a domain owner review ambiguous labels. For permission-sensitive systems, record forbidden results as well as relevant ones.

Step 3: freeze the baseline

Record every setting that can change retrieval:

• corpus snapshot and index version
• chunking rules
• embedding model
• keyword, vector, and hybrid weights
• metadata filters
• reranker model and candidate count
• final context limit

Without this snapshot, a metric change is difficult to explain or reproduce.

Step 4: compute metrics at fixed cutoffs

Choose k values that match the pipeline, such as k = 5, 10, and 50. Report at least:

• Recall@k for evidence coverage
• MRR for first-result rank
• nDCG@k for graded ranking quality
• irrelevant-chunk rate or Precision@k for noise
• forbidden-retrieval count for access-control regressions

Compare the same queries, labels, and cutoffs across the baseline and candidate.

Step 5: evaluate the final answer separately

For each query, save:

• retrieved chunk IDs and scores
• final chunks sent to the model
• generated answer
• cited chunk IDs
• latency and cost

Then score groundedness, answer relevance, and completeness. Use deterministic checks for required IDs, dates, citation presence, and access rules. Use a stable human rubric or LLM judge for semantic quality.

Step 6: run the oracle-context test

Replace retrieved context with the known relevant evidence and generate the answer again.

• normal fails, oracle passes: retrieval or context assembly problem
• both fail: generation, prompt, or grading problem
• normal passes, retrieval metrics look weak: labels or answer leakage may be wrong

This test turns a vague “RAG quality” issue into a narrower engineering task.

Step 7: set regression gates by segment

Use baseline-relative gates, for example:

• zero forbidden-document retrievals
• no drop in Recall@10 for policy and safety queries
• no material nDCG@10 regression overall
• P95 latency and cost stay within the agreed budget
• manually review every newly failed query

Keep both aggregate metrics and per-query diffs. The diffs explain the score.

Step 8: refresh the set from production failures

Add sampled failures, low-confidence answers, citation complaints, and no-answer mistakes to the evaluation set. Keep a stable core for trend comparison, then add a rotating set for new behavior and drift.

Pitfalls

• evaluating only final answers and losing the retrieval failure signal
• generating all test questions synthetically and missing real query language
• using one relevant chunk when the answer requires several sources
• changing chunk IDs without migrating relevance labels
• optimizing Recall@k by flooding the context with weak matches
• averaging across segments and hiding permission or no-answer failures
• changing the judge prompt between runs
• tuning on the test set until it becomes a training set
• treating retrieval similarity scores as comparable across different systems
• ignoring index freshness, latency, and cost while optimizing relevance

Checklist

• [ ] Retrieval and answer generation are evaluated separately
• [ ] The evaluation set includes real user language and known failures
• [ ] Queries are segmented by task and risk
• [ ] Relevant evidence has stable source IDs or quoted spans
• [ ] Relevance supports graded labels where useful
• [ ] Multi-document questions label every required source
• [ ] No-answer and permission-sensitive cases are included
• [ ] The corpus, index, chunking, embeddings, filters, and reranker are versioned
• [ ] Recall@k is measured at pipeline-relevant cutoffs
• [ ] MRR tracks the first useful result
• [ ] nDCG@k tracks graded order quality
• [ ] Precision or irrelevant-chunk rate tracks context noise
• [ ] Forbidden-document retrieval is a zero-tolerance metric
• [ ] Final answers are checked for groundedness and completeness
• [ ] Oracle-context runs separate retrieval failures from generation failures
• [ ] Release gates compare the same cases with a fixed baseline
• [ ] Per-query regressions are reviewed, not only aggregate scores
• [ ] Production failures feed a versioned expansion set

FAQ

1. Which metric should be the primary RAG retrieval metric?

Use the metric that matches the task. Recall@k fits evidence coverage, MRR fits first-answer lookup, and nDCG fits graded ranked results. Most production RAG systems need at least one coverage metric and one ranking metric.

2. What value of k should I use?

Match k to real pipeline stages. If you retrieve 50 candidates, rerank 10, and send 4 chunks, measure those cutoffs. Reporting only Recall@100 can hide a poor final context.

3. Can an LLM judge replace relevance labels?

It can accelerate labeling and cover cases without ground truth, but sample its decisions for human review. Keep a small, trusted labeled core for regression gates.

4. How large should the evaluation set be?

Start with 30–100 representative cases and expand from real failures. Coverage across query types matters more than collecting a large pile of similar questions.

Sources

• Evaluate AI Search retrieval quality (Microsoft Learn)
• RAG evaluators: retrieval, groundedness, relevance, and completeness (Microsoft Learn)
• RAGAS: Automated Evaluation of Retrieval Augmented Generation (EACL 2024)
• Evaluation in information retrieval (Stanford IR book)

Internal links

• Parent hub: AI development
• Related:
  • Hybrid search and reranking for RAG
  • RAG chunking and indexing checklist
  • LLM evals and regression tests

Disclaimer

General engineering guidance only. Validate evaluation rules against your data, access model, and risk requirements.