Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

Retrieval-augmented generation prepends retrieved passages to the prompt and hopes the model uses them well. It retrieves even when the input needs no facts (write an essay about your summer), it passes along irrelevant passages, and nothing checks that the answer follows the evidence. Akari Asai, Hannaneh Hajishirzi and colleagues at the University of Washington, AI2 and IBM proposed Self-RAG, which makes those judgments part of the model's own output.

The model gets four kinds of reflection tokens. Retrieve says whether to fetch passages before the next sentence. IsRel says whether a retrieved passage is relevant. IsSup says whether the sentence just written is fully, partially or not supported by the passage. IsUse rates the whole answer's usefulness from 1 to 5.

Training happens in two steps. GPT-4 labels 4,000 to 20,000 examples per token type, and a critic (Llama 2 7B) learns to reproduce those labels. The critic and a retriever then annotate 150,000 instruction-output pairs offline: they insert passages where retrieval helps, and add the relevance, support and utility tokens. The generator (Llama 2 7B or 13B) is trained on that corpus with the ordinary next-token loss, so at inference it emits reflection tokens itself, without a separate critic or reward model.

At inference, the model works one sentence at a time. If it predicts retrieval (or the probability of Retrieve=Yes passes a threshold), it retrieves passages, writes one candidate sentence per passage in parallel, and scores each by its relevance, support and utility tokens. A segment-level beam search keeps the best candidates. The weights on each score can be changed at inference: weight support higher for citation precision, lower for fluency.

On six tasks (open-domain QA, fact verification, multiple-choice reasoning, biographies and long-form QA with citations), Self-RAG beat open models with or without retrieval and beat retrieval-augmented ChatGPT on several.

Self-RAG is one of the most cited RAG papers and the origin of patterns you will see everywhere: decide whether to retrieve, grade passages before using them, check that the answer is supported, and cite per sentence.

You can apply the pattern without training. The four judgments map directly onto pipeline steps: a retrieval router, a passage grader, a groundedness check on each sentence, an answer-quality score. Many production RAG systems implement them with prompts or small classifiers rather than special tokens.

Retrieval decisions are where the risk is. In the paper's own ablation, letting the model decide when to retrieve, with no threshold to push it, lost 17 points on PopQA. The reported numbers use a threshold that makes it retrieve almost always. For factual products, default to retrieving.

Grounding checks cost compute. Writing one candidate per passage and keeping a beam multiplies generation work by up to ten per sentence. Budget latency before promising per-sentence verification.

Citation precision is a trainable property. Open models with plain retrieval almost never produced supported citations in this benchmark; a model trained to cite reached 70%.