From RAG to Memory: Non-Parametric Continual Learning for Large Language Models

Embedding retrieval finds passages that look like the question. Multi-hop questions need a passage that does not: "What county is Erik Hort's birthplace a part of?" needs the passage about Erik Hort and then the passage about Montebello, which never mentions him. HippoRAG (2024) handled this by extracting a knowledge graph with an LLM and running Personalized PageRank from the entities in the question. It helped on multi-hop benchmarks but fell below plain embedding retrieval on simple factual questions, as did GraphRAG, LightRAG and RAPTOR.

HippoRAG 2 makes three changes. Passage nodes: every passage joins the graph, connected by "contains" edges to the phrases extracted from it, so PageRank scores passages directly. Query-to-triple linking: instead of extracting entities from the question, the whole query is embedded and matched against triples such as (Erik Hort, born in, Montebello), which carry the relation the question asks about. Recognition memory: an LLM looks at the top 5 matched triples and keeps up to 4 that matter.

The kept triples' phrases, plus every passage weighted by its similarity times 0.05, seed PageRank with damping 0.5. The top 5 passages by score go to the reader.

On seven benchmarks with a Llama-3.3-70B reader, average F1 is 59.8 against 57.0 for NV-Embed-v2, the strongest embedding baseline, and 53.1 for the original HippoRAG. Recall@5 on the three multi-hop sets with passage labels rises by 6.9 points on average. Indexing needs an LLM pass over every passage: 99.5 minutes on 4 H100s for 11,656 passages, against 12.1 for embeddings.

Customers ask questions whose answer is spread across documents: which suppliers serve the plant that failed the audit, which contracts reference the clause that legal just changed, which pump uses the seal named in a recall. Dense retrieval returns the first hop and stops. This paper is the cleanest evidence so far that a graph layer can recover the second hop without losing simple lookups, which is what blocked most graph-RAG proposals in production.

It also tells you what to put in the budget. Indexing is an LLM call per passage, about 790 input and 260 output tokens each on MuSiQue, and the index lives on a GPU box for queries: 9.9 GB of memory during QA beyond model weights, against 1.7 GB for embeddings alone. That is far cheaper than GraphRAG (115.5M input tokens for the same corpus, against 9.2M), and far more than a vector index.

The gains are uneven. They are large where questions chain entities (2Wiki +9.5 F1) and close to zero where the answer sits in one passage (HotpotQA +0.2) or needs reading a whole novel (NarrativeQA +0.2). Run your own multi-hop sample before promising anything, and keep the embedding retriever as the fallback: the method already uses it when the filter keeps no triples.