From Local to Global: A Graph RAG Approach to Query-Focused Summarization

Ask a vector RAG system "what was the refund policy in 2023?" and it does well: find the few chunks about refunds, answer from them. Ask it "what are the main themes in these thousand interviews?" and it fails with no sign of failing. It retrieves the few chunks most similar to the question and summarizes those, presenting a sample as if it were the whole picture. The authors call these sensemaking questions, and the task query-focused summarization over a corpus too large for any context window.

GraphRAG, from Microsoft Research, does the expensive work at indexing time. An LLM reads every chunk and extracts entities (people, organizations, places, concepts) and the relationships between them, with short descriptions. Those form a knowledge graph. A community-detection algorithm (Leiden) splits the graph into groups of closely connected entities, at several levels of granularity. The LLM then writes a summary of each community, bottom-up.

At question time, each community summary is used to write a partial answer, in parallel (map), and the partial answers are combined into one (reduce). Every community gets a say, so the answer covers the corpus rather than a top-k sample.

On two corpora of about a million tokens each, podcast transcripts and news articles, an LLM judge preferred GraphRAG's answers to vector RAG's for comprehensiveness and diversity roughly three times out of four. Vector RAG still gave the more direct answers. The method is open source and widely copied.

Customers ask global questions all the time, they just do not call them that: "what are customers complaining about this quarter?", "what risks show up across these contracts?", "summarize what the field teams are reporting". Vector RAG answers them badly and confidently. This paper is the clearest statement of why and of one fix.

Spot the question type before choosing the architecture. If the answer depends on a few specific passages, vector RAG fits. If it depends on patterns across the whole collection, retrieval of a top-k sample is the wrong tool. Most enterprise assistants get both kinds, so routing by question type matters.

The cost moves to indexing. Every chunk passes through an LLM to extract entities, and every community gets an LLM-written summary. Indexing the one-million-token podcast corpus took 281 minutes of GPT-4-Turbo calls. That is affordable once; it is a real line item for a corpus that changes daily, because updates mean re-extracting and re-summarizing.

The evaluation is softer than it looks. The win rates come from an LLM judging answers to LLM-generated questions, with no ground truth and, as the authors acknowledge, no measurement of fabrication. "More comprehensive and diverse" is not the same as "more correct". Before recommending GraphRAG to a customer, test it on their questions with their experts.