Normal RAG (Retrieval-Augmented Generation) is by now a well-worn recipe: chop your documents into flat chunks, embed each one into a vector, and at query time fetch the handful of chunks that sit closest to the question. It's cheap, fast, and genuinely good at one thing — "find me the passage that answers this."
But it quietly falls apart on two kinds of questions. The first is the big-picture question — "what are the main themes across all of these reports?" — where no single chunk holds the answer. The second is the connect-the-dots question, where the answer is scattered across pieces that don't individually look like the query. RAPTOR and GraphRAG are two well-known answers to exactly this, and they fix it in very different ways. This post walks through both — plainly, and step by step.
I'll assume you know the basics of chunking and retrieval already — if not, I wrote separate posts on chunking and retrieval accuracy first.
Here's the whole problem in one picture. Plain vector search embeds each chunk once and returns the ones nearest the query. Ask it something specific and it shines. Ask it something that spans the corpus, and it hands back a few disconnected fragments.
The word "themes" doesn't point at any one chunk, so cosine similarity just grabs the five that happen to say "incident" the loudest — likely all from the same noisy report — and the model answers as if that one report were the whole story. Two failure modes are hiding here, and each technique targets one:
1 · Big-picture / summarization. "What are the themes?", "summarize everything about billing." The answer is a synthesis of many chunks, but top-k only returns a few — so you get a keyhole view. RAPTOR tackles this by pre-computing summaries at every level.
2 · Connect-the-dots / multi-hop. "Which incidents share a root cause?" needs to link facts from different documents that never mention each other. Similarity can't follow a chain of relationships. GraphRAG solves this by making those relationships explicit as a graph.
Let's start with the smaller leap.
RAPTOR stays inside the world you already know — chunks and embeddings — and adds one thing on top: a tree of summaries. The insight is simple. If big-picture questions fail because there's no chunk that summarizes the whole, then build those summaries ahead of time — and not just one, but a whole hierarchy, from tight local summaries up to a bird's-eye view of everything.
The name spells out the recipe: Recursive Abstractive Processing for Tree-Organized Retrieval. You recursively embed, cluster, and summarize your chunks into a tree, then retrieve from every level of it at once. It's from Sarthi et al. at Stanford (2024), with an official implementation. Here's how the tree gets built:
1Start exactly like normal RAG: cut the corpus into small chunks (~100 tokens each) and embed them. These become the leaf nodes — the bottom layer of the tree. Nothing new yet.
2Cluster the leaves — the careful part. Embeddings have hundreds of dimensions, where distances get unreliable, so RAPTOR first shrinks them with UMAP, then runs a Gaussian Mixture Model for soft clustering: a leaf can belong to more than one cluster (the ringed dot), and the number of clusters is chosen automatically. Related passages group up no matter which document they came from.
3Summarize each cluster. Hand every cluster to an LLM to write one short summary. Each summary becomes a new parent node — a single node that stands in for its whole cluster of leaves.
4Now recurse. Treat those summaries as a new layer, re-embed them, cluster and summarize again — and keep going, layer by layer, until the top is down to a handful of high-level summaries (RAPTOR caps this at a few layers). The result is a tree: detailed at the bottom, ever more abstract toward the top.
5Retrieval — the "collapsed tree". Flatten every node (leaves and summaries) into one pool and rank them all by cosine similarity to the query, taking the top ones up to a token budget. A billing question pulls the billing summary directly — one node that already synthesizes the whole topic — instead of scattered fragments. RAPTOR tested this against a top-down tree traversal (walk the tree layer by layer), and the simple collapsed pool won — so it's the default.
The payoff: because summaries live right alongside the raw chunks in the same index, a broad question retrieves a ready-made synthesis, while a narrow question still finds its exact leaf. It's an elegant fix for the big-picture problem, and it's cheap to bolt onto an existing vector pipeline — same embeddings, same retrieval, just extra summary nodes. In the paper, paired with GPT-4, it pushed the best QuALITY benchmark score up by about 20 absolute points (to 82.6%).
The catches are real, though. Every summary is one more LLM call at indexing time, and summaries can smooth over or drop the very detail you needed — a game of telephone up the tree. And it's still fundamentally similarity retrieval: RAPTOR can summarize a topic beautifully, but it doesn't model how two specific entities relate. That's exactly the gap the next technique fills.
GraphRAG (from Microsoft Research, 2024, with a widely-used open-source library) makes a bigger bet. Instead of storing text and matching on similarity, it reads the whole corpus and builds an explicit knowledge graph: the entities (people, products, places, events) become nodes, and the relationships between them become edges. Once the facts are wired together, you can answer questions by traversing the connections, not just by fuzzy-matching words.
The subtitle of the paper — "From Local to Global" — is the whole idea. There are two phases: an expensive indexing phase that builds the graph and pre-summarizes it, and a query phase that reads from it. Let's build it first.
1Chunk the corpus. Exactly like any RAG pipeline, split every document into bite-size chunks. GraphRAG keeps these around — they're the raw text the next step reads, and later what local search retrieves — but on their own they're still just flat text, with no structure yet.
2Extract entities & relationships. An LLM reads every chunk and pulls out the entities and the relationships connecting them — turning free text into structured (node)—[relationship]→(node) triples. It also records the chunk each item came from (a text_unit_ids link), so local search can fetch the raw text back later. This is the expensive step: one LLM pass over the whole corpus.
3Merge into one graph. Stitch every chunk's triples together: the same entity across ten chunks collapses to one node, and repeated links between the same pair collapse to one edge. Two things happen on merge — the edge's weight is simply a count of how many times that pair co-occurred (no LLM strength score; each co-occurrence just adds 1), and however differently each mention described the link, all those descriptions are handed to the LLM and summarized into one. So the whole corpus becomes a single connected map of facts.
4Describe & embed each entity. The extractor didn't just name each entity — it also wrote a short description of it from every chunk it appeared in. On merge, those descriptions are fused into one, and then the entity's name + description is embedded into a vector. That vector is exactly what local search matches the query against — this is where each entity gets its embedding.
5Find communities. A clustering algorithm (GraphRAG uses Leiden) partitions the graph into communities — clumps of nodes densely wired to each other but loosely to the rest. Here the infrastructure facts fall into one community, the billing facts into another. On a real corpus this is hierarchical: communities nest inside bigger communities.
6Summarize each community. An LLM writes a community report for each one — but note what it's fed: the descriptions of that community's entities and relationships (plus any claims), added in order of prominence until a token budget, not the original source chunks. If a community is too big to fit, GraphRAG substitutes its sub-community reports for the detail — that's the bottom-up part. These reports become the pre-computed "global" knowledge the query phase reads from.
That's the index: a graph of entities and relationships, carved into communities, each with its own summary. It's a lot of machinery — but notice what you now have that flat chunks never did: an explicit statement that the outage was caused by the TLS cert and hit the EU region, ready to be traversed.
This is where "from local to global" pays off. For a broad, whole-corpus question, GraphRAG doesn't search for chunks at all — it runs a map-reduce over the community reports.
1Map — every community answers separately. The exact same question is run against each community report on its own, in parallel. Each run lists the points from its slice of the graph that help answer the question, and scores every point 0–100 for relevance — so off-topic bits (a hardware spec, a headcount) score near zero. Because every community is asked, nothing in the corpus is skipped.
2Reduce — keep the best, write one answer. Pool the key points from all communities into a single list, sort by the map-step scores, and drop the low ones. Whatever survives (trimmed to fit the context window) goes to a final LLM call that writes it up as one coherent answer. Since the points came from every community, distinct themes all make it in — exactly what flat top-k missed.
Global search is GraphRAG's headline mode, but it's not the only one — and the other mode matters just as much. For a specific question, GraphRAG runs local search, and this is where the connect-the-dots, multi-hop reasoning actually happens.
Global search sweeps every community; local search does the opposite — it zooms in. It finds the entities your question is about, gathers their immediate graph neighborhood plus the original text those entities came from, and answers from that tight bundle. Take "what caused Tuesday's outage?"
1Find the entry entity. How does it know where to start? Every entity in the graph was stored with an embedding of its name and description. So local search just embeds the question and runs a nearest-neighbor search across the whole entity list — plain vector similarity, exactly like normal RAG, but over entities instead of chunks. The top match (or few) becomes the entry point; here Tue outage wins and the billing entities score too low to matter.
2Walk one hop out — and stop. From each entry entity, local search grabs its directly connected neighbors and the relationships between them. That's the whole traversal — a single hop, not a deep crawl. The relationships are ranked (ones joining two entry entities first, then by how many entry entities they touch and their weight) and capped at a budget of about top-k × the number of entry entities, so the neighborhood stays bounded instead of exploding outward.
3Pull the original text back in. This is what separates local search from global: it doesn't answer from the graph alone. Using the text_unit_ids link saved at build time, it follows each selected entity back to the original source chunks it was extracted from and adds them to the context — along with the community reports those entities sit in, and any recorded claims. Structured graph facts and unstructured raw text, side by side.
4Fit it to the window, then answer. Everything gathered is ranked and packed into one fixed token budget, split by type: the raw source chunks (the biggest slice), the graph facts — the matched entities with their descriptions, the relationships, and any claims — and the community reports. That bundle goes to the LLM, which answers by connecting facts the graph made explicit — something flat similarity search would never have retrieved together.
So which mode runs for a given question? The paper doesn't say — you just pick the method per call. But it's easy to automate with a simple intent-classification router (the same trick from the retrieval post): a small LLM classifier reads the question and sends whole-corpus themes to global, specific-entity questions to local. There's also DRIFT search, a hybrid that blends both so you needn't choose.
The strengths are unique: GraphRAG genuinely shines on global sensemaking and multi-hop questions, and its answers are traceable — you can point at the exact entities and edges behind them. Microsoft's paper reports clear wins over vanilla RAG on the comprehensiveness and diversity of answers to whole-corpus questions.
But the cost is the real catch, and it comes in a few flavors:
Which brings us to the honest comparison.
Let's put all three on the same big-picture question and watch them diverge — "What are the main themes across Nimbus's reports?"
1Normal RAG. Top-k similarity has no notion of "the whole corpus." It returns a few near chunks, which cluster around one loud topic — so the summary is narrow and misses everything else.
2RAPTOR. Because summary nodes sit in the same index, the query lands on one that already blends multiple reports. A solid big-picture answer — though it's still whichever summary is most similar, so a theme with no close-matching summary can slip through.
3GraphRAG. Global search touches every community by construction, so distinct themes all make it into the answer — the most complete and diverse of the three. The price is the expensive graph you had to build first.
4The takeaway. There's no winner — they climb a ladder of power vs. cost. More structure buys better answers on hard, holistic questions, and charges you for it at indexing time. Match the tool to the question you actually get.
Both techniques exist for the same reason — flat chunks lose the big picture — but they buy it back differently: RAPTOR pre-computes summaries so no question lacks a synthesis; GraphRAG pre-computes relationships so no connection is invisible. Here's the cheat sheet:
| Approach | Adds | Pros | Cons | Reach for it when… |
|---|---|---|---|---|
| Normal RAG | flat chunks | Cheap, fast, simple; instant to update; strong on pinpoint lookups. | No big-picture; can't connect dots across documents. | Questions are answerable by a single passage — most FAQ/support/doc search. |
| RAPTOR | summary tree | Big-picture answers at every abstraction level; cheap add-on to a vector pipeline; keeps detail leaves too. | An LLM summary per cluster at index time; summaries can lose detail; still pure similarity, no explicit relations. | Long documents and thematic/summarization questions — books, papers, reports. |
| GraphRAG | knowledge graph | Global sensemaking + multi-hop reasoning; traceable, explainable answers; consults the whole corpus. | Very expensive to build (LLM per chunk + per community); slow to update; overkill & noisy on simple lookups; quality hinges on extraction. | Whole-corpus "what are the themes / how does it all connect" questions over a large, static corpus. |
My honest advice mirrors the last two posts: start with the cheapest thing that works. Hybrid search plus a reranker answers most questions, and it costs almost nothing. Only when you can point at the exact questions it's failing on — the sweeping "summarize everything" ones, or the "how do these connect" ones — is it worth paying for a summary tree or a knowledge graph. The structure is powerful, but you pay for it up front, every time the corpus changes.
The through-line across all of this: RAG quality is mostly about giving the model the right shape of context. Flat chunks are one shape. A tree of summaries is another. A graph of relationships is a third. Pick the shape your questions actually need.
Thanks for reading, and hope you got something out of it.