My last post walked a progression: plain RAG, then RAPTOR, then GraphRAG — each one adding more structure on top of the last. But look closely and the foundation never moved. All of them share the same underlying engine: chop text into chunks, turn them into vectors, match by similarity.
PageIndex — from Vectify AI, open-sourced in September 2025 — asks a more heretical question: what if we skip that engine? Its pitch is right on the tin: no embeddings, no chunking, no vector DB. Instead of shredding your document into a pile of vectors, it turns it into something you already understand — a table of contents — and instead of matching a query by similarity, it hands that table of contents to an LLM and lets it reason about where to look, the way you'd flip through a report yourself.
The whole case for PageIndex fits in one line the authors keep repeating: similarity ≠ relevance. Cosine search is very good at finding text that looks like your query. But the passage that looks most like the question often isn't the one that answers it — especially in long, dense professional documents, the kind where every sentence is packed with domain terms.
Take a 180-page annual report and ask "what's the methodology behind the goodwill impairment?" The paragraph you need might read "see Appendix G for the impairment methodology" — which shares almost no words with a chunk that actually explains it, so similarity sails right past the pointer. Three of the failure modes PageIndex calls out hide in questions like this:
1 · Chunk boundaries. Fixed token length cuts slice a table or an argument in half, so no single chunk holds the whole thought.
2 · Cross-references. "See Appendix G", "as noted in Section 4.2" — the answer lives somewhere a keyword match will never reach.
3 · Intent vs. content. A query expresses what you want, not the words of the answer. "Is Nimbus's revenue at risk?" won't cosine-match the risk section, which never brags about revenue.
Here's the thing: you as human don't have this problem. Handed a 180-page report, you don't scan all 180 pages for lexical overlap. You open the table of contents, jump to "Risk Factors," read it, and if it says "see Appendix G" you follow the pointer. PageIndex's entire bet is that an LLM can do exactly this — navigate a document by its structure and reason about where the answer lives — and that beats similarity match.
PageIndex has two phases, like GraphRAG: an indexing phase that reads a document and turns it into a tree, and a query phase that reasons over that tree. The index is not a vector store — it's a plain JSON file that mirrors the document's own outline. Here's how it gets built (following the open-source repo, MIT-licensed):
1Start by not shredding. Where vector RAG's first move is to cut the PDF into hundreds of fixed-size chunks and embed each one, PageIndex does neither. It keeps the document whole and starts the way a person would — by reading its structure.
2Detect the structure. An LLM scans the opening pages (default: first 20) for a real table of contents. If one exists, it's used; where structure is missing, the model infers the section boundaries itself from the content (see details in below note).
3Build nodes, recursively. Every section and subsection becomes a node, with its page range recorded, and an LLM writes a one-line summary for each. Nodes are capped by size (defaults: ≤10 pages / ≤20k tokens), so a fat section splits into child nodes — giving a tree that's shallow at the top and detailed toward the leaves.
4The index is a JSON tree. The whole document becomes a nested tree of title / node_id / page range / summary / child nodes — human-readable, no vector database anywhere. Each node_id maps straight back to its raw page text, so once you've picked a node you get the real content, never a lossy summary.
That's the whole index: a tree that is the document's outline, with a summary hung on each branch. Notice what you didn't do — no embedding model, no chunk-size tuning, no vector store to stand up. And notice what you can do that you couldn't before: read the index and understand it at a glance. Now let's use it.
This is where PageIndex differs the most from everything in the last post. There's no query embedding, no cosine, no top-k. Retrieval is LLM tree search: the model is handed the whole tree and reasons its way to the right nodes, exactly like a person skimming a table of contents. Take "What caused the EU outage, and how did it hit revenue?"
1The whole tree goes into the prompt. No embedding, no vector search. The entire table of contents — every title, summary, and node_id — is small enough to fit in the context window, so it's handed to the LLM whole, alongside the question. Out comes JSON: a reasoning trace plus the list of node_ids to open.
2Reasoning you can read. The node_list is the pick; the thinking trace says why each section made the cut and why the rest were skipped. Mapped back onto the tree, two sections light up — and unlike an opaque cosine score, you can audit the call.
3Pull the real pages. Each picked node_id maps back to its actual page text. PageIndex loads the full content of those sections — not a summary, not a boundary-sliced chunk — so the model answers from complete, in-context passages.
4Answer, grounded and cited. Those same sections go to an LLM, which writes the answer citing the exact pages. In the open-source version this is a single pass — one tree-search call picks the nodes, then the answer gets written. Every claim traces to a specific page — a paper trail vector RAG can't produce.
That flow — hand over the whole tree → reason → pick sections → read → answer — is the whole idea, and it's pure reasoning over structure. No vector ever gets computed at query time.
The open-source repo does exactly the single pass above. The hosted product turns that one pick into a genuine search — and the model it borrows from is the one behind AlphaGo: a value-function-guided tree search. The idea, conceptually:
1Why search at all? On a small document the single pass we just saw is fine — one LLM call reads the whole tree. But trees get deep, and across a big collection, huge. Running a full LLM reasoning call on every branch to decide what's worth opening gets slow and expensive. So the hosted version searches instead of reading everything.
2Score each branch cheaply. A lightweight value function rates every branch for how likely it holds the answer — a fast estimate, not an expensive reasoning pass. It's the move behind AlphaGo: judge how promising a position is without playing every game out to the end.
3Expand the promising branches, skip the rest. The search descends into the high scorers — MD&A and Risk Factors — scoring their children in turn, while the low-value Business branch is pruned untouched. The expensive reasoning goes only where the payoff looks high, not evenly across the whole tree.
4Land on the answer, then stop. The best leaves — Revenue and Outages — are the sections pulled for the answer. An agent checks after each round whether it has gathered enough, and once it has, it writes the grounded, cited answer instead of expanding further.
One honest caveat: that's the concept, not an official spec. PageIndex names its production method "value-function-based Monte-Carlo tree search" but says the mechanics are "coming soon" — so treat the steps above as the general idea it's built on, not their published algorithm.
Everything so far worked on a single document. But the reason enterprises reach for PageIndex — legal discovery, financial filings, compliance — is corpora: thousands or millions of documents at once. A per-document method runs straight into a wall here: you obviously can't stuff every document's tree into one prompt. PageIndex's answer is its File System (an Enterprise/Cloud feature, not the open-source repo), and the idea is pleasingly recursive — a tree of trees.
1Every document gets tags. Phase 1 already gave each document its own table-of-contents tree. Now, as a document is added, an LLM also reads it and labels it with metadata tags — a topic or category, plus fields like vendor, region, or year. The Annual Report comes out tagged Financials, Risk & Legal, and EU; a supplier contract gets Risk & Legal and EU.
2Each tag is a folder. Here's the part worth pinning down: a folder isn't a place a file sits in — each distinct tag simply becomes a folder. The Financials folder is just "every document tagged Financials"; EU is every EU-tagged doc. So a document hangs under every folder whose tag it carries — the Annual Report is under all three at once (the blue edges). Those tag-folders, the documents, and each document's own sections wire up into one continuous tree.
3Search descends it, pruning as it goes. The value-guided search from the last section starts at the root. It scores the tag-folders and documents by their labels: the topic folders Financials and Risk & Legal light up, the weaker EU folder and the off-topic MSA contract are pruned, and the Annual Report — under both hot folders — is the pick. Only then does it drop into that document's own section tree for the exact pages. Corpus → folder → document → section, with most of the million trees never touched.
So yes — the value-function search is what answers the scaling question too, and that's the whole elegance of it. Because the corpus folders, the documents, and their sections are one tree, a single search policy runs from topic all the way down to paragraph, pruning hard at every level so the LLM only ever reads the sliver that matters. It's the same recursion PageIndex used to index one document, applied one level up to index a whole library.
One thing the flat picture above understates: the folder layer needn't be a single row. Those tags nest — a broad Finance folder can hold narrower Filings and Statements sub-folders beneath it — and the different facets (topic, region, year) give overlapping hierarchies a document threads through at once. So it's less "one shelf of folders" and more a multi-level directory the search prunes down through, exactly like the sections inside a single document. (How deep PageIndex actually builds it isn't published — treat depth as illustrative.)
The upside. On its home turf — long, structured professional documents — PageIndex delivers. Its headline result is Mafin 2.5, a retrieval system built on it, which scores 98.7% on FinanceBench — up from 38% for an earlier version. You get that without an embedding model, a chunk size to tune, or a vector database to stand up and keep in sync — the index is a plain, readable tree. Answers come back cited to the exact page.
The cost. A lot of work at query time, and that isn't free:
We've seen how PageIndex works, scales, and costs. The last question is how it stacks up against the alternatives — plain vector RAG, and RAPTOR and GraphRAG from the last post. They rhyme on the surface: most add structure an LLM built, then retrieve over it. But the machinery pulls apart in ways worth seeing. Same question, four approaches:
1The sharpest contrast — plain vector RAG. Cosine grabs whatever chunk looks most like the words, even the wrong, boundary-sliced "revenue" one, and misses the causal link. PageIndex reasons that the outage lives in Risk Factors and its revenue effect in MD&A, and reads both whole sections. The rest of the field sits between these two poles.
2Four structures, built four ways. RAG adds none — just flat chunks. The other three add structure an LLM built, but from opposite directions: RAPTOR invents a tree bottom-up by clustering vectors, GraphRAG extracts an entity graph across the corpus, and PageIndex takes the document's own table of contents as-is. A statistical artifact, a web of entities, and the author's outline.
3Three of the four still bottom out in a vector. RAG and RAPTOR rank by cosine; GraphRAG's local search starts with an entity-embedding lookup, then walks edges mechanically. PageIndex never embeds anything — retrieval is an LLM reasoning over structure, in the spirit of the LLM wiki's agent deciding which page to read next.
4And they aim at different targets. RAG for pinpoint facts anywhere, RAPTOR for summarizing length, GraphRAG for corpus-wide sensemaking, PageIndex for deep, precise answers inside one long document. Different jobs, not just different engines — match the tool to the question you actually get.
So the family resemblance is real but shallow. The three structured approaches all give more traceable answers than flat top-k — you can point at what got retrieved and why. But RAPTOR and GraphRAG keep the vector engine and bolt structure beside it; PageIndex rips the engine out and makes reasoning over the document's own structure the entire mechanism, and it's the only one aimed at going deep inside a single long document rather than sweeping a corpus. Which leaves just one question — when is that trade worth making?
The last post covered three approaches; PageIndex adds a fourth that leans in a new direction — it isn't more structure so much as a sideways step, trading the vector engine for reasoning. Here's the full board:
| Approach | Structure it adds | Retrieval | Best for | Main cost |
|---|---|---|---|---|
| Normal RAG | flat chunks + vectors | Cosine top-k | Pinpoint facts across any corpus | Blind to structure & the big picture |
| RAPTOR | bottom-up summary tree | Cosine over all levels | Summarizing long documents | Still similarity; summaries lose detail |
| GraphRAG | entity graph (corpus) | Traversal + map-reduce | Whole-corpus sensemaking & multi-hop | Very expensive to build & update |
| PageIndex | table-of-contents tree | LLM tree-search reasoning | Precise, cited Q&A over long structured single docs (finance, legal) | Slow & costly per query; needs real structure; multi-doc scale is a paid add-on |
My honest take: PageIndex isn't a replacement for vector RAG, and the "vectors are dead" framing oversells it. It's a precision instrument for a specific job — going deep into a long, well-structured, high-stakes document where a wrong answer is expensive and an auditable one is worth paying for. That's exactly why finance and legal are its home turf. For a support FAQ or a consumer chatbot, cosine top-k is still the right, cheap answer.
Which lands back on the through-line from the last post: RAG quality is mostly about giving the model the right shape of context. Flat chunks are one shape, a summary tree another, a graph a third. PageIndex's bet is that for a long professional document, the best shape was there all along — the author already wrote the table of contents. Don't approximate it with clusters or graphs. Just read it like a human would.
Thanks for reading, I'll keep updating as the reasoning-RAG space shakes out.