GraphRAG Retrieval Is Three Decisions: Granularity, Mechanism, and Paradigm

Traditional vector RAG retrieves by embedding the question and finding semantically similar chunks, often augmented with lexical search, filtering, or reranking. This approach works when the answer is explicitly described in one or more chunks. However, it breaks down when the answer depends on relationships between facts. 

The question "Does my application depend on a compromised package?" illustrates this limitation. The vulnerable package may be several layers deep in the dependency tree, and no single chunk contains the answer. The answer emerges by following a chain of dependencies, but similarity search can struggle because the answer is distributed across multiple relationships rather than represented as a single semantic concept.

GraphRAG addresses this issue by retrieving from a knowledge graph, where entities are connected through explicit relationships. However, GraphRAG is not a replacement for vector RAG. It is a retrieval paradigm for problems where relationships, graph structure, and provenance matter. For many applications, vector retrieval remains the fastest and most effective way to find semantically relevant information, while graph retrieval adds value when the answer depends on connected facts.

But graph retrieval isn't a single technique; it requires deciding what to retrieve, how to retrieve it, and whether retrieval should happen in one pass or multiple stages.

If these decisions are made correctly, GraphRAG can answer questions that vector search cannot; otherwise, it may result in a system that's slower and no more accurate than a well-designed vector RAG.

This article focuses on retrieval. Graph construction is a separate issue, and schema quality affects everything. Here, we assume a well-built graph. The examples are based on a small software supply-chain graph.

It's fictional, but the shape is from a real incident. In 2018, the npm package event-stream was compromised after a malicious dependency, flatmap-stream, was introduced into its dependency chain. The challenge is discovering the path shopping-app -> analytics-js -> event-stream -> flatmap-stream and connecting it to CVE-2024-1. No single text chunk contains this chain, and none of those package names indicate "compromised package."

Scanners such as npm audit answer this question easily because they're built for this structure. GraphRAG can answer broader relationship questions around the same graph structure, especially when the answer requires combining multiple sources.

Decision 1: Granularity

Graph retrieval returns one of four units, from finest to coarsest.

1. Node (for example, flatmap-stream and its attributes). 
2. Triplet (for example, flatmap-stream -[HAS_VULNERABILITY]-> CVE-2024-1). 
3. Path: like the red route. Paths are good for multi-hop and provenance questions. 
4. Subgraph: a connected region, like the payment component. 

Finer units are precise, but coarser units have more context, and more noise. Choose a unit that matches the question. Each mechanism, described in the next section, produces some units more naturally than others.

Decision 2: The Six Mechanisms

1. Similarity

Embed graph elements, retrieve the ones nearest the question vector. Most systems use this to find starting points. It answers "find things like this" questions on its own.

CALL db.index.vector.queryNodes('pkgEmbeddings', 5, $questionVector)
YIELD node, score
RETURN node.name, score

A simple query like "Which of our packages resemble this known-bad one?" is all that's needed. However, similarity over plain text embeddings has its limitations; it's blind to structure. So it only finds nodes that match the query and misses connected facts that don't resemble it. This method is useful for locating entry points and serving semantic lookups, but for anything that requires multiple hops, it's best to hand off to a structural mechanism.

2. Structural Traversal

To get around the limitations of similarity searches, walk outward from the entry points, using techniques like neighbor expansion, breadth-first or depth-first search, and pathfinding. The granularity of what you collect depends on the approach; collecting neighbors gives you a local subgraph, while tracing routes between two entities gives you paths.

// Entity-centric: what does the app build on? (a neighborhood)
MATCH (:Package {name: 'shopping-app'})-[:DEPENDS_ON*1..2]->(dep:Package)
RETURN DISTINCT dep

// Connection question: how does the app reach vulnerable packages?
MATCH p = (:Package {name: 'shopping-app'})-[:DEPENDS_ON*..5]->(bad:Package)
WHERE (bad)-[:HAS_VULNERABILITY]->(:CVE)
RETURN p

For instance, the second query might return shopping-app -> analytics-js -> event-stream -> flatmap-stream, along with any other route to a vulnerable package. When the question is about risk, you want every possible path, not just the shortest one, because a second route is a second exposure.

For large graphs, it is often more efficient to start from known vulnerable nodes and traverse backward, or constrain the search from the application side, depending on the query. Traversal is the cheapest, fastest, and easiest mechanism to explain, since you can read the route. But it has a weakness: fan-out. The practical depth limit depends on how constrained the walk is; unconstrained neighbor expansion grows rapidly, so shallow expansion is often preferred. A typed, direction-constrained path search, like the one mentioned earlier, prunes most of that growth and remains tractable deeper, which is why a four-hop dependency chase basically works in this case, but a generic four-hop expand-everything doesn't. Use traversal when the question is anchored on specific entities, and it's the best approach.

3. Graph Algorithms

Two things are needed for traversal: a starting point, and a rule for edge selection. But what if you're missing one or both? There are two graph algorithms that can help. 

Personalized PageRank is useful when you have a starting point but no rule. It assigns a score to the mentioned entities, lets the score spread across the edges, and ranks nodes by the score they receive. Nodes that are highly reachable from the seeds through many strong paths receive higher scores. This helps find relevant nodes even if they're far away and don't share any words with the question. HippoRAG uses this for retrieval.  

CALL gds.pageRank.stream('supplyChain', {
  sourceNodes: $seedEntities, dampingFactor: 0.85 })
YIELD nodeId, score
RETURN gds.util.asNode(nodeId).name AS entity, score
ORDER BY score DESC LIMIT 10

For example, on our graph, if we seed analytics-js and CVE-2024-1, flatmap-stream receives a high score because it is strongly connected to both seed regions. Nothing in the query named it, though. Furthermore, directionality matters. For dependency graphs, reverse traversal or an appropriate projection is often required because vulnerabilities may be sink nodes.

Community detection is especially useful for broad exploratory questions where no specific entity is known. A question like "what are the main risk areas across our dependencies?" is about the whole graph. The approach is to cluster the graph into communities, have an LLM summarize each community, and answer broad questions from those summaries. This is a key component of the Microsoft GraphRAG approach. On our graph, clustering gives us three communities: a payment-and-media stack, a web-framework stack, and the analytics subsystem with the compromised package.

Cost is the main difference between the two. PageRank is expensive at query time because its scores depend on the seeds and can't be precomputed. On a large graph, you need to bound the projection to a region found by a similarity pass first. Community detection is expensive at index time because the LLM summaries are costly to compute and need to be redone when the graph changes. But queries after that are cheap. So pick PageRank when you have entities but no target. Pick communities when there's no starting entity.

4. Declarative Query

With exact structural constraints in the question, it's best to query the graph directly. The model translates the question into a query language, like Neo4j's Text2Cypher. Schema grounding and validation significantly improve generated queries.

// "Which volunteer-maintained packages also have a known CVE?"
MATCH (pkg:Package)-[:MAINTAINED_BY]->(:Maintainer {kind: 'volunteer'})
MATCH (pkg)-[:HAS_VULNERABILITY]->(cve:CVE)
RETURN pkg.name, cve.id

This returns results like flatmap-stream / CVE-2024-1, combining a maintainer condition and a vulnerability condition that fuzzy mechanisms can only approximate, resulting in an exact and auditable outcome. This risk lies in the generation step, with models often inventing relationship types that sound right but don't exist in the schema; a well-formed query over imaginary edges returns zero rows without error. 

To mitigate this, ground the generation in the actual schema, validate the query before running it, and treat an empty result as a signal to fall back to another mechanism. Use this approach for questions that reduce to filters, counts, or joins across relationship types.

5. Generative Retrieval

This mechanism works in two stages: first generating a retrieval plan that defines the relationship pattern to follow, and then translating that plan into a graph query. Reasoning on Graphs (RoG) works this way, with an LLM generating planning paths and the system retrieving the concrete paths that satisfy them. 

plan  ← LLM("what relation path answers this?", schema)
        → "DEPENDS_ON* , then HAS_VULNERABILITY"
paths ← graph.match(seed='shopping-app', pattern=plan)

The model infers the shape of the traversal, and the graph supplies the instances. This approach fits questions where the right pattern isn't known in advance, and you don't want to hand-write a template for it. It relies on the model understanding the schema, and a one-shot plan can't correct itself unless you make the retrieval iterative.

6. Learned Retrieval

A model can be trained to do the selecting, with a graph neural network scoring nodes for relevance to the question. Approaches such as G-Retriever formulate subgraph selection as an optimization problem, including variants inspired by Steiner tree formulations.

The retriever is a trained component; it embeds the question, scores candidate nodes and edges, and returns the highest-value connected subgraph as evidence. This approach has demonstrated strong accuracy on hard multi-hop benchmarks, but it comes with training, serving infrastructure, and transparency costs. They're suitable for accuracy-critical question answering over a stable schema, but they're rarely the first build.

Decision 3: The Paradigm

You've still got to decide how many times to go to the graph. This affects both latency and accuracy. A simple approach is one retrieval, gathering everything in a single pass, which keeps latency low and is suitable for real-time answers.  

Iterative retrieval is another option, involving multiple passes that build on each other, useful when one pass isn't enough. It comes in two versions: fixed-rounds and adaptive. The adaptive version stops once the model has gathered sufficient information. Most production systems opt for a multi-stage approach, chaining different mechanisms together. A common pattern is using similarity retrieval to find entry points, structural traversal to expand context, and reranking to select the final evidence set.

In practice, mechanisms usually combine in specific ways. Combining vector retrieval with graph retrieval is often described as HybridRAG. Letting a large language model plan the stages at query time is referred to as agentic retrieval, which is a composition pattern rather than a new mechanism.

Matching Questions to Mechanisms

A production system doesn't pick a mechanism per question at runtime. During the planning phase, you assess the questions, implement two or three mechanisms that cover them, and route between those in production. This table supports that assessment. 

Start with the basics. A similarity pass for entry points and structural traversal to expand. Add mechanisms as needed. PageRank or generative planning for harder questions, declarative queries for exact constraints, community summaries for thematic breadth, and learned retrieval when accuracy justifies it. 

Conclusion

Graph retrieval involves making three key decisions. First, you need to choose the right granularity; this could be a node, triplet, path, or subgraph, depending on the answer you're looking for. The mechanism is also crucial: it's about selecting the right approach, such as similarity, traversal, graph algorithms, declarative queries, generative planning, or learned retrieval. 

Then there's the model: whether to use a single-pass, iterative, or multi-stage approach. By making these decisions with your system's specific questions in mind, you can design a tailored retrieval architecture rather than relying on trial and error.