RAG or Graph? The right question is: what problem are you trying to solve?

RAG resolves the recovery of semantically relevant content in unstructured corpora. A knowledge graph resolves navigation over explicit relationships between entities. They are different primitives, with different guarantees, and the choice between them should not be made at the technology level. It should be made at the query level that the system needs to answer.

I will address this with the rigor that the decision deserves: complete pipeline, guarantees and limitations of each approach, when to combine them in GraphRAG, and what changes when the system moves from prototype to production with multiple tenants, latency SLAs, and audit requirements.

RAG solves a specific problem: semantic relevance over text

RAG exists to answer a class of questions: "what does my content say about this." The actual pipeline, not the simplified version of "generate embedding and query vector database," has more stages than most initial implementations assume:

Sources → Ingestion → Parsing → Normalization → Chunking →
Metadata Enrichment → Embedding → Indexing → Retrieval →
Filtering → Reranking → Context Assembly → Generation → Citations → Evaluation

Each stage carries decisions that determine whether the system works in production or only in the demo:

Chunking. Fixed-size token chunking is the simplest approach and the one that most degrades relevance in structured documents (contracts, policies, regulations). It is worth separating three strategies that are often treated as synonyms but are not. Structure-aware chunking respects the structural boundaries of the document, title, section, paragraph, and avoids cutting a clause in half. Semantic chunking uses meaning similarity between consecutive excerpts to decide where one idea ends and another begins, regardless of the visual structure of the document. Hierarchical chunking preserves the clause-section-document relationship as a navigable tree, which often surpasses flat chunking in legal and regulatory content. Well-calibrated chunk overlap, in any of the three strategies, reduces context fragmentation at the edges.

Retrieval. Pure vector search (dense retrieval) has a known blind spot: it fails on queries with exact terms, acronyms, contract numbers, or specific domain nomenclature, because embeddings capture semantic similarity, not exact lexical matching. Therefore, hybrid search combining dense retrieval with BM25 (sparse retrieval) has become one of the main strategies in corporate RAG, and is often a more robust baseline than isolated dense retrieval when the corpus has identifiers, codes, and domain nomenclature.

Reranking. Initial retrieval optimizes for recall, not precision. A reranker (cross-encoder) applied to the top-k retrieved corrects a good portion of the false positives that initial retrieval brings due to vector proximity without real relevance.

Vector database. Before introducing a dedicated vector database (Qdrant, Pinecone, Weaviate, Milvus), I evaluate whether PostgreSQL with pgvector meets volume, QPS, and metadata filtering requirements. A separate vector database adds operational surface, another system to monitor, back up, and keep consistent. It is only justified when volume, dimensionality, or throughput truly demand it.

The structural point is this: RAG retrieves relevant content for a question. It does not know, and should not know, if two retrieved excerpts represent a real business relationship between entities. Vector proximity between "Client A" and "Contract 93821" does not imply that Client A possesses Contract 93821. This is a statistical inference about text, not a factual statement about data. Treating semantic similarity as if it were a business relationship is a recurring source of silent hallucination in corporate RAG systems.

Graph resolves another problem: navigation over explicit relationships

When the question changes form, "which clients have contracts associated with product X and were impacted by process Y," RAG is not the right tool. This question does not ask for a semantically similar excerpt. It asks for a verifiable path in an entity graph:

Client --has--> Contract --uses--> Product --depends_on--> Process --executed_by--> System

An entity graph (or knowledge graph, the formal name of the concept) models knowledge as entities, typed relationships, and attributes, typically persisted in a graph database (Neo4j, Amazon Neptune, TigerGraph) or, on a smaller scale, as relational tables with strong foreign key modeling when the graph is shallow and the query pattern is predictable. The choice between a native graph and a relational model depends on traversal depth: for one to two hop queries, a well-indexed relational database suffices. For multi-hop traversals, variable paths, and impact analysis (blast radius), a native graph database with a query language like Cypher or Gremlin avoids costly recursive joins and makes the query readable and performant.

The real gain of a relationship graph appears in impact and dependency questions:

"Which clients could be impacted if the Anti-Fraud Model V4 becomes unavailable?"

Anti-Fraud Model V4 <--used_by-- System <--depends-- Product <--associated_with-- Contract <--has-- Client

This is a reverse traversal over a dependency graph, the type of query for which vector search simply does not have an equivalent primitive. Domains where this capability often justifies the investment: finance (risk, anti-fraud, regulated products), insurance (claims and interconnected policies), telecom (network topology and outage impact), health (relational clinical history), supply chain (supplier traceability), and compliance (mapping regulatory exposure).

GraphRAG: when the question requires content and relationship at the same time

The question that exposes the limitation of treating RAG and Graph as mutually exclusive is this: "which clients are impacted by the regulatory change described in Resolution X." It has two parts that require different mechanisms. The first part, "what does the resolution say," is document retrieval. The second, "which clients, contracts, and products are related to this rule," is graph traversal. A GraphRAG architecture can combine both mechanisms to answer composite questions like this in a single retrieval layer:

                         User's Question
                                 |
                        Query Understanding
                                 |
                 -----------------------------------
                 |                                 |
         Semantic Search                     Graph Traversal
                 |                                 |
        Document Content                     Relationships
                 |                                 |
                 -----------------------------------
                                 |
                          Context Builder
                                 |
                                LLM
                                 |
                    Evidence-based response + citations

The point that often goes unnoticed is that GraphRAG does not replace embeddings; it orchestrates two complementary retrieval mechanisms. Vector search finds semantic similarity in unstructured text. Graph navigates explicit and verifiable relationships. A mature corporate architecture tends to compose various retrieval sources under a query router:

Vector Search + Keyword Search (BM25) + SQL + Graph + APIs + Business Rules

with the LLM acting as a query understanding layer, planning which sources to trigger and composing the final response, not as a source of truth about business relationships. Allowing the model to infer relationships between entities from textual proximity, instead of querying the graph, opens the door to exactly the type of hallucination that the architecture should prevent.

A caution with the term GraphRAG

The diagram above describes the most common architectural pattern of graph-augmented RAG. But the term GraphRAG also names a specific reference implementation, published by Microsoft, that goes beyond this composition.

In it, an indexing pipeline extracts entities, relationships, and claims from the entire corpus using LLM, organizes these entities into a hierarchy of communities, and generates summaries by community. The query layer offers distinct modes: Global Search, for holistic questions about the entire dataset, navigating community summaries; Local Search, which expands from specific entities and their direct neighbors in the graph; and DRIFT Search, which combines the two, starting from community summaries to generate refined sub-questions through local search.

If the goal is summarization and reasoning over an entire corpus, this hierarchical approach solves a problem that the simple composition of vector search plus graph traversal does not solve alone. If the goal is to answer specific questions that cross document content with structured relationships, the simpler composition is often sufficient and cheaper to operate.

Structural Comparison

Vector similarity is not truth, it is hypothesis

It is worth reinforcing this because it is the most recurring architectural error I see in corporate RAG implementations. Two texts may be close in vector space without any real business relationship between them. "Client A," "Credit Product," and "Contract 93821" may appear semantically close in an embedding without that meaning Client A --has--> Contract 93821. This statement is only reliable when it comes from a structured source of truth, whether a transactional database, a CRM, or a curated graph. And the same caution applies to the graph itself: a graph populated by automatic extraction of entities and relationships via LLM, without a validation step, carries equivalent risk of incorrect relationships, poorly done entity resolution, and outdated data. The reliability of the graph does not come from the fact that it is a graph; it comes from the quality of the process that feeds it. Treating embeddings as a source of business facts is the type of decision that goes unnoticed in demonstration and becomes a reliability incident in production, because the model will respond confidently about a relationship that never existed.

Governance is what separates demo from production system

In demonstration, it is enough for the answer to seem correct. In corporate production, the system needs to answer questions that the demo never asked: where did this information come from, which document and which version were used, which data originated this relationship, who has permission to access this information, when was it updated, and if the answer is reproducible given the same data state. This is not an implementation detail; it is a first-class non-functional requirement in any corporate AI architecture: lineage, index and graph versioning, tenant authorization, auditing, data quality and provenance, observability of the retrieval pipeline, and retention policies. In regulated sectors, knowing how the answer was constructed is as relevant as the answer itself, because it is what sustains an audit or a contractual dispute.

A direct heuristic for decision-making

I need to discover what documents say: RAG.

I need to discover how entities are related: Graph.

I need to understand content and then navigate the relationships it triggers: GraphRAG.

And the question that comes before any of these three: is this complexity justified by the volume and real criticality of the questions that the system will answer, or am I adding a retrieval mechanism because it is in vogue? If the vast majority of the system's questions are answered by document search, introducing a graph database, an entity extraction pipeline, and a routing layer adds permanent operational cost to solve a small fraction of cases. Mature architecture includes knowing when not to add a technology.

Conclusion

RAG, Graph, and GraphRAG do not compete with each other because they solve different classes of knowledge retrieval problems. RAG answers what the content says. Graph answers how entities relate. GraphRAG answers both things at the same time when the business question truly requires it. The architectural decision that separates a