How to Use Knowledge Graphs for RAG (GraphRAG): A Complete Implementation Guide

GraphRAG enhances Retrieval-Augmented Generation by structuring external knowledge as entities and relationships, enabling multi-hop reasoning and hierarchical summarization that pure vector search cannot achieve.

Knowledge graphs for RAG transform unstructured documents into structured semantic networks where nodes represent entities and edges encode relationships. This approach—implemented in the bojieli/ai-agent-book repository—eliminates the ambiguity of keyword matching while supporting complex, multi-hop queries across interconnected concepts. By combining graph traversal with community detection and hierarchical summarization, GraphRAG delivers more precise context to large language models than traditional dense retrieval alone.

Core Benefits of GraphRAG

Using knowledge graphs for RAG provides three distinct architectural advantages over flat vector databases:

• Structured semantics – Graph nodes capture discrete entities (e.g., CPU registers, instructions) while typed edges encode specific relationships (e.g., "belongs-to", "depends-on"). This eliminates the semantic drift common in pure text similarity search.

• Multi-hop reasoning – Traversing graph edges allows retrieval of indirect, higher-order connections. A query can move from a register to its containing subsystem to all dependent instructions, uncovering relationships invisible to single-pass vector retrieval.

• Efficient summarization – Community detection algorithms group tightly related nodes into clusters, enabling hierarchical summarization that produces concise abstracts for each sub-graph. This dramatically reduces token consumption during LLM generation.

Architectural Pipeline

The GraphRAG implementation in chapter3/structured-index/ follows a six-stage pipeline orchestrated through main.py.

Document Ingestion and Chunking

Raw PDF or Markdown documents first undergo semantic chunking with a soft limit of 4096 characters while respecting paragraph boundaries. The chunker in chapter3/structured-index/main.py prepares text segments for graph construction without breaking semantic continuity.

Entity and Relation Extraction

Each chunk feeds into the GraphRAGIndexer class defined in graphrag_indexer.py. This wrapper implements named-entity recognition to identify nodes and dependency parsing to infer typed, weighted edges between entities. Source metadata including chunk IDs and page numbers attaches directly to graph elements for provenance tracking.

Graph Construction and Enrichment

The system constructs a property graph where:

• Nodes represent extracted entities
• Edges represent inferred relationships with type and weight attributes

Post-processing invokes detect_communities() to cluster related entities (e.g., all registers belonging to the same functional unit) and hierarchical_summarization() to generate textual abstracts for each community.

Index Persistence

The fully materialized graph and its community summaries serialize to indexes/graphrag/ via save_index(), enabling fast reloading without reprocessing source documents.

Query-Time Retrieval

At inference time, the pipeline accepts natural language queries, performs graph traversal (single-hop or multi-hop), retrieves relevant node IDs, fetches associated text chunks, and forwards the assembled context to the LLM for generation.

Building a GraphRAG Index

Constructing a knowledge graph index requires executing the build command through the CLI entry point.

python main.py build path/to/document.pdf --type graphrag

This command triggers the following method chain inside graphrag_indexer.py:

from graphrag_indexer import GraphRAGIndexer
from config import get_graphrag_config

cfg = get_graphrag_config()
indexer = GraphRAGIndexer(cfg)

# Execute full indexing pipeline

indexer.build_knowledge_graph(document_text)  # Extract entities and relations

indexer.detect_communities()                  # Cluster related entities

indexer.hierarchical_summarization()          # Generate community summaries

indexer.save_index()                          # Persist to indexes/graphrag/

The build_knowledge_graph() method handles entity extraction and relationship inference, while detect_communities() applies graph clustering algorithms to identify semantic groupings. The resulting index supports both local file storage and optional caching in chapter3/structured-index/cache/ for intermediate chunking results.

Querying with GraphRAG

Single-Hop Retrieval

For direct entity lookup, execute a single-hop query:

python main.py query "SSE registers" --type graphrag

This retrieves nodes directly connected to "SSE registers", fetches their underlying text chunks, and injects them into the LLM context window.

Multi-Hop Reasoning

To leverage graph traversal for indirect relationships, enable multi-hop expansion:

python main.py query "CPU register relationships" \
    --type graphrag --multi-hop 2

The --multi-hop 2 flag instructs the system to follow up to two edges from the initial entity match. For example, the query might traverse from a CPU register to its associated instruction, then to the containing functional unit, retrieving higher-order dependencies unavailable through vector similarity alone.

Programmatic API Access

Integrate GraphRAG directly into Python applications:

from graphrag_indexer import GraphRAGIndexer
from config import get_graphrag_config

cfg = get_graphrag_config()
indexer = GraphRAGIndexer(cfg)

# Load existing index or build new

indexer.build_knowledge_graph(document_text)
indexer.detect_communities()
indexer.hierarchical_summarization()
indexer.save_index()

# Search with graph traversal

answers = indexer.search("instruction relationships", top_k=5)
print(answers)

The search() method handles graph traversal and context assembly internally, returning ranked text chunks ready for LLM consumption.

Hybrid Retrieval: Combining GraphRAG with Dense Vectors

The repository implements hybrid RAG that combines relational graph traversal with semantic vector re-ranking. After graph retrieval identifies candidate nodes, dense embeddings re-rank the results for optimal relevance.

Execute hybrid retrieval using the --rerank flag:

python main.py query "Cache coherence protocol" \
    --type graphrag --multi-hop 2 --rerank hnsw

This pipeline first performs multi-hop graph traversal to identify relevant entity communities, then applies HNSW (Hierarchical Navigable Small World) vector search to re-rank the extracted chunks. This dual-modality approach ensures both structural relationship accuracy and semantic content relevance.

Key Implementation Files

The GraphRAG system spans several critical files in chapter3/structured-index/:

• main.py – CLI entry point parsing build and query commands, dispatching to appropriate indexers based on the --type flag.

• graphrag_indexer.py – Core wrapper implementing GraphRAGIndexer class with methods for build_knowledge_graph(), detect_communities(), hierarchical_summarization(), and search().

• config.py – Configuration management for chunk sizes, embedding models, storage paths (indexes/graphrag/), and backend selection (GraphRAG vs. RAPTOR tree index).

• README.md – User-facing documentation detailing command-line flags, usage examples, and external project references.

• IMPLEMENTATION_GUIDE.md – Developer documentation mapping each GraphRAG method to pipeline stages and explaining community detection algorithms.

Summary

• GraphRAG structures knowledge as entities and relationships rather than flat text chunks, enabling precise semantic retrieval through graph traversal.

• Multi-hop reasoning follows edges to uncover indirect connections between concepts, supporting complex queries impossible with vector similarity alone.

• Community detection and hierarchical summarization reduce context volume by clustering related entities and generating concise sub-graph abstracts.

• Hybrid retrieval combines graph traversal with dense vector re-ranking (--rerank hnsw) to maximize both relational and semantic relevance.

• The bojieli/ai-agent-book repository provides a complete implementation via GraphRAGIndexer in chapter3/structured-index/graphrag_indexer.py, supporting both CLI and programmatic Python APIs.

Frequently Asked Questions

What is the difference between GraphRAG and traditional vector RAG?

Traditional vector RAG retrieves context based on embedding similarity between queries and text chunks, which can miss relational nuances and indirect connections. GraphRAG explicitly models entities as nodes and relationships as edges, enabling structured traversal that captures "depends-on" or "belongs-to" semantics. This structural awareness eliminates ambiguity in entity references and supports logical reasoning across multiple hops.

How does multi-hop reasoning work in GraphRAG?

Multi-hop reasoning executes graph traversal across multiple edge levels to retrieve indirectly connected information. When using the --multi-hop 2 flag, the system first identifies entities matching the query, then follows up to two relationship edges outward to collect related nodes. For example, querying "CPU registers" with multi-hop might retrieve not just register definitions, but also their parent subsystems and dependent instructions, building a comprehensive context graph for the LLM.

What are communities in GraphRAG and why are they important?

Communities are clusters of densely connected nodes identified by the detect_communities() method during index building. These groups represent semantic themes (e.g., all components of a memory management unit). The hierarchical_summarization() method generates concise abstracts for each community, enabling the retrieval system to return high-level summaries rather than raw text chunks. This compression reduces token usage while preserving essential relational context.

Can GraphRAG be combined with other retrieval methods?

Yes, the implementation supports hybrid retrieval through the --rerank parameter. After GraphRAG identifies candidate nodes via graph traversal, the system can apply HNSW dense-vector re-ranking to optimize semantic relevance. This combination leverages the relational precision of knowledge graphs with the semantic flexibility of embedding-based search, configurable through config.py and the CLI interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}