Implementing GraphRAG for Structured Knowledge Retrieval: A Complete Guide

GraphRAG transforms unstructured text into a searchable knowledge graph using LLM-powered entity extraction and multi-hop traversal, enabling agentic systems to retrieve structured, explainable evidence instead of relying solely on semantic similarity.

Implementing GraphRAG for structured knowledge retrieval involves building a pipeline that extracts entities and relations from raw documents, constructs a traversable graph, and exposes search capabilities via API endpoints. In the bojieli/ai-agent-book repository, the complete reference implementation lives in chapter3/structured-index/ and demonstrates how to integrate graph-based retrieval with agentic RAG systems.

What Is GraphRAG?

GraphRAG (Graph-based Retrieval-Augmented Generation) is a retrieval architecture that constructs a knowledge graph from source documents, then traverses that graph to answer complex, relationship-heavy questions. Unlike dense vector retrieval, which matches semantic similarity, GraphRAG follows explicit entity-relation paths to uncover connections between concepts that may never appear in the same text chunk.

Architecture Overview

The GraphRAG implementation in ai-agent-book follows a five-stage pipeline orchestrated by the GraphRAGIndexer class.

Chunking and LLM-Based Entity Extraction

Raw text is first split into fixed-size chunks according to the chunk_size parameter defined in GraphRAGConfig. Each chunk is sent to an LLM (via OpenRouter) which extracts structured entity-relation triples—subject, predicate, object—limited by the max_knowledge_triples setting.

Graph Construction with NetworkX

Extracted triples populate a NetworkX directed graph where nodes represent entities and edges encode relation types. This graph structure preserves directional relationships (e.g., "regulates", "governs", "implements") that vector databases typically lose.

Community Detection

The pipeline optionally runs community detection algorithms (such as Louvain) to group densely connected subgraphs. These communities enable hierarchical summarization and allow the system to return answers at varying levels of granularity.

Index Persistence

The constructed graph, community assignments, and per-node summaries are serialized to a binary .pkl file via GraphRAGIndexer.save(). Subsequent queries load this index using GraphRAGIndexer.load(), eliminating the need to rebuild the graph for every session.

Multi-Hop Search and Retrieval

The multi_hop_search() method performs graph traversal to find paths between entities. Instead of returning isolated text chunks, it delivers connected paths (e.g., "GDPR → Article 45 → Adequacy Decision → US") with natural-language summaries that provide clear provenance chains.

Core Implementation Files

The GraphRAG system is modularized across four key files in chapter3/structured-index/.

graphrag_indexer.py

The GraphRAGIndexer class in chapter3/structured-index/graphrag_indexer.py handles the complete lifecycle: entity extraction via LLM prompts, NetworkX graph construction, community detection, and the multi_hop_search() method for traversing entity connections.

config.py

Centralized configuration lives in chapter3/structured-index/config.py via the GraphRAGConfig dataclass. This defines model names, chunk sizes, maximum triples per chunk, community algorithms, and loads environment variables such as the OpenRouter API key. Access the config through the get_graphrag_config() helper function.

api_service.py

The FastAPI wrapper in chapter3/structured-index/api_service.py exposes two critical endpoints: POST /build for triggering index construction from uploaded documents, and GET /query for executing multi-hop graph searches. The service runs on localhost:8000 and returns JSON responses containing ranked paths and concatenated evidence strings.

main.py

The command-line interface in chapter3/structured-index/main.py provides a convenient entry point for building indexes from text files and running demonstration queries without writing custom Python scripts.

Building a GraphRAG Index

To construct a knowledge graph from your documents, install the dependencies and use the CLI orchestrator:

# Install GraphRAG dependencies

pip install -r chapter3/structured-index/requirements.txt

# Build index from plain text

python chapter3/structured-index/main.py \
    --mode build \
    --input docs/intel_sdm.txt \
    --output indexes/graph_rag.pkl

Behind the scenes, main.py instantiates GraphRAGConfig, creates a GraphRAGIndexer, calls indexer.build_from_file() to process the document, and persists the result using indexer.save().

Querying the Knowledge Graph

Load a persisted index and perform both simple entity lookups and complex multi-hop traversals:

from graphrag_indexer import GraphRAGIndexer
from config import get_graphrag_config

# Initialize configuration and indexer

cfg = get_graphrag_config()
indexer = GraphRAGIndexer(cfg)

# Load existing index

indexer.load("indexes/graph_rag.pkl")

# Simple entity search

results = indexer.search("PCI-DSS", top_k=5)

# Multi-hop traversal: find connections between entities

paths = indexer.multi_hop_search(
    start_entity="PCI-DSS",
    target_entity="data-encryption",
    max_hops=3,
    top_k=3,
)

for path in paths:
    print(" → ".join(path["entities"]), f"({path['relation_summary']})")

The multi_hop_search() method returns ranked paths containing the sequence of entities traversed and a natural-language summary of the relations connecting them.

Integrating with Agentic-RAG

The GraphRAG service integrates seamlessly with agentic systems as a specialized tool.

Direct API Usage

Query the running FastAPI service from any Python client:

import requests

payload = {
    "question": "How does GDPR relate to cross-border data transfer?",
    "search_type": "graph",
    "max_hops": 2,
    "top_k": 3,
}

response = requests.get(
    "http://localhost:8000/query",
    params=payload
)

data = response.json()
print(data["evidence"])  # Ready-to-use context for LLM prompts

Agent Tool Integration

Register GraphRAG as a tool within your Agentic-RAG system (as demonstrated in chapter9/agentic-rag/):

class GraphRAGTool:
    name = "graph_rag"
    description = "Retrieve structured, multi-hop knowledge from the GraphRAG index."
    
    def run(self, question: str) -> str:
        resp = requests.get(
            "http://localhost:8000/query",
            params={
                "question": question,
                "search_type": "graph",
                "max_hops": 3,
                "top_k": 5
            }
        )
        return resp.json()["evidence"]

When the agent encounters questions requiring relational reasoning—such as "Which regulations govern data-transfer between EU and US?"—it invokes this tool to retrieve explicit provenance chains that vector search cannot provide.

Summary

• GraphRAG converts unstructured text into a traversable NetworkX knowledge graph through LLM-based entity extraction.
• The GraphRAGIndexer class in chapter3/structured-index/graphrag_indexer.py manages extraction, construction, community detection, and multi-hop search.
• Configuration is centralized in GraphRAGConfig (config.py), controlling chunk sizes, model selection, and community algorithms.
• The FastAPI service (api_service.py) exposes /build and /query endpoints for remote integration with agentic systems.
• Multi-hop search answers "how is A connected to B" questions by returning explicit entity paths rather than semantically similar text chunks.

Frequently Asked Questions

What is the difference between GraphRAG and standard vector RAG?

Standard vector RAG retrieves chunks based on embedding similarity, which struggles with multi-hop relationships and explicit entity connections. GraphRAG stores entities as nodes and relations as edges, enabling precise traversal across documents to answer questions like "how does X regulate Y" that require following specific relational chains.

How does the GraphRAG system handle document updates?

The current implementation in ai-agent-book builds static indexes from input files. To update the knowledge graph, you must rebuild the index using main.py --mode build or call the /build endpoint with the new document corpus. The binary .pkl format ensures fast loading but requires full reconstruction for updates.

Can I use GraphRAG without the FastAPI service?

Yes. The GraphRAGIndexer class can be imported and used directly in Python scripts without running api_service.py. Instantiate the class with GraphRAGConfig, call build_from_file() for construction, and use multi_hop_search() for queries—all without network overhead.

What LLM providers are supported for entity extraction?

The implementation uses OpenRouter by default, configured through environment variables loaded in GraphRAGConfig. You can modify graphrag_indexer.py to point to other providers (OpenAI, Anthropic, or local models) by adjusting the API client initialization while preserving the expected JSON output format for entity-relation triples.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}