Building Production-Ready RAG Systems with Knowledge Graphs: A Complete Implementation Guide

Production-ready RAG systems combine dense vector search, sparse BM25, and knowledge graph traversal orchestrated by an LLM using the ReAct pattern to deliver precise answers to complex multi-entity queries.

The bojieli/ai-agent-book repository provides a complete open-source blueprint for building production-ready RAG systems with knowledge graphs. Following the Agent = LLM + Context + Tools formula, the codebase implements a modular architecture that upgrades traditional text-based retrieval with structured knowledge representation and agentic reasoning loops.

Core Architecture Components

Production-ready RAG requires five integrated components working across the chapter3/ directory. Each component addresses specific failure modes of naive vector-only retrieval.

The Agent Foundation

At the system’s core lies the Agent = LLM + Context + Tools paradigm implemented throughout the codebase. The LLM (configurable for Kimi, OpenRouter, or local Ollama via docs/en/README.md) generates responses and orchestrates tool calls. The Context Engine maintains system prompts and KV-cache-optimized histories (see chapter2/system-hint/view_trajectory.py). Tools provide deterministic access to external data sources through the MCP protocol defined in chapter4/README.en.md.

Knowledge Graph Store

The chapter3/structured-index/ project implements two structured indexing strategies that eliminate context loss from naive chunking:

• RAPTOR: Hierarchical summarization that preserves high-level semantics across document sections
• GraphRAG: Entity-relationship extraction enabling queries like "Show all cases where the defendant was sentenced for fraud involving more than $1M"

Both indexes encode structured relations beyond plain text, with GraphRAG storing triples in Neo4j-compatible format while RAPTOR maintains tree-structured summaries.

Hybrid Retrieval Pipeline

The chapter3/retrieval-pipeline/ combines three retrieval modalities:

1. Dense vectors: FAISS or HNSW indexes for semantic similarity
2. Sparse BM25: Inverted index for exact term matching
3. Graph traversal: Relationship-aware retrieval from structured indexes

This hybrid approach ensures high recall while maintaining the precision needed for multi-entity queries.

Implementing the Knowledge Graph Index

Building a production GraphRAG index involves entity extraction and relationship mapping. The structured-index module provides deterministic indexing pipelines.

Install the chapter 3 dependencies using the reproducible lockfile:

uv sync --locked --extra ch3

Create the GraphRAG index from your document corpus:

# scripts/build_graph_rag.py

from structured_index.graph_rag import GraphRAGBuilder

builder = GraphRAGBuilder(
    source_dir="data/documents",      # folder with .txt/.pdf files

    output_dir="graph_rag_index",
    embedding_model="sentence-transformers/all-MiniLM-L6-v2",
)
builder.build()
print("GraphRAG index built at graph_rag_index/")

Execute the indexing pipeline:

python scripts/build_graph_rag.py

The builder extracts entities, relationships, and community summaries, storing them in a format optimized for graph traversal queries.

Building the Agentic RAG Loop

The chapter3/agentic-rag/ directory implements the ReAct pattern, allowing the LLM to iteratively refine queries based on intermediate retrieval results.

Initialize the agent with graph-based retrieval capabilities:

# chapter3/agentic_rag/main.py

from agentic_rag.agent import AgenticRAGAgent
from structured_index.graph_rag import GraphRAGRetriever

retriever = GraphRAGRetriever(index_path="graph_rag_index")
agent = AgenticRAGAgent(
    llm_name="gpt-4o-mini",          # supports Ollama, OpenRouter, etc.

    retriever=retriever,
    max_iterations=3,
)

response = agent.ask("What are the legal consequences for a company that breaches GDPR in Europe?")
print(response)

Run the agent:

python chapter3/agentic_rag/main.py

The execution flow follows three steps: the GraphRAGRetriever fetches relevant triples, the LLM evaluates sufficiency and potentially reformulates the query, and the loop continues until max_iterations or answer satisfaction.

Hybrid Retrieval for Production Queries

For direct pipeline access without the agent loop, use the HybridRetriever to combine dense, sparse, and graph signals:

from retrieval_pipeline.pipeline import HybridRetriever
from evaluation.metrics import ndcg_score

retriever = HybridRetriever(
    dense_path="dense_index",
    sparse_path="bm25_index",
    graph_path="graph_rag_index",
)

# Single query with fused results

results = retriever.query("What is the penalty for insider trading?")

The retriever implements fallback strategies (dense → sparse → graph) ensuring reliability even when individual modalities return empty results.

Production Deployment Considerations

Deploying knowledge graph RAG requires attention to scalability, observability, and security beyond the core algorithms.

Scalability and Environment Management

Pin exact dependency versions using the provided uv.lock file to ensure reproducible builds across environments. The vector store abstraction allows swapping FAISS for managed services like Pinecone without modifying retrieval logic. Containerize using the uv toolchain:

docker build -t rag-agent . --build-arg UV_EXTRA=ch3

Observability and Security

The log-sanitization/ tool redacts API credentials from logs, enforcing the security requirement that secrets stored in .env files (copied from .env.example) never appear in telemetry. Integrate OpenTelemetry exporters to trace the ReAct loop iterations and retrieval latencies.

Evaluation Framework

Benchmark your knowledge graph implementation using the evaluation suite in chapter6/user-memory-system-evaluation/:

# chapter6/evaluation/rag_eval.py

queries = ["What is the penalty for insider trading?"]
ground_truth = [["Section 10-2 of the Securities Act", "Fine up to $5M"]]
preds = retriever.query_batch(queries)
print("NDCG:", ndcg_score(preds, ground_truth))

Measure recall, latency, and factual correctness specifically testing whether graph traversal improves answer accuracy for multi-hop questions compared to vector-only baselines.

Summary

• Knowledge graphs eliminate context loss by encoding entity relationships that chunking destroys, implemented via GraphRAG and RAPTOR in chapter3/structured-index/.
• Hybrid retrieval combines three modalities: dense vectors (FAISS/HNSW), sparse BM25, and graph traversal for robust recall and precision.
• Agentic reasoning uses ReAct: The AgenticRAGAgent in chapter3/agentic-rag/ iteratively refines queries based on retrieved graph triples.
• Production hardening requires: dependency locking with uv, credential sanitization via log-sanitization/, and evaluation using chapter6/ metrics.
• Deployment flexibility: Support for local (Ollama), commercial (OpenRouter), or regional (Kimi) LLM providers via unified configuration.

Frequently Asked Questions

What is the difference between RAPTOR and GraphRAG indexing?

RAPTOR creates hierarchical tree summaries of document clusters, preserving semantic flow across long contexts without explicit relationship modeling. GraphRAG extracts entities and relationships into a graph structure, enabling precise queries about specific connections between objects. Use RAPTOR for narrative coherence and GraphRAG when answering relationship-heavy questions like "Which suppliers worked with both Company A and Company B?" Both implementations reside in chapter3/structured-index/.

How does the ReAct pattern improve RAG accuracy over single-shot retrieval?

The ReAct pattern implemented in chapter3/agentic-rag/ allows the LLM to perform multi-hop reasoning by evaluating retrieved graph triples and deciding whether to search again with refined parameters. Unlike single-shot retrieval, this iterative loop can bridge disconnected facts through intermediate entities, correct initial query misunderstandings, and verify answer completeness against the knowledge graph structure before final output.

What retrieval strategy should I use for production knowledge graph RAG?

Deploy the hybrid retrieval strategy from chapter3/retrieval-pipeline/ using a fallback chain: start with dense vector search for semantic similarity, fall back to BM25 for exact keyword matching if confidence is low, and activate graph traversal when queries contain explicit entity references or relationship questions. This maximizes recall while minimizing latency for simple queries.

How do I evaluate whether knowledge graphs improve my RAG system?

Use the NDCG and recall metrics in chapter6/user-memory-system-evaluation/ to compare hybrid retrieval against vector-only baselines. Specifically test multi-hop questions requiring connection of two or more entities—this is where GraphRAG typically shows 15-30% improvement in answer accuracy. Monitor latency overhead from graph traversal; production systems should cache frequent entity lookups to maintain sub-second response times.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}