# Building Production-Ready RAG Systems with Knowledge Graphs: A Complete Implementation Guide

> Build production-ready RAG systems with knowledge graphs using LLMs and ReAct. Master dense vector search, BM25, and graph traversal for complex queries. Learn implementation now.

- Repository: [Bojie Li/ai-agent-book](https://github.com/bojieli/ai-agent-book)
- Tags: how-to-guide
- Published: 2026-08-01

---

**Production-ready RAG systems combine dense vector search, sparse BM25, and knowledge graph traversal orchestrated by an LLM using the ReAct pattern to deliver precise answers to complex multi-entity queries.**

The `bojieli/ai-agent-book` repository provides a complete open-source blueprint for building production-ready RAG systems with knowledge graphs. Following the **Agent = LLM + Context + Tools** formula, the codebase implements a modular architecture that upgrades traditional text-based retrieval with structured knowledge representation and agentic reasoning loops.

## Core Architecture Components

Production-ready RAG requires five integrated components working across the `chapter3/` directory. Each component addresses specific failure modes of naive vector-only retrieval.

### The Agent Foundation

At the system’s core lies the **Agent = LLM + Context + Tools** paradigm implemented throughout the codebase. The **LLM** (configurable for Kimi, OpenRouter, or local Ollama via [`docs/en/README.md`](https://github.com/bojieli/ai-agent-book/blob/main/docs/en/README.md)) generates responses and orchestrates tool calls. The **Context Engine** maintains system prompts and KV-cache-optimized histories (see [`chapter2/system-hint/view_trajectory.py`](https://github.com/bojieli/ai-agent-book/blob/main/chapter2/system-hint/view_trajectory.py)). **Tools** provide deterministic access to external data sources through the MCP protocol defined in [`chapter4/README.en.md`](https://github.com/bojieli/ai-agent-book/blob/main/chapter4/README.en.md).

### Knowledge Graph Store

The `chapter3/structured-index/` project implements two structured indexing strategies that eliminate context loss from naive chunking:

- **RAPTOR**: Hierarchical summarization that preserves high-level semantics across document sections
- **GraphRAG**: Entity-relationship extraction enabling queries like *"Show all cases where the defendant was sentenced for fraud involving more than $1M"*

Both indexes encode structured relations beyond plain text, with GraphRAG storing triples in Neo4j-compatible format while RAPTOR maintains tree-structured summaries.

### Hybrid Retrieval Pipeline

The `chapter3/retrieval-pipeline/` combines three retrieval modalities:

1. **Dense vectors**: FAISS or HNSW indexes for semantic similarity
2. **Sparse BM25**: Inverted index for exact term matching
3. **Graph traversal**: Relationship-aware retrieval from structured indexes

This hybrid approach ensures high recall while maintaining the precision needed for multi-entity queries.

## Implementing the Knowledge Graph Index

Building a production GraphRAG index involves entity extraction and relationship mapping. The `structured-index` module provides deterministic indexing pipelines.

Install the chapter 3 dependencies using the reproducible lockfile:

```bash
uv sync --locked --extra ch3

```

Create the GraphRAG index from your document corpus:

```python

# scripts/build_graph_rag.py

from structured_index.graph_rag import GraphRAGBuilder

builder = GraphRAGBuilder(
    source_dir="data/documents",      # folder with .txt/.pdf files

    output_dir="graph_rag_index",
    embedding_model="sentence-transformers/all-MiniLM-L6-v2",
)
builder.build()
print("GraphRAG index built at graph_rag_index/")

```

Execute the indexing pipeline:

```bash
python scripts/build_graph_rag.py

```

The builder extracts entities, relationships, and community summaries, storing them in a format optimized for graph traversal queries.

## Building the Agentic RAG Loop

The `chapter3/agentic-rag/` directory implements the ReAct pattern, allowing the LLM to iteratively refine queries based on intermediate retrieval results.

Initialize the agent with graph-based retrieval capabilities:

```python

# chapter3/agentic_rag/main.py

from agentic_rag.agent import AgenticRAGAgent
from structured_index.graph_rag import GraphRAGRetriever

retriever = GraphRAGRetriever(index_path="graph_rag_index")
agent = AgenticRAGAgent(
    llm_name="gpt-4o-mini",          # supports Ollama, OpenRouter, etc.

    retriever=retriever,
    max_iterations=3,
)

response = agent.ask("What are the legal consequences for a company that breaches GDPR in Europe?")
print(response)

```

Run the agent:

```bash
python chapter3/agentic_rag/main.py

```

The execution flow follows three steps: the **GraphRAGRetriever** fetches relevant triples, the LLM evaluates sufficiency and potentially reformulates the query, and the loop continues until `max_iterations` or answer satisfaction.

## Hybrid Retrieval for Production Queries

For direct pipeline access without the agent loop, use the `HybridRetriever` to combine dense, sparse, and graph signals:

```python
from retrieval_pipeline.pipeline import HybridRetriever
from evaluation.metrics import ndcg_score

retriever = HybridRetriever(
    dense_path="dense_index",
    sparse_path="bm25_index",
    graph_path="graph_rag_index",
)

# Single query with fused results

results = retriever.query("What is the penalty for insider trading?")

```

The retriever implements fallback strategies (dense → sparse → graph) ensuring reliability even when individual modalities return empty results.

## Production Deployment Considerations

Deploying knowledge graph RAG requires attention to scalability, observability, and security beyond the core algorithms.

### Scalability and Environment Management

Pin exact dependency versions using the provided `uv.lock` file to ensure reproducible builds across environments. The vector store abstraction allows swapping FAISS for managed services like Pinecone without modifying retrieval logic. Containerize using the `uv` toolchain:

```bash
docker build -t rag-agent . --build-arg UV_EXTRA=ch3

```

### Observability and Security

The `log-sanitization/` tool redacts API credentials from logs, enforcing the security requirement that secrets stored in `.env` files (copied from `.env.example`) never appear in telemetry. Integrate OpenTelemetry exporters to trace the ReAct loop iterations and retrieval latencies.

### Evaluation Framework

Benchmark your knowledge graph implementation using the evaluation suite in `chapter6/user-memory-system-evaluation/`:

```python

# chapter6/evaluation/rag_eval.py

queries = ["What is the penalty for insider trading?"]
ground_truth = [["Section 10-2 of the Securities Act", "Fine up to $5M"]]
preds = retriever.query_batch(queries)
print("NDCG:", ndcg_score(preds, ground_truth))

```

Measure **recall**, **latency**, and **factual correctness** specifically testing whether graph traversal improves answer accuracy for multi-hop questions compared to vector-only baselines.

## Summary

- **Knowledge graphs eliminate context loss** by encoding entity relationships that chunking destroys, implemented via GraphRAG and RAPTOR in `chapter3/structured-index/`.
- **Hybrid retrieval combines three modalities**: dense vectors (FAISS/HNSW), sparse BM25, and graph traversal for robust recall and precision.
- **Agentic reasoning uses ReAct**: The `AgenticRAGAgent` in `chapter3/agentic-rag/` iteratively refines queries based on retrieved graph triples.
- **Production hardening requires**: dependency locking with `uv`, credential sanitization via `log-sanitization/`, and evaluation using `chapter6/` metrics.
- **Deployment flexibility**: Support for local (Ollama), commercial (OpenRouter), or regional (Kimi) LLM providers via unified configuration.

## Frequently Asked Questions

### What is the difference between RAPTOR and GraphRAG indexing?

**RAPTOR** creates hierarchical tree summaries of document clusters, preserving semantic flow across long contexts without explicit relationship modeling. **GraphRAG** extracts entities and relationships into a graph structure, enabling precise queries about specific connections between objects. Use RAPTOR for narrative coherence and GraphRAG when answering relationship-heavy questions like *"Which suppliers worked with both Company A and Company B?"* Both implementations reside in `chapter3/structured-index/`.

### How does the ReAct pattern improve RAG accuracy over single-shot retrieval?

The ReAct pattern implemented in `chapter3/agentic-rag/` allows the LLM to perform **multi-hop reasoning** by evaluating retrieved graph triples and deciding whether to search again with refined parameters. Unlike single-shot retrieval, this iterative loop can bridge disconnected facts through intermediate entities, correct initial query misunderstandings, and verify answer completeness against the knowledge graph structure before final output.

### What retrieval strategy should I use for production knowledge graph RAG?

Deploy the **hybrid retrieval** strategy from `chapter3/retrieval-pipeline/` using a fallback chain: start with dense vector search for semantic similarity, fall back to BM25 for exact keyword matching if confidence is low, and activate graph traversal when queries contain explicit entity references or relationship questions. This maximizes recall while minimizing latency for simple queries.

### How do I evaluate whether knowledge graphs improve my RAG system?

Use the **NDCG and recall metrics** in `chapter6/user-memory-system-evaluation/` to compare hybrid retrieval against vector-only baselines. Specifically test multi-hop questions requiring connection of two or more entities—this is where GraphRAG typically shows 15-30% improvement in answer accuracy. Monitor latency overhead from graph traversal; production systems should cache frequent entity lookups to maintain sub-second response times.