How to Build a Knowledge Graph-Enhanced RAG System: A Complete Implementation Guide

A knowledge graph-enhanced RAG system combines vector similarity search with structured entity-relationship traversal, enabling multi-hop reasoning and context fusion through the GraphRAG and KnowledgeGraph classes in the davidkimai/context-engineering repository.

This guide walks through building a production-ready implementation using the Software 3.0 stack architecture found in the course materials. You will learn to construct a pipeline that leverages semantic graph traversal alongside traditional retrieval methods for higher-quality, fact-grounded generation.

Architecture Overview

The system implements a three-layer architecture that maps directly onto the Software 3.0 stack used throughout the repository. Each layer handles distinct responsibilities while maintaining clean separation between prompt communication, programming implementation, and protocol orchestration.

Layer 1: Prompt Communication

This layer generates graph-aware prompts that describe the user query, identified entities, and desired relationships. The implementation relies on the graph-aware prompt template defined in 03_graph_enhanced_rag.md at line 72, specifically the GRAPH_QUERY_TEMPLATE constant.

Layer 2: Programming Implementation

This layer provides the concrete graph algorithms, embedding handling, and hybrid retrieval logic. Core classes reside in 00_COURSE/02_context_processing/labs/structured_data_lab.py:

• KnowledgeGraph (line 109): Handles graph storage, schema validation, and GNN-style transforms.
• GraphRAG (line 78): Orchestrates query encoding, entity similarity search, sub-graph extraction, and document fusion.

Layer 3: Protocol Orchestration

This layer coordinates multi-hop reasoning, semantic integration, and final context synthesis for the LLM. The _create_unified_context method inside GraphRAG (line 66) fuses query, entity, and document embeddings using a learned projection (self.context_fusion).

Data Flow and Processing Pipeline

Understanding the end-to-end data flow is essential for implementing custom extensions. The system processes information through four distinct stages that bridge unstructured text and structured knowledge.

Document Ingestion

Each document is parsed, and its entities and relations are extracted using any NER/RE pipeline. These are stored in the KnowledgeGraph instance, while the document's vector embedding is saved in self.document_embeddings.

Query Processing

The user query is embedded and transformed by self.query_encoder. The system then retrieves the most similar entities via KnowledgeGraph.similarity_search and documents linked to those entities through self.entity_document_map.

Sub-graph Expansion

For each top entity, the system extracts a 1-hop neighbourhood using KnowledgeGraph.get_subgraph to provide relational context that captures multi-hop relationships.

Context Fusion

Entity embeddings, weighted document embeddings, and the original query embedding are concatenated and projected through self.context_fusion to produce a single "unified context" vector suitable for any LLM-based generator.

Implementation Guide

The following runnable snippets demonstrate the three core stages of the pipeline. All file paths reference the repository's source code for direct verification.

Setting Up the Knowledge Graph

Begin by defining your schema and initializing the graph storage layer. The KnowledgeGraph class requires a dimensionality parameter (d_model) and an optional schema for validation.

from pathlib import Path
import numpy as np
from structured_data_lab import KnowledgeGraph, Entity, Relation, EntityType, RelationType, Schema

# Define a simple schema (optional but recommended)

schema = Schema(
    name="DemoSchema",
    entity_types={EntityType.PERSON, EntityType.ORGANIZATION},
    relation_types={RelationType.ASSOCIATED_WITH}
)

kg = KnowledgeGraph(d_model=256, schema=schema)

# Add some entities

alice = Entity(id="e1", name="Alice", entity_type=EntityType.PERSON)
acme = Entity(id="e2", name="Acme Corp", entity_type=EntityType.ORGANIZATION)
kg.add_entity(alice)
kg.add_entity(acme)

# Add a relation

kg.add_relation(Relation(source=alice, target=acme, relation_type=RelationType.ASSOCIATED_WITH))

Source: KnowledgeGraph implementation – structured_data_lab.py line 109.

Indexing Documents with Graph Links

Use the GraphRAG class to index documents while maintaining links between text chunks and graph entities. This establishes the bridge between vector retrieval and structured knowledge.

from structured_data_lab import GraphRAG

rag = GraphRAG(d_model=256)

# Pretend we have a document embedding (e.g., from a transformer)

doc_emb = np.random.randn(256)

# Add the document together with its extracted entities/relations

rag.add_document(
    document_id="doc-001",
    content="Alice works at Acme Corp developing AI.",
    entities=[alice, acme],
    relations=[Relation(source=alice, target=acme, relation_type=RelationType.ASSOCIATED_WITH)],
    document_embedding=doc_emb
)

Source: GraphRAG.add_document – structured_data_lab.py line 91.

Retrieving Graph-Enhanced Context

Execute hybrid retrieval that combines entity similarity search with document retrieval. The retrieve_context method handles sub-graph extraction and context fusion automatically.

# Encode the query (in practice use a real encoder)

query = "What AI projects is Alice doing at Acme?"
query_emb = np.random.randn(256)

result = rag.retrieve_context(
    query=query,
    query_embedding=query_emb,
    top_k_entities=3,
    top_k_documents=2
)

print("Relevant entities:", [e.name for e, _ in result["relevant_entities"]])
print("Top documents:", [doc_id for doc_id, _ in result["relevant_documents"]])
print("Unified context shape:", result["unified_context"].shape)

Source: GraphRAG.retrieve_context – structured_data_lab.py line 78.

Prompting the LLM with Unified Context

Convert the unified context vector into a format suitable for your LLM. While the embedding-to-text conversion is domain-specific, you can leverage the graph-aware prompt template provided in the course materials.

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")
model = AutoModelForCausalLM.from_pretrained("gpt2")

# Convert the unified context embedding into a textual prompt (simple example)

prompt = f"""[GRAPH CONTEXT]\n{result['unified_context']}\n\nUser: {query}\nAnswer:"""
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Note: The repository provides a graph-aware prompt template you can adapt (see GRAPH_QUERY_TEMPLATE in 03_graph_enhanced_rag.md line 72).

Key Files and References

The following files contain the complete implementation and theoretical foundation for the knowledge graph-enhanced RAG system:

File	Purpose	Direct Link
00_COURSE/02_context_processing/labs/structured_data_lab.py	Full implementation of Entity, Relation, Schema, KnowledgeGraph, and GraphRAG.	View on GitHub
00_COURSE/04_retrieval_augmented_generation/03_graph_enhanced_rag.md	Conceptual overview, stack diagram, and the graph-aware prompt template.	View on GitHub
40_reference/retrieval_indexing.md	Details on hybrid retrieval strategies (vector + graph) that can be plugged into GraphRAG.	View on GitHub
40_reference/field_mapping.md	Guidance on mapping domain fields to graph schema concepts.	View on GitHub
40_reference/symbolic_residue_types.md	Provides symbolic residue types useful for encoding relational semantics in prompts.	View on GitHub

Summary

• Knowledge graph-enhanced RAG systems combine vector similarity search with structured entity-relationship traversal to enable multi-hop reasoning.
• The Software 3.0 stack architecture separates concerns into prompt communication, programming implementation, and protocol orchestration layers.
• Core classes KnowledgeGraph (line 109) and GraphRAG (line 78) in structured_data_lab.py handle graph storage, schema validation, and hybrid retrieval logic.
• Context fusion occurs through the _create_unified_context method (line 66), which projects concatenated embeddings into a unified vector space for LLM consumption.
• Graph-aware prompts can be constructed using the GRAPH_QUERY_TEMPLATE found in 03_graph_enhanced_rag.md (line 72).

Frequently Asked Questions

What is the difference between standard RAG and knowledge graph-enhanced RAG?

Standard RAG retrieves documents based solely on vector similarity between the query and document embeddings. Knowledge graph-enhanced RAG adds a structured layer where entities and relationships are explicitly modeled, enabling multi-hop reasoning through relationship traversal and context fusion that combines semantic similarity with topological relevance. According to the davidkimai/context-engineering source code, this hybrid approach uses GraphRAG.retrieve_context to merge entity similarity search with document retrieval.

How does the context fusion mechanism work in GraphRAG?

The context fusion mechanism implemented in _create_unified_context (line 66 of structured_data_lab.py) concatenates three embedding vectors: the encoded query, the weighted entity embeddings from the knowledge graph, and the document embeddings linked to those entities. This concatenated tensor is then projected through a learned linear layer (self.context_fusion) to produce a unified context vector that preserves relational semantics while maintaining compatibility with standard LLM input dimensions.

What schema validation does the KnowledgeGraph class provide?

The KnowledgeGraph class (line 109 of structured_data_lab.py) accepts an optional Schema object that defines valid entity_types and relation_types. When adding entities or relations, the system validates against these predefined sets, ensuring graph consistency. This prevents orphaned relations and maintains type safety across the knowledge base, which is critical for reliable multi-hop traversal during the retrieval phase.

Can I use existing NER models with this knowledge graph implementation?

Yes, the GraphRAG.add_document method (line 91 of structured_data_lab.py) is designed to accept pre-extracted entities and relations from any external NER/RE pipeline. You simply pass the extracted Entity and Relation objects along with the document embedding. This decoupled design allows integration with spaCy, Hugging Face transformers, or commercial NLP APIs while maintaining the repository's standardized graph schema.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}