How to Build GraphRAG Pipelines with Entity-Aware Document Chunking

Use Semantica's EntityAwareChunker class to split documents while preserving entity boundaries, then feed chunks into KGBuilder for knowledge graph construction and retrieval.

GraphRAG pipelines require more than naive text splitting—splitting mid-entity destroys the very relationships your knowledge graph needs to capture. Semantica solves this with knowledge-graph-aware chunkers that treat entities as indivisible units. The EntityAwareChunker class in semantica/split/kg_chunkers.py and its underlying split_entity_aware algorithm in semantica/split/methods.py form the foundation of entity-aware document processing for production GraphRAG systems.

How EntityAwareChunker Preserves Entity Boundaries

The EntityAwareChunker class delegates boundary-respecting splits to split_entity_aware. Here's how the algorithm works:

1. Entity Extraction — A configurable NERExtractor identifies all entity spans in the input text.

2. Boundary Mapping — Start and end character positions populate entity_boundaries, a set tracking exactly where entities exist.

3. Sentence-Level Traversal — The text splits via _split_sentences_regex, then each sentence is evaluated against entity boundaries.

4. Boundary-Respecting Chunk Construction — When adding a sentence would exceed chunk_size and that sentence contains an entity boundary, the chunker refuses to split—preserving the entity span intact. Otherwise, a Chunk dataclass emits with metadata: method name, chunk size, entity count, and extracted entity objects.

5. Fallback Handling — Without the optional semantic_extract package, the system gracefully degrades to a generic recursive splitter.

The resulting Chunk objects carry embedded metadata that downstream components like KGBuilder use to construct nodes and edges without semantic fragmentation.

Basic GraphRAG Pipeline Architecture

raw_documents → EntityAwareChunker → Chunk list
          ↓
    Entity & Relation extraction → Knowledge graph construction
          ↓
    Vector store indexing (optional) → Graph-based retrieval
          ↓
    LLM generation (GraphRAG) → Final answer

This architecture matters because GraphRAG retrieval operates over structured subgraphs, not raw text windows. When EntityAwareChunker prevents split-entity artifacts, your retrieved subgraphs contain whole entities with intact relationships—directly improving LLM answer quality.

Implementing Entity-Aware Chunking in Python

Basic EntityAwareChunker Usage

from semantica.split.kg_chunkers import EntityAwareChunker

text = """
Apple announced its new iPhone 15 at the September event. Tim Cook
highlighted the camera improvements. The device will be available
in the United States, Europe, and Asia starting next month.
"""

chunker = EntityAwareChunker(chunk_size=120, ner_method="ml")
chunks = chunker.chunk(text)

for i, c in enumerate(chunks, 1):
    print(f"Chunk {i}:")
    print(c.text)
    print("Metadata →", c.metadata)
    print("-" * 40)

The ner_method parameter accepts "ml", "huggingface", or "pattern" to match your NER requirements and dependency availability.

Integrating Chunks into Knowledge Graph Construction

from semantica.kg.builder import KGBuilder
from semantica.split.kg_chunkers import EntityAwareChunker

# 1. Chunk with entity boundary preservation

chunker = EntityAwareChunker(chunk_size=200, ner_method="huggingface")
chunks = chunker.chunk(text)

# 2. Build knowledge graph from semantically coherent chunks

builder = KGBuilder()
for chunk in chunks:
    # KGBuilder extracts entities & relations from each intact chunk

    builder.add_document(chunk.text, metadata=chunk.metadata)

graph = builder.build()

# 3. Graph-based retrieval (e.g., subgraph for "Apple")

subgraph = graph.query_nodes(label="Apple")
answer = llm.generate(
    prompt=f"Based on this sub-graph: {subgraph}",
    context=chunk.text
)
print(answer)

The KGBuilder leverages chunk.metadata directly, ensuring graph nodes retain provenance to their source chunks.

Hybrid Vector-Graph Indexing for Advanced GraphRAG

For production systems requiring both semantic similarity and graph traversal, combine EntityAwareChunker with vector stores:

from semantica.vector_store.pgvector import PGVectorStore
from semantica.split.kg_chunkers import EntityAwareChunker

# Entity-aware chunking

chunker = EntityAwareChunker(chunk_size=150, ner_method="pattern")
chunks = chunker.chunk(text)

# Index in PostgreSQL with pgvector extension

store = PGVectorStore(connection_string="postgresql://...")
for chunk in chunks:
    store.add_texts([chunk.text], metadata=[chunk.metadata])

# Hybrid retrieval: vector similarity + graph expansion

retrieved = store.similarity_search("latest iPhone release", k=3)

# Expand vector results via graph traversal for relationship context

This hybrid GraphRAG approach retrieves via vector similarity, then enriches results through graph neighbor traversal—EntityAwareChunker ensures both retrieval modalities operate on semantically intact units.

Key Source Files in the Semantica Repository

Component	File Path	Purpose
Entity-aware chunker class	semantica/split/kg_chunkers.py	Implements EntityAwareChunker with progress tracking
Core splitting algorithm	semantica/split/methods.py	Contains split_entity_aware with boundary-respecting logic
Chunk data structure	semantica/split/semantic_chunker.py	Defines Chunk dataclass used throughout pipelines
Graph builder	semantica/kg/builder.py	Transforms chunks into knowledge graph nodes and edges

Summary

• EntityAwareChunker in kg_chunkers.py provides the primary interface for entity-aware document splitting in Semantica.

• split_entity_aware in methods.py implements the boundary-detection algorithm that prevents entity fragmentation.

• GraphRAG pipelines benefit from intact entity spans because retrieval operates over subgraphs—split entities would break relationship chains.

• Hybrid indexing combines EntityAwareChunker with vector stores like PGVectorStore for production Retrieval-Augmented Generation systems.

Frequently Asked Questions

What happens if no NER extractor is available?

split_entity_aware falls back to a generic recursive splitter defined in methods.py. The chunker degrades gracefully but loses entity boundary protection—monitor logs for fallback activation in production.

How does chunk_size interact with entity boundaries?

The chunk_size parameter sets a target maximum, not a hard limit. When a sentence containing an entity boundary would exceed chunk_size, the algorithm extends the current chunk rather than splitting mid-entity. This means actual chunks may slightly exceed chunk_size to preserve semantic coherence.

Can I use custom NER models with EntityAwareChunker?

Yes. The ner_method parameter accepts string identifiers that resolve through Semantica's extractor registry. For custom models, implement the NERExtractor protocol from semantic_extract and register your implementation—EntityAwareChunker will delegate extraction to your model automatically.

What's the performance cost of entity-aware chunking?

Entity extraction adds overhead proportional to document length and NER model complexity. The pattern method (rule-based) is fastest; huggingface transformer models provide better accuracy but require GPU for batch processing. Profile your specific chunk_size and ner_method combination—methods.py contains optimization hooks for batch entity boundary computation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}