How to Configure Entity-Aware Text Chunking for GraphRAG Pipelines in Semantica

Use the EntityAwareChunker class from semantica.split.kg_chunkers to split documents into chunks that preserve named entity boundaries, then feed the resulting Chunk objects into KGBuilder and optional vector stores for graph-based retrieval.

Entity-aware text chunking is a critical preprocessing step for GraphRAG pipelines because split entities break graph connectivity and degrade LLM answer quality. The open-source semantica-agi/semantica repository provides a dedicated EntityAwareChunker that respects entity boundaries during document splitting. In this guide, you will learn how to configure and wire this chunker into a complete GraphRAG pipeline using Semantica's knowledge graph and vector store components.

How Entity-Aware Chunking Works in Semantica

As implemented in the semantica-agi/semantica source code, the core algorithm lives in semantica/split/methods.py inside the split_entity_aware function, while the high-level interface is the EntityAwareChunker class defined in semantica/split/kg_chunkers.py. The chunker processes raw text through four main stages.

Entity Extraction and Boundary Mapping

First, split_entity_aware invokes a configurable NERExtractor to identify all entity spans in the input text. The start and end character positions of each detected entity are stored in an entity_boundaries set. This boundary map ensures that the splitter knows exactly where entities begin and end before any text is divided.

Sentence-Level Traversal

Next, the text is divided into sentences using an internal regex-based splitter, _split_sentences_regex. The algorithm traverses these sentences sequentially. Before a sentence is appended to the current chunk, the system checks whether the sentence crosses any position stored in entity_boundaries.

Chunk Construction and Boundary Protection

If adding a sentence would exceed the target chunk_size and that sentence contains an entity boundary, the chunk is kept intact rather than split mid-entity. Otherwise, a new Chunk dataclass is emitted from semantica/split/semantic_chunker.py. Each Chunk carries metadata including the method name, chunk size, the number of entities contained, and the list of extracted entity objects. This behavior guarantees that retrieved sub-graphs contain whole entities with their surrounding context.

Fallback for Missing Dependencies

If the optional semantic_extract package is not installed, split_entity_aware automatically falls back to a generic recursive splitter. This ensures the pipeline remains functional even when advanced NER extractors are unavailable.

Configuring the EntityAwareChunker

To configure entity-aware text chunking, import EntityAwareChunker and set the desired chunk_size and ner_method.

from semantica.split.kg_chunkers import EntityAwareChunker

text = """
Apple announced its new iPhone 15 at the September event. Tim Cook
highlighted the camera improvements. The device will be available
in the United States, Europe, and Asia starting next month.
"""

chunker = EntityAwareChunker(chunk_size=120, ner_method="ml")
chunks = chunker.chunk(text)

for i, c in enumerate(chunks, 1):
    print(f"Chunk {i}:")
    print(c.text)
    print("Metadata →", c.metadata)
    print("-" * 40)

The ner_method parameter accepts options such as "ml", "huggingface", and "pattern", allowing you to trade off between accuracy and execution speed based on your runtime environment.

Integrating Chunks into a GraphRAG Pipeline

After chunking, the list of Chunk objects feeds directly into Semantica’s graph ingestion utilities.

Building a Knowledge Graph with KGBuilder

The KGBuilder class in semantica.kg.builder consumes chunks and materializes nodes and edges. Because EntityAwareChunker preserves whole entities, the resulting graph avoids broken references and maintains semantic coherence.

from semantica.kg.builder import KGBuilder
from semantica.split.kg_chunkers import EntityAwareChunker

# 1. Chunk the document

chunker = EntityAwareChunker(chunk_size=200, ner_method="huggingface")
chunks = chunker.chunk(text)

# 2. Build a knowledge graph from the chunks

builder = KGBuilder()
for chunk in chunks:
    # The builder extracts entities & relations from each chunk

    builder.add_document(chunk.text, metadata=chunk.metadata)

graph = builder.build()

# 3. Perform a graph-based retrieval (e.g., find nodes about "Apple")

subgraph = graph.query_nodes(label="Apple")
answer = llm.generate(prompt=f"Based on this sub-graph: {subgraph}", context=chunk.text)
print(answer)

Adding Hybrid Vector-Graph Retrieval

For hybrid retrieval, index the chunks in PGVectorStore alongside the knowledge graph. This enables similarity search over text while preserving the structured relationships in the graph.

from semantica.vector_store.pgvector import PGVectorStore
from semantica.split.kg_chunkers import EntityAwareChunker

# Chunk the document

chunker = EntityAwareChunker(chunk_size=150, ner_method="pattern")
chunks = chunker.chunk(text)

# Index chunks in a PGVector store

store = PGVectorStore(connection_string="postgresql://...")
for chunk in chunks:
    store.add_texts([chunk.text], metadata=[chunk.metadata])

# Retrieve relevant chunks for a query

retrieved = store.similarity_search("latest iPhone release", k=3)

# Combine with graph retrieval as needed

Key Source Files and Components

Understanding the module layout helps when customizing or debugging your GraphRAG pipeline.

• semantica/split/kg_chunkers.py — Implements the EntityAwareChunker class and ties it to progress-tracking utilities.
• semantica/split/methods.py — Contains split_entity_aware, the core algorithm that respects entity boundaries.
• semantica/split/semantic_chunker.py — Defines the Chunk dataclass used throughout the pipeline.
• semantica/kg/builder.py — Provides KGBuilder to transform chunks into graph nodes and edges.
• semantic_extract — Optional dependency supplying the NERExtractor interface.

Summary

• The EntityAwareChunker class in semantica.split.kg_chunkers is the primary entry point for entity-aware text chunking in Semantica.
• The split_entity_aware function in semantica/split/methods.py enforces boundary protection by checking entity spans against sentence splits.
• Each emitted Chunk includes metadata on entity count and extracted objects, making it ideal for downstream graph construction.
• KGBuilder consumes these chunks to create a knowledge graph where entities remain whole and semantically connected.
• Optional PGVectorStore integration enables hybrid vector-graph retrieval for advanced GraphRAG pipelines.

Frequently Asked Questions

What happens if an entity is longer than the configured chunk_size?

The split_entity_aware algorithm prioritizes entity integrity over strict size limits. If a sentence containing an entity boundary would exceed chunk_size, the chunk is not split mid-entity, which means individual chunks may occasionally exceed the target size to preserve the full entity span.

Which NER methods does EntityAwareChunker support?

The ner_method parameter accepts "ml", "huggingface", and "pattern" according to the Semantica source code. The underlying NERExtractor is provided by the optional semantic_extract package; if that package is unavailable, the system falls back to a generic recursive splitter.

How does entity-aware chunking improve GraphRAG quality?

By preventing entities from being split across chunks, the EntityAwareChunker ensures that KGBuilder extracts whole nodes and their surrounding context. This produces a knowledge graph with intact relationships, so retrieval returns coherent sub-graphs rather than fragmented mentions that confuse the downstream LLM.

Can I use EntityAwareChunker without building a knowledge graph?

Yes. The Chunk objects returned by EntityAwareChunker can be indexed directly in PGVectorStore or any other vector database for standard RAG. However, pairing the chunker with Semantica’s graph components unlocks the full GraphRAG capability.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}