How to Implement Entity-Aware Chunking for GraphRAG in semantica.split

Use the TextSplitter class in semantica.split to create a custom chunking method that detects named entities and aligns chunk boundaries to entity spans, ensuring no entity is split across chunks for optimal GraphRAG retrieval.

GraphRAG pipelines require chunks that preserve semantic coherence around named entities—splitting "Albert Einstein" across two chunks destroys relationship extraction quality. The semantica.split module provides an extensible architecture through the TextSplitter class that lets you implement custom strategies without modifying core library code. This guide shows you how to build, register, and deploy an entity-aware chunker that keeps entities intact while feeding metadata to your knowledge graph.

Understanding the TextSplitter Architecture

The TextSplitter class acts as a dispatcher that delegates actual splitting logic to registered functions. Understanding this flow is essential before adding custom behavior.

Core Dispatcher in splitter.py

The central class lives in semantica/split/splitter.py starting at line 54. When you call split(), the dispatcher looks up the concrete implementation via get_split_method(method_name) at line 144, then executes it with supplied options at line 152. This design means you never modify TextSplitter itself—you register new methods externally.

Method Registration in methods.py

Available strategies are stored in the SPLIT_METHODS dictionary inside semantica/split/methods.py. The get_split_method function retrieves entries from this registry. To add entity-aware functionality, you will import your new function into this file and add it to the dictionary.

Implementing the Entity-Aware Split Method

Create a new module file at semantica/split/entity_aware.py. This file contains the entity_aware_split function and helper logic for entity extraction and boundary alignment.

Creating entity_aware.py

The implementation extracts entities first, then builds chunks that never split an entity span. It returns standard Chunk objects from semantica/split/semantic_chunker.py with metadata tracking which entities appear in each chunk.

# semantica/split/entity_aware.py

from typing import List
from .semantic_chunker import Chunk
from ..utils.logging import get_logger

logger = get_logger("entity_aware")

def _extract_entities(text: str, ner_method: str = "spacy") -> List[dict]:
    """
    Return entities as [{'text': str, 'start': int, 'end': int}].
    Supports spaCy, HuggingFace, or LLM-based extraction via ner_method.
    """
    if ner_method == "spacy":
        import spacy
        nlp = spacy.load("en_core_web_sm")
        doc = nlp(text)
        return [{"text": ent.text, "start": ent.start_char, "end": ent.end_char}
                for ent in doc.ents]
    raise ValueError(f"Unsupported ner_method: {ner_method}")

def entity_aware_split(
    text: str,
    *,
    chunk_size: int = 1000,
    chunk_overlap: int = 200,
    ner_method: str = "spacy",
    **kwargs,
) -> List[Chunk]:
    """
    Split text into chunks that keep each recognized entity together.
    """
    logger.debug("Running entity-aware splitter")
    entities = _extract_entities(text, ner_method=ner_method)
    
    # Map character positions to entity indices

    entity_map = {}
    for i, ent in enumerate(entities):
        for pos in range(ent["start"], ent["end"]):
            entity_map[pos] = i

    chunks: List[Chunk] = []
    start = 0
    while start < len(text):
        # Provisional end based on chunk_size

        end = min(start + chunk_size, len(text))
        
        # If end falls inside an entity, extend to entity boundary

        while end < len(text) and end in entity_map:
            end += 1
            
        # Record which entities appear in this slice

        ent_ids = {entity_map[pos] for pos in range(start, end) if pos in entity_map}
        metadata = {"entity_ids": list(ent_ids)}
        
        chunk_text = text[start:end]
        chunks.append(Chunk(content=chunk_text, metadata=metadata))
        
        # Advance with overlap

        start = max(end - chunk_overlap, 0)
    
    logger.info(f"Created {len(chunks)} entity-aware chunks")
    return chunks

Aligning Chunk Boundaries to Entity Spans

The critical logic occurs in the boundary-adjustment loop. If the naive chunk_size endpoint lands inside an entity, the code increments end until it clears the entity boundary. This guarantees entity integrity—no named entity straddles two chunks. Each resulting Chunk object carries an entity_ids list in its metadata dictionary, enabling downstream GraphRAG components to link chunks to specific knowledge graph nodes.

Registering and Configuring the Method

After implementing the logic, you must expose it to TextSplitter through the registration system and provide default configuration values.

Adding to SPLIT_METHODS

Edit semantica/split/methods.py to import and register your function:

# semantica/split/methods.py

from .entity_aware import entity_aware_split

SPLIT_METHODS = {
    # ... existing methods ...

    "entity_aware": entity_aware_split,
}

Now get_split_method("entity_aware") returns your function, making it available to TextSplitter instantiations.

Setting Default Options

Default parameters are read from the split configuration system (see line 47 in splitter.py). Add an entry to semantica/split/config.yaml (or extend the existing config object) to define sensible defaults:

entity_aware:
  chunk_size: 1000
  chunk_overlap: 200
  ner_method: spacy

The TextSplitter constructor merges these values into the options dict via _load_method_configs (lines 106-113 in splitter.py), allowing users to override them at runtime.

Usage Examples

Once registered, you can invoke entity-aware chunking through the standard TextSplitter interface.

Basic Entity-Aware Chunking

Instantiate the splitter with method="entity_aware" and pass the text:

from semantica.split import TextSplitter

splitter = TextSplitter(
    method="entity_aware",
    chunk_size=1200,
    chunk_overlap=250,
    ner_method="spacy",
)

text = "Albert Einstein was born in Ulm. He developed the theory of relativity..."
chunks = splitter.split(text)

for i, chunk in enumerate(chunks):
    print(f"Chunk {i+1}: {chunk.content}")
    print(f"Entities: {chunk.metadata['entity_ids']}")

Fallback Chains for Reliability

TextSplitter supports fallback chains starting at line 139 in splitter.py. If the entity-aware method fails (e.g., missing spaCy model), it automatically tries the next method:

splitter = TextSplitter(method=["entity_aware", "recursive"])

If entity_aware_split raises an exception, the dispatcher catches it and attempts the recursive splitter instead.

Integrating with GraphRAG Pipelines

Feed the chunk metadata into your knowledge graph to enable entity-to-chunk retrieval:

from semantica.split import TextSplitter

splitter = TextSplitter(method="entity_aware", ner_method="spacy")
chunks = splitter.split(long_document)

# Store in vector database and link to graph

for chunk in chunks:
    vector_id = vector_store.add_document(chunk.content)
    for entity_id in chunk.metadata["entity_ids"]:
        graph.create_edge(entity_id=entity_id, chunk_id=vector_id)

This creates bidirectional links between graph entities and text chunks, allowing GraphRAG queries to retrieve contextually relevant passages via entity traversal.

Summary

• Architecture: TextSplitter in semantica/split/splitter.py delegates to registered functions via get_split_method, enabling custom strategies without core code changes.
• Implementation: Create semantica/split/entity_aware.py with entity_aware_split that extracts entities first, then adjusts chunk boundaries to avoid splitting entity spans.
• Registration: Import the function into semantica/split/methods.py and add it to the SPLIT_METHODS dictionary to expose it via the method="entity_aware" parameter.
• Configuration: Define defaults in config.yaml or pass options directly to TextSplitter; the class merges these through _load_method_configs.
• GraphRAG Integration: Each returned Chunk contains an entity_ids metadata field that maps directly to knowledge graph nodes, enabling entity-aware retrieval.

Frequently Asked Questions

What is entity-aware chunking and why does it matter for GraphRAG?

Entity-aware chunking ensures that named entities (people, organizations, locations) never straddle chunk boundaries. In GraphRAG, if an entity like "New York City" is split across chunks, relationship extraction fails and graph queries return incomplete context. By aligning chunk boundaries to entity spans, you preserve the semantic units that graph databases index.

How does TextSplitter handle method failures?

TextSplitter implements fault tolerance through fallback chains. When you pass a list to the method parameter (e.g., method=["entity_aware", "recursive"]), the code loops through options starting at line 139 in splitter.py. If entity_aware_split raises an exception—such as a missing NER library—the dispatcher automatically tries the next method in the list.

Can I use different NER backends with entity_aware_split?

Yes. The _extract_entities helper function accepts a ner_method parameter that supports multiple backends. The example implementation includes spaCy via spacy.load("en_core_web_sm"), but you can extend the conditional logic to support HuggingFace transformers, Azure AI Language, or LLM-based extraction by adding elif branches that return the standard {"text": str, "start": int, "end": int} format.

How do I link chunks to a knowledge graph after splitting?

Each Chunk object returned by entity_aware_split contains a metadata dictionary with an entity_ids key listing the indices of entities present in that chunk. Iterate through the chunks and use these IDs to create edges in your graph database (Neo4j, Amazon Neptune, etc.) connecting the entity nodes to the chunk vector IDs stored in your retrieval system. This enables traversal from entities to their full textual context during GraphRAG inference.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}