Entity-Aware, Relation-Aware, and Graph-Based Chunking in Semantica’s TextSplitter

Semantica’s TextSplitter provides three advanced chunking strategies—entity-aware, relation-aware, and graph-based—that preserve semantic coherence by respecting named entity boundaries, subject-predicate-object triplets, and knowledge graph community structures.

The semantica-agi/semantica repository implements sophisticated text segmentation algorithms that move beyond traditional character or token-based splitting. These entity-aware, relation-aware, and graph-based chunking methods ensure that downstream knowledge graph construction and GraphRAG pipelines receive semantically intact units rather than arbitrarily sliced text.

Understanding the Three Chunking Strategies

Entity-Aware Chunking

Entity-aware chunking uses Named Entity Recognition (NER) to identify and preserve complete entity spans across chunk boundaries. The algorithm locates entity spans within the text, groups surrounding content up to the target chunk size, and adjusts boundaries to avoid cutting entities mid-sentence.

The core implementation resides in semantica/split/methods.py, where the split_entity_aware function accepts parameters including ner_method (supporting "ml", "regex", or "llm") and preserve_entities to control boundary behavior source. This ensures that named entities like "New York City" or "OpenAI" remain unsplit across chunks, maintaining context for downstream entity linking tasks.

Relation-Aware Chunking

Relation-aware chunking extracts subject-predicate-object triples from text and ensures each triplet remains intact within a single chunk. This preserves relational context critical for knowledge graph extraction pipelines where broken triples would lose semantic meaning.

The split_relation_aware function in semantica/split/methods.py implements this logic, extracting triples using configurable relation extraction methods and grouping them while respecting chunk_size limits source. By keeping triples like ("Elon Musk", "founded", "SpaceX") together, this method supports accurate knowledge graph construction from chunked text.

Graph-Based Chunking

Graph-based chunking leverages the structure of the underlying knowledge graph—using centrality metrics or community detection algorithms—to define chunk boundaries that align with dense sub-graphs. This approach segments text according to topological relationships rather than linear position.

The split_graph_based function in semantica/split/methods.py supports strategies such as "community" or "centrality" with algorithms including "louvain" or "leiden" source. By analyzing the temporary graph built from extracted entities and relations, this method produces chunks that correspond to semantically coherent knowledge communities.

Implementation Architecture

The chunking system separates concerns between high-level chunker classes and low-level splitting algorithms. The semantica/split/kg_chunkers.py module defines EntityAwareChunker, RelationAwareChunker, and GraphBasedChunker classes that orchestrate the splitting process and track progress source source.

These classes utilize the Chunk dataclass from semantic_chunker.py to encapsulate text segments with metadata including start/end indices and preservation flags source. Users can also invoke these chunkers via the CLI using the --strategy option to select "entity-aware", "relation-aware", or "graph-based" approaches source.

Practical Usage Examples

All three chunkers implement a consistent interface for easy integration:

from semantica.split.kg_chunkers import (
    EntityAwareChunker,
    RelationAwareChunker,
    GraphBasedChunker,
)

# Entity-aware with LLM-based NER

entity_chunker = EntityAwareChunker(
    chunk_size=1200,
    chunk_overlap=200,
    ner_method="llm"
)
chunks = entity_chunker.chunk(text)

# Relation-aware preserving triplets

relation_chunker = RelationAwareChunker(
    chunk_size=1000,
    relation_method="ml"
)
triplet_chunks = relation_chunker.chunk(text)

# Graph-based using Louvain community detection

graph_chunker = GraphBasedChunker(
    chunk_size=1500,
    strategy="community",
    algorithm="louvain"
)
graph_chunks = graph_chunker.chunk(text)

Each call returns a List[Chunk] where each chunk contains the text content, byte-level start and end indices, and metadata dictionaries capturing the chunking strategy applied.

Summary

• Entity-aware chunking preserves named entity spans using configurable NER methods to prevent boundary cuts within entities.
• Relation-aware chunking maintains subject-predicate-object triple integrity essential for knowledge graph extraction.
• Graph-based chunking utilizes community detection or centrality algorithms to align chunks with knowledge graph topology.
• All strategies implement a unified interface through kg_chunkers.py with core algorithms residing in methods.py.
• The Chunk dataclass standardizes output across all methods, supporting metadata-rich downstream processing.

Frequently Asked Questions

How does entity-aware chunking differ from standard sentence-based splitting?

Standard sentence-based splitting divides text at punctuation boundaries without considering semantic units. Entity-aware chunking specifically identifies named entities using methods like "ml", "regex", or "llm" and adjusts chunk boundaries to keep entire entities within single chunks, preventing context loss for downstream NER or entity linking tasks.

Can relation-aware chunking handle complex multi-hop relationships?

The current implementation focuses on preserving individual subject-predicate-object triplets within chunk boundaries. While it ensures single-hop relational integrity, complex multi-hop relationships spanning multiple triplets may require graph-based chunking, which uses community detection to group related triples into cohesive chunks.

What graph algorithms are available for graph-based chunking?

According to the source code in semantica/split/methods.py, graph-based chunking supports strategies like "community" and "centrality" with algorithms including "louvain" and "leiden". These algorithms analyze the temporary knowledge graph built from extracted entities and relations to determine optimal split points.

How do I configure the overlap between chunks in these specialized chunkers?

All three chunkers accept standard parameters chunk_size and chunk_overlap (defaulting to 1000 and 200 characters respectively). The overlap applies after semantic boundary adjustment, ensuring that context continuity is maintained even when strict entity or relation preservation creates hard boundaries.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}