# How to Set the Chunking Engine in Cognee: CLI and Python API Guide

> Easily set the chunking engine in Cognee using the CLI chunker flag or Python API chunker parameter. Explore TextChunker, LangchainChunker, and CsvChunker options now.

- Repository: [Topoteretes/cognee](https://github.com/topoteretes/cognee)
- Tags: how-to-guide
- Published: 2026-03-16

---

**Pass the `--chunker` flag in the CLI or provide the `chunker` parameter to `cognee.cognify()` to switch between TextChunker, LangchainChunker, or CsvChunker.**

Cognee is an open-source graph RAG framework that uses a **chunking engine** to partition raw documents into manageable text segments before processing them through LLM-based pipelines. The framework supports multiple chunking strategies that you can configure at runtime through either command-line arguments or direct API calls. Understanding how to set the chunking engine in Cognee allows you to optimize tokenization granularity for your specific document types and downstream vector stores.

## Understanding Cognee's Chunking Architecture

The chunking engine selection determines how documents are split into `DocumentChunk` objects during the ingestion pipeline. According to the Cognee source code, the system supports three distinct chunking implementations defined in [`cognee/cli/config.py`](https://github.com/topoteretes/cognee/blob/main/cognee/cli/config.py):

```python

# cognee/cli/config.py

CHUNKER_CHOICES = ["TextChunker", "LangchainChunker", "CsvChunker"]

```

These chunkers are instantiated dynamically based on your configuration and are passed through the pipeline to document-type-specific readers in `cognee/modules/data/processing/document_types/`.

## Setting the Chunking Engine via CLI

The simplest way to set the chunking engine in Cognee is through the `cognee-cli cognify` command. The CLI handler is implemented in [`cognee/cli/commands/cognify_command.py`](https://github.com/topoteretes/cognee/blob/main/cognee/cli/commands/cognify_command.py), which parses the `--chunker` argument and resolves it to the appropriate class.

### Supported CLI Options

The CLI recognizes the following chunker names:

- **TextChunker**: Default paragraph-based splitting (no external dependencies)
- **LangchainChunker**: Uses LangChain's `RecursiveCharacterTextSplitter` with overlap support
- **CsvChunker**: Row-based splitting for tabular data ingestion

### CLI Usage Example

To process documents with the LangchainChunker, run:

```bash
cognee-cli cognify \
    --chunker LangchainChunker \
    --chunk-size 512 \
    --chunks-per-batch 100 \
    path/to/documents

```

Behind the scenes, [`cognee/cli/commands/cognify_command.py`](https://github.com/topoteretes/cognee/blob/main/cognee/cli/commands/cognify_command.py) dynamically imports the requested class:

```python

# cognee/cli/commands/cognify_command.py (excerpt)

if args.chunker == "LangchainChunker":
    from cognee.modules.chunking.LangchainChunker import LangchainChunker
    chunker_class = LangchainChunker
elif args.chunker == "CsvChunker":
    from cognee.modules.chunking.CsvChunker import CsvChunker
    chunker_class = CsvChunker
else:
    chunker_class = TextChunker

```

The selected `chunker_class` is then forwarded to the cognify pipeline defined in [`cognee/pipelines.py`](https://github.com/topoteretes/cognee/blob/main/cognee/pipelines.py).

## Setting the Chunking Engine via Python API

For programmatic control, you can set the chunking engine by passing a chunker class (not an instance) to the high-level `cognify` function in [`cognee/api/v1/cognify/cognify.py`](https://github.com/topoteretes/cognee/blob/main/cognee/api/v1/cognify/cognify.py).

### The cognify Function Signature

The `chunker` parameter accepts a callable class definition:

```python

# cognee/api/v1/cognify/cognify.py (excerpt)

async def cognify(
    user: User = Depends(get_authenticated_user),
    documents: List[Document],
    chunker: Callable = TextChunker,
    chunk_size: Optional[int] = None,
    chunks_per_batch: Optional[int] = None,
    ...
):
    await run_cognify_pipeline(
        user=user,
        documents=documents,
        chunker=chunker,
        chunk_size=chunk_size,
        chunks_per_batch=chunks_per_batch,
    )

```

### Python Usage Example

To use a custom chunking engine in your Python code:

```python
import asyncio
from cognee.api.v1.cognify.cognify import cognify
from cognee.modules.chunking.LangchainChunker import LangchainChunker
from cognee.modules.data.processing.document_types import TextDocument

async def main():
    docs = [TextDocument("example.txt")]
    
    await cognify(
        user=None,
        documents=docs,
        chunker=LangchainChunker,  # Set the chunking engine

        chunk_size=1024,
        chunks_per_batch=50,
    )

if __name__ == "__main__":
    asyncio.run(main())

```

## How Chunkers Work Internally

Once you set the chunking engine, the system instantiates your chosen class within document-specific readers. For example, [`TextDocument.py`](https://github.com/topoteretes/cognee/blob/main/TextDocument.py) implements the `read` method that accepts the chunker class:

```python

# cognee/modules/data/processing/document_types/TextDocument.py (excerpt)

async def read(self, chunker_cls: Chunker, max_chunk_size: int):
    chunker = chunker_cls(self, max_chunk_size=max_chunk_size, get_text=get_text)
    async for chunk in chunker.read():
        yield chunk

```

This pattern is consistent across all document types including [`UnstructuredDocument.py`](https://github.com/topoteretes/cognee/blob/main/UnstructuredDocument.py) and [`PdfDocument.py`](https://github.com/topoteretes/cognee/blob/main/PdfDocument.py). The chunker's `read()` coroutine yields `DocumentChunk` objects that flow into graph-building and vector-indexing stages.

## Available Chunking Engines

Cognee provides three built-in chunking engines located in `cognee/modules/chunking/`:

**TextChunker** ([`TextChunker.py`](https://github.com/topoteretes/cognee/blob/main/TextChunker.py))

The default engine that performs simple paragraph-based splitting. This is the most reliable option with no external dependencies, ideal for general text documents.

**LangchainChunker** ([`LangchainChunker.py`](https://github.com/topoteretes/cognee/blob/main/LangchainChunker.py))

Leverages LangChain's `RecursiveCharacterTextSplitter` with configurable overlap. Better suited for long, unstructured text where you need semantic boundaries preserved across chunks.

**CsvChunker** ([`CsvChunker.py`](https://github.com/topoteretes/cognee/blob/main/CsvChunker.py))

Specialized for tabular data, splitting CSV rows into separate chunks. Essential when ingesting structured datasets that require row-level granularity in your knowledge graph.

## Summary

- **Set the chunking engine in Cognee** by passing `--chunker` to the CLI or the `chunker` parameter to `cognify()`.
- Available options are `TextChunker` (default), `LangchainChunker`, and `CsvChunker`, defined in [`cognee/cli/config.py`](https://github.com/topoteretes/cognee/blob/main/cognee/cli/config.py).
- The CLI handler in [`cognee/cli/commands/cognify_command.py`](https://github.com/topoteretes/cognee/blob/main/cognee/cli/commands/cognify_command.py) dynamically imports chunker classes based on the string name provided.
- In the Python API, pass the chunker class (not an instance) to the `chunker` argument in [`cognee/api/v1/cognify/cognify.py`](https://github.com/topoteretes/cognee/blob/main/cognee/api/v1/cognify/cognify.py).
- Document readers in `cognee/modules/data/processing/document_types/` instantiate the chunker class and yield chunks through the `read()` coroutine.

## Frequently Asked Questions

### How do I change the default chunker for all operations?

Pass the desired chunker class explicitly in every API call or CLI command. Cognee does not currently persist a global configuration file for chunking defaults, but you can wrap the `cognify` function in your own module that presets `chunker=LangchainChunker`.

### What is the difference between TextChunker and LangchainChunker?

**TextChunker** uses simple paragraph boundaries and requires no external dependencies, making it faster for basic text. **LangchainChunker** uses `RecursiveCharacterTextSplitter` with overlap support, providing better semantic coherence for complex documents but requiring the LangChain library.

### Can I use a custom chunking engine not listed in CHUNKER_CHOICES?

Yes. When using the Python API, you can pass any callable class that implements the chunker interface to the `chunker` parameter. The CLI restricts choices to `CHUNKER_CHOICES` for safety, but programmatic usage accepts custom implementations as long as they match the expected signature in `cognee/modules/data/processing/document_types/`.

### Does the chunk size parameter affect all chunking engines?

Yes. The `chunk_size` and `chunks_per_batch` parameters are forwarded to the pipeline regardless of which engine you set. However, specific chunkers may interpret `chunk_size` differently—LangchainChunker uses it as a token limit, while CsvChunker may use it as a row limit depending on implementation.