# How to Configure chunk_token_size and chunk_overlap_token_size in LightRAG

> Learn how to configure chunk_token_size and chunk_overlap_token_size in LightRAG for optimal text splitting. Control text granularity and context preservation easily.

- Repository: [✨Data Intelligence Lab@HKU✨/LightRAG](https://github.com/HKUDS/LightRAG)
- Tags: how-to-guide
- Published: 2026-03-23

---

**LightRAG exposes `chunk_token_size` (default 1200) and `chunk_overlap_token_size` (default 100) through environment variables (`CHUNK_SIZE`, `CHUNK_OVERLAP_SIZE`), CLI flags (`--chunk-size`, `--chunk-overlap-size`), and Python constructor arguments to control text granularity and cross-chunk context preservation.**

LightRAG splits input documents into token-sized chunks before embedding generation and knowledge graph insertion. The `chunk_token_size` parameter defines the maximum tokens per segment, while `chunk_overlap_token_size` specifies how many tokens repeat across adjacent chunks to preserve semantic continuity. Proper configuration of these parameters lets you optimize retrieval accuracy, manage embedding API costs, and respect your vector store's token limits.

## Understanding the Text Splitting Parameters

Two dataclass fields in [`lightrag/lightrag.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/lightrag.py) govern document segmentation:

- **`chunk_token_size`**: The hard limit on tokens per chunk. When a document exceeds this value, LightRAG divides it into consecutive segments. The default is `1200` tokens, read from the `CHUNK_SIZE` environment variable if not provided explicitly.
- **`chunk_overlap_token_size`**: The number of tokens shared between consecutive chunks. This overlap ensures that phrases crossing a chunk boundary appear in both chunks, improving recall during retrieval. The default is `100` tokens, read from `CHUNK_OVERLAP_SIZE`.

The built-in splitter `chunking_by_token_size` (referenced via `LightRAG.chunking_func` receives these values as `max_tokens` and `overlap`, applying them after tokenization.

## Configuration Methods

LightRAG supports three configuration layers, allowing you to set defaults globally or override them per instance.

### Environment Variables

Set process-wide defaults that apply to all `LightRAG` instances:

```bash
export CHUNK_SIZE=800
export CHUNK_OVERLAP_SIZE=200
python -m lightrag.server --working-dir ./rag

```

These variables act as fallbacks when you do not provide explicit Python arguments or CLI flags.

### Python API Constructor

Override defaults for a specific instance by passing arguments directly to the `LightRAG` class:

```python
from lightrag import LightRAG

rag = LightRAG(
    working_dir="my_rag",
    llm_model_func=my_llm,
    embedding_func=my_embed,
    chunk_token_size=512,
    chunk_overlap_token_size=256
)
await rag.initialize_storages()

```

Constructor arguments take precedence over environment variables.

### Command Line Interface

When running the LightRAG server, the parser in [`lightrag/api/config.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/api/config.py) exposes dedicated flags:

```bash
python -m lightrag.server \
  --working-dir ./rag \
  --chunk-size 600 \
  --chunk-overlap-size 150 \
  --llm-binding openai \
  --embedding-binding openai

```

The configuration parser maps `--chunk-size` to `args.chunk_size` and `--chunk-overlap-size` to `args.chunk_overlap_size`, injecting these values into the `LightRAG` dataclass during startup.

## Implementation and Runtime Behavior

### Tokenization and Fallbacks

The default `chunking_by_token_size` function tokenizes content using `tiktoken` (or your configured tokenizer), then slices the token array into chunks of `max_tokens` length, stepping forward by `max_tokens - overlap` tokens each iteration. If the tokenizer fails to load, the function falls back to character-based splitting while still respecting the overlap size constraint.

### Dynamic Query-Time Adjustment

During retrieval operations, LightRAG calculates an `available_chunk_tokens` budget based on the query context and LLM token limits. As implemented in [`lightrag/operate.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/operate.py) (lines 4053-4068), this dynamic value passes as `chunk_token_limit` to the chunking routine, potentially reducing the effective chunk size below your configured `chunk_token_size` to prevent context window overflow.

## Custom Chunking Functions

For domain-specific splitting (e.g., by paragraphs or semantic boundaries), provide a callable matching the expected signature:

```python
def semantic_chunker(tokenizer, content, split_by_char, split_by_char_only,
                     overlap, max_tokens):
    """Preserve paragraph boundaries where possible."""
    paragraphs = content.split("\n\n")
    chunks = []
    current_tokens = []
    
    for para in paragraphs:
        para_tokens = tokenizer.encode(para)
        if len(current_tokens) + len(para_tokens) > max_tokens and current_tokens:
            # Flush current buffer

            chunks.append(tokenizer.decode(current_tokens))
            # Carry over overlap

            overlap_start = max(0, len(current_tokens) - overlap)
            current_tokens = current_tokens[overlap_start:] + para_tokens
        else:
            current_tokens.extend(para_tokens)
    
    if current_tokens:
        chunks.append(tokenizer.decode(current_tokens))
    
    return [{"content": c, "tokens": len(tokenizer.encode(c)), 
             "chunk_order_index": i} for i, c in enumerate(chunks)]

rag = LightRAG(
    working_dir="rag",
    chunk_token_size=1024,
    chunk_overlap_token_size=64,
    chunking_func=semantic_chunker
)

```

Your custom function receives `max_tokens` and `overlap` as configured, allowing you to implement alternative logic while respecting the user-specified constraints.

## Summary

- **Parameter defaults**: `chunk_token_size` defaults to `1200` (env: `CHUNK_SIZE`); `chunk_overlap_token_size` defaults to `100` (env: `CHUNK_OVERLAP_SIZE`).
- **Configuration hierarchy**: Constructor arguments override CLI flags, which override environment variables.
- **Source files**: Definitions reside in [`lightrag/lightrag.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/lightrag.py), CLI parsing in [`lightrag/api/config.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/api/config.py), and dynamic limits in [`lightrag/operate.py`](https://github.com/HKUDS/LightRAG/blob/main/lightrag/operate.py).
- **Chunking function**: Default `chunking_by_token_size` handles tokenization; fallback to character-based splitting occurs if `tiktoken` is unavailable.
- **Query adaptation**: Runtime token budgeting may temporarily reduce chunk size during retrieval to fit within LLM context windows.

## Frequently Asked Questions

### What happens if chunk_token_size exceeds my embedding model's limit?

LightRAG passes the full chunk to your configured `embedding_func`. If the chunk exceeds the model's maximum input tokens, the embedding API will likely truncate the text or return an error. Keep `chunk_token_size` within your specific model's limit (commonly 512, 1024, or 2048 tokens).

### Does increasing chunk_overlap_token_size consume more storage?

Yes. Overlap causes redundant token storage across adjacent chunks, increasing the total number of embeddings generated per document. While this improves retrieval recall by preserving boundary context, it increases vector database storage costs and insertion time.

### Can I modify chunk sizes for documents already in storage?

These parameters only affect new ingestion operations. Existing chunks remain at their original sizes. To apply new settings, clear your working directory or storage backend and re-process your documents.

### How does LightRAG handle documents smaller than chunk_token_size?

Documents containing fewer tokens than `chunk_token_size` are stored as single chunks. The `chunk_overlap_token_size` parameter only applies when content requires splitting into multiple segments.