# Optimizing LangExtract Performance for Long Documents: 8 Proven Strategies

> Boost LangExtract performance for long documents. Learn 8 proven strategies to tune chunk buffers, batch sizes, and worker pools for efficient parallel processing and reduced API calls.

- Repository: [Google/langextract](https://github.com/google/langextract)
- Tags: performance
- Published: 2026-02-16

---

**Optimize LangExtract performance for long documents by tuning chunk buffers, batch sizes, and worker pools to minimize API calls while maximizing parallel throughput.**

The `google/langextract` library processes lengthy texts by splitting them into manageable chunks that fit within model context windows. When working with long documents, understanding the internal tokenization and chunking pipeline is essential for optimizing LangExtract performance and controlling inference costs.

## How LangExtract Processes Long Documents

### Tokenization and Text Chunking

In [`langextract/core/tokenizer.py`](https://github.com/google/langextract/blob/main/langextract/core/tokenizer.py), the `RegexTokenizer` class converts raw text into tokens with character offsets. The `ChunkIterator` class in [`langextract/chunking.py`](https://github.com/google/langextract/blob/main/langextract/chunking.py) then groups these tokens into `TextChunk` objects while respecting the `max_char_buffer` limit. The helper function `_tokens_exceed_buffer` enforces buffer boundaries and handles sentence boundaries and newline breaks.

### Batching and Parallel Execution

The `make_batches_of_textchunk` function in [`langextract/chunking.py`](https://github.com/google/langextract/blob/main/langextract/chunking.py) packs consecutive chunks into batches for single inference calls. Parallel execution is orchestrated by `annotate_text` and `annotate_documents` in [`langextract/annotation.py`](https://github.com/google/langextract/blob/main/langextract/annotation.py), which launch a `ThreadPoolExecutor` when `max_workers` is greater than 1.

## Core Parameters for LangExtract Performance Optimization

### Buffer Size and Chunk Management

The `max_char_buffer` parameter controls how many characters each chunk contains. Larger buffers reduce the total number of chunks and API calls, but increase latency per request and risk truncation. For English prose, 1500–2000 characters balances efficiency and context preservation.

### Parallel Processing Configuration

The `max_workers` parameter sets the number of concurrent threads in the `ThreadPoolExecutor`. Set this to match your CPU core count using `os.cpu_count()`, but remain below API rate limits. The `batch_length` parameter determines how many chunks are sent per request. Set `batch_length` equal to or greater than `max_workers` to prevent underutilization.

### Context Windows and Multiple Passes

The `context_window_chars` parameter adds overlap between chunks to improve coreference resolution across boundaries. A small overlap of 50–100 characters usually suffices without significantly increasing chunk count. The `extraction_passes` parameter reprocesses documents to boost recall, but multiplies token costs. Use 2–3 passes only when extraction accuracy is critical.

## Practical Optimization Strategies

1. **Increase `max_char_buffer` modestly** – Test with sample documents to find the largest buffer that avoids model truncation. This directly reduces the number of `TextChunk` objects created.

2. **Match `batch_length` to `max_workers`** – Ensure `batch_length` is at least equal to `max_workers`. The code warns when `batch_length < max_workers` because it wastes parallel capacity.

3. **Add minimal context overlap** – Enable `context_window_chars` with 100 characters to fix pronoun resolution without bloating the chunk count.

4. **Limit extraction passes** – Start with one pass. Increase to two only if logs show missed entities, as each pass doubles token consumption.

5. **Select appropriate tokenizers** – Use `RegexTokenizer` for English text in [`langextract/core/tokenizer.py`](https://github.com/google/langextract/blob/main/langextract/core/tokenizer.py). Switch to `UnicodeTokenizer` for CJK scripts where character boundaries differ.

6. **Right-size worker pools** – Set `max_workers` based on `os.cpu_count()` but verify against your API provider's rate limits to avoid throttling.

## Performance-Focused Code Example

```python
import langextract as lx
import os
import textwrap

prompt = textwrap.dedent("""\
    Extract all medication names, dosage amounts, and administration routes.
    Return results as a JSON array of objects with fields:
      - name
      - dosage
      - route
""")

examples = [...]  # your ExampleData objects

result = lx.extract(
    text_or_documents="https://example.com/large_clinical_report.txt",
    prompt_description=prompt,
    examples=examples,
    max_char_buffer=1500,          # larger chunks → fewer API calls

    batch_length=20,               # match workers for better parallelism

    max_workers=8,                 # use available CPU cores

    context_window_chars=100,      # small overlap for coreference

    extraction_passes=2,           # second pass to catch missed meds

    tokenizer=lx.core.tokenizer.RegexTokenizer(),  # fast default tokenizer

)

```

This configuration optimizes LangExtract performance by:

- Reducing chunk count by approximately 50% compared to the default 1000-character buffer
- Maximizing throughput with 8 concurrent workers processing 20-chunk batches
- Maintaining extraction accuracy with minimal context overlap and a second verification pass

## Monitoring Performance Impact

Enable detailed logging by setting `debug=True` in your extraction call. LangExtract writes timing data to `extractor_debug.log` via [`langextract/core/debug_utils.py`](https://github.com/google/langextract/blob/main/langextract/core/debug_utils.py). Look for these log entries:

```

[DEBUG] ChunkIterator created with 342 chunks (max_char_buffer=1500)
[INFO] 8 workers processing 20-chunk batches → total requests: 18
[INFO] Extraction completed in 12.3s, cost: 0.032 USD

```

Iteratively adjust `max_char_buffer`, `batch_length`, and `max_workers` based on these metrics until you achieve your target latency and cost constraints.

## Summary

- LangExtract processes long documents by tokenizing text and chunking it into `TextChunk` objects via `ChunkIterator` in [`langextract/chunking.py`](https://github.com/google/langextract/blob/main/langextract/chunking.py)
- **Increase `max_char_buffer`** to reduce API call volume, balancing against model context limits
- **Match `batch_length` to `max_workers`** to fully utilize the `ThreadPoolExecutor` in [`langextract/annotation.py`](https://github.com/google/langextract/blob/main/langextract/annotation.py)
- **Use `context_window_chars`** sparingly (50–100 characters) to improve coreference without bloating chunks
- **Limit `extraction_passes`** to control costs—start with one pass and increase only if recall is insufficient
- **Monitor via `debug=True`** and `extractor_debug.log` to measure chunk counts, request batches, and total latency

## Frequently Asked Questions

### What is the optimal `max_char_buffer` size for processing long documents?

For English prose, 1500–2000 characters typically provides the best balance between reducing API calls and avoiding model truncation. For noisy or multi-language text, keep the buffer at or below 1000 characters to prevent token overflow. Test with representative samples from your corpus to find the upper limit your specific model can handle.

### How does `batch_length` interact with `max_workers` for parallel processing?

The `batch_length` parameter determines how many `TextChunk` objects are sent per inference request, while `max_workers` controls how many concurrent threads run in the `ThreadPoolExecutor` defined in [`langextract/annotation.py`](https://github.com/google/langextract/blob/main/langextract/annotation.py). For optimal throughput, set `batch_length` equal to or greater than `max_workers`. If `batch_length` is smaller, the system warns that parallel capacity is being wasted because workers will sit idle waiting for batches.

### When should I use multiple `extraction_passes` versus a single pass?

Use multiple `extraction_passes` only when extraction recall is critical and you can tolerate multiplied token costs. Each pass re-runs the full annotation pipeline via `annotator.annotate_text`, effectively doubling or tripling your API usage. Start with a single pass and inspect results; if entities are consistently missed at chunk boundaries or in complex sections, increase to two passes. For most high-volume applications, a single pass with an optimized `context_window_chars` overlap provides sufficient accuracy.

### Which tokenizer should I choose for non-English documents?

For documents containing Chinese, Japanese, or Korean (CJK) text, use `UnicodeTokenizer` instead of the default `RegexTokenizer`. The default tokenizer in [`langextract/core/tokenizer.py`](https://github.com/google/langextract/blob/main/langextract/core/tokenizer.py) is optimized for space-separated languages like English, whereas `UnicodeTokenizer` handles character boundaries in East Asian scripts more accurately, reducing token count and improving chunk alignment. For other non-Latin scripts (Arabic, Devanagari, etc.), evaluate both tokenizers on a sample to see which produces fewer tokens per character.