# How LangExtract Chunking, Parallel Processing, and Multi-Pass Strategy Work: A Deep Dive

> Explore how LangExtract's chunking, parallel processing, and multi-pass strategy enhance text extraction recall. Learn about ThreadPoolExecutor and batch processing.

- Repository: [Google/langextract](https://github.com/google/langextract)
- Tags: deep-dive
- Published: 2026-02-16

---

**LangExtract breaks documents into text chunks, processes them in parallel batches using ThreadPoolExecutor, and optionally runs multiple extraction passes to merge non-overlapping results for improved recall.**

LangExtract is an open-source Python library from Google that structures unstructured text using large language models (LLMs). Understanding how LangExtract chunking, parallel processing, and multi-pass strategy work is essential for optimizing extraction pipelines on large documents. This article examines the core implementation in [`langextract/chunking.py`](https://github.com/google/langextract/blob/main/langextract/chunking.py), [`langextract/annotation.py`](https://github.com/google/langextract/blob/main/langextract/annotation.py), and the provider modules to reveal how the framework scales from raw text to structured data.

## LangExtract Chunking: From Raw Text to TextChunk Objects

The chunking layer transforms input documents into discrete `TextChunk` objects that respect character limits, sentence boundaries, and token constraints. This process ensures that no single LLM prompt exceeds the model's context window while preserving semantic coherence.

### Tokenization and Sentence Detection

The `ChunkIterator` class in **[`langextract/chunking.py`](https://github.com/google/langextract/blob/main/langextract/chunking.py)** drives the initial text segmentation. It first converts raw text into a `TokenizedText` object, then iterates over sentences using the iterator protocol. This sentence-aware approach prevents the chunker from cutting mid-sentence, which is critical for maintaining extraction quality on entity-rich text.

### Creating TextChunk Objects with ChunkIterator

Inside `ChunkIterator.__next__` (lines 41-88 of [`chunking.py`](https://github.com/google/langextract/blob/main/chunking.py)), the algorithm accumulates sentences until reaching `max_char_buffer`, then yields a `TextChunk` pydantic model containing the text span, start/end indices, and metadata. The chunker respects newline boundaries and token limits, ensuring that each `TextChunk` is a self-contained unit suitable for independent LLM processing.

### Batching Chunks for LLM Processing

Once individual chunks are created, the `make_batches_of_textchunk` function (lines 265-280 in [`chunking.py`](https://github.com/google/langextract/blob/main/chunking.py)) groups consecutive `TextChunk` objects into fixed-size batches. The `batch_length` parameter controls how many chunks are packed into a single LLM request. This batching layer is crucial for efficiency: it reduces API overhead by sending multiple text segments in one call while maintaining the granularity needed for parallel processing.

## Parallel Processing Architecture in LangExtract

LangExtract leverages Python's `concurrent.futures` module to execute LLM calls across multiple threads, significantly reducing latency when processing large batches of chunks.

### ThreadPoolExecutor for Concurrent LLM Calls

Parallelism occurs at the provider level. When the number of prompts in a batch exceeds 1 and the provider's `max_workers` configuration is greater than 1, the framework initializes a `ThreadPoolExecutor`. This executor dispatches each prompt to a separate worker thread, allowing multiple LLM requests to run simultaneously against the API.

The `max_workers` parameter acts as an upper bound on concurrency. The actual thread count is the minimum of `max_workers` and the number of prompts in the current batch, preventing resource waste on small batches.

### Provider-Specific Implementation (Gemini and OpenAI)

The parallel processing logic is implemented consistently across providers. In **[`langextract/providers/gemini.py`](https://github.com/google/langextract/blob/main/langextract/providers/gemini.py)** (lines 6-38), the provider checks batch size and `max_workers`, then either runs prompts sequentially or distributes them via `ThreadPoolExecutor`. The code preserves result ordering by indexing the `results` list before yielding `ScoredOutput` objects.

The OpenAI provider in **[`langextract/providers/openai.py`](https://github.com/google/langextract/blob/main/langextract/providers/openai.py)** mirrors this architecture, accepting `max_workers` as a constructor argument and using identical thread-pool logic for concurrent API calls. This design ensures that switching between Gemini and OpenAI models does not affect parallelization behavior.

## Multi-Pass Extraction Strategy

While chunking and parallelism optimize throughput, the multi-pass strategy improves recall by running the extraction pipeline multiple times and merging unique results.

### Sequential Pass Loop and Extraction Passes

The `extraction_passes` parameter in the `extract` function controls how many times the annotation layer processes the document. Inside **[`langextract/annotation.py`](https://github.com/google/langextract/blob/main/langextract/annotation.py)**, the `_annotate_documents_sequential_passes` function implements this by iterating `for pass_idx in range(extraction_passes)` and running the full single-pass pipeline (`_annotate_documents_single_pass`) on each iteration.

Each pass re-tokenizes the text, rebuilds prompts (including any sliding-window context from `context_window_chars`), and queries the LLM independently. This repetition allows the model multiple opportunities to identify entities that might have been missed due to prompt phrasing or attention window limitations in previous passes.

### Merging Non-Overlapping Results

After completing all passes, the framework calls `_merge_non_overlapping_extractions` (also in [`annotation.py`](https://github.com/google/langextract/blob/main/annotation.py)) to consolidate results. This function compares extractions across passes and retains non-overlapping entities. When overlaps occur, the merger prefers the first occurrence, ensuring deterministic output while maximizing coverage.

The merging logic operates on the principle that valid extractions found in any pass are likely true positives, while missed entities in one pass might be caught in another. This union approach significantly improves recall without requiring a larger context window or more expensive model calls.

### Why Multi-Pass Improves Recall

Multi-pass extraction addresses the inherent variability in LLM attention mechanisms. By varying the chunk boundaries slightly on each pass (due to re-tokenization) and giving the model multiple independent attempts at the same text, the system surfaces entities that might fall outside the model's focal area in any single inference. The automatic deduplication ensures this increased recall does not introduce redundant outputs.

## Complete Implementation Example

The following example demonstrates how to configure chunking, parallel processing, and multi-pass extraction in a single `extract` call:

```python
from langextract import extract

# Example clinical text

text = """
Dr. Alice Smith prescribed 10 mg of Lisinopril to John Doe.
She mentioned that the medication helped his blood pressure.
"""

# Configure extraction with chunking, parallelism, and multi-pass

result = extract(
    text_or_documents=text,
    prompt_description="Extract medication entities and patient information.",
    examples=[...],                     # List[ExampleData] defining the schema

    model_id="gemini-1.5-pro",         # Any supported model (Gemini or OpenAI)

    max_char_buffer=300,               # Chunk size: max characters per TextChunk

    batch_length=5,                    # Up to 5 chunks per batch (LLM call)

    max_workers=4,                     # Parallel LLM calls via ThreadPoolExecutor

    extraction_passes=2,               # Two sequential passes for improved recall

    context_window_chars=100,          # Cross-chunk coreference context

)

print(result.extractions)   # Structured list of Medication, Person, etc.

```

**Key configuration parameters explained:**

- **`max_char_buffer`**: Determines the maximum characters a single `TextChunk` may contain before the `ChunkIterator` yields it.
- **`batch_length`**: Controls how many `TextChunk` objects are grouped into one LLM request by `make_batches_of_textchunk`.
- **`max_workers`**: Sets the upper bound of threads the provider (Gemini or OpenAI) will use in its `ThreadPoolExecutor` for parallel inference.
- **`extraction_passes`**: Specifies how many times the `annotation` layer runs the full pipeline, merging results via `_merge_non_overlapping_extractions`.

## Summary

LangExtract combines three architectural strategies to balance throughput, cost, and accuracy:

- **Chunking**: The `ChunkIterator` in [`langextract/chunking.py`](https://github.com/google/langextract/blob/main/langextract/chunking.py) segments documents into `TextChunk` objects respecting sentence boundaries and `max_char_buffer`, while `make_batches_of_textchunk` groups them for efficient LLM batching.
- **Parallel Processing**: Both the Gemini and OpenAI providers utilize `ThreadPoolExecutor` (when `max_workers` > 1 and batch size > 1) to dispatch multiple prompts concurrently, significantly reducing latency for large documents.
- **Multi-Pass Extraction**: The `annotation` module runs the pipeline `extraction_passes` times via `_annotate_documents_sequential_passes`, then merges non-overlapping results using `_merge_non_overlapping_extractions` to maximize recall without duplication.

## Frequently Asked Questions

### How does LangExtract handle very large documents that exceed LLM context windows?

LangExtract uses the `ChunkIterator` class in [`langextract/chunking.py`](https://github.com/google/langextract/blob/main/langextract/chunking.py) to automatically segment large documents into `TextChunk` objects that respect `max_char_buffer` limits. Each chunk is processed independently (or in batches) through the LLM, ensuring no single prompt exceeds the model's context window while maintaining sentence-level coherence.

### What is the difference between `batch_length` and `max_workers` in LangExtract?

`batch_length` (controlled via `make_batches_of_textchunk` in [`chunking.py`](https://github.com/google/langextract/blob/main/chunking.py)) determines how many `TextChunk` objects are grouped into a single LLM API call, affecting request efficiency. `max_workers` (implemented in [`langextract/providers/gemini.py`](https://github.com/google/langextract/blob/main/langextract/providers/gemini.py) and [`openai.py`](https://github.com/google/langextract/blob/main/openai.py)) controls the number of threads in the `ThreadPoolExecutor` used to process multiple batches or prompts concurrently, affecting parallel throughput.

### How does multi-pass extraction improve results without causing duplicates?

When `extraction_passes` is greater than 1, the `annotation` module runs the full extraction pipeline multiple times via `_annotate_documents_sequential_passes`. After all passes complete, `_merge_non_overlapping_extractions` (in [`annotation.py`](https://github.com/google/langextract/blob/main/annotation.py)) compares results across passes and retains only unique, non-overlapping entities, preferring the first occurrence when overlaps exist. This union approach improves recall by giving the LLM multiple independent attempts at the text while automatic deduplication prevents redundant outputs.