# How LangExtract Handles Source Grounding and Text Alignment: A Technical Deep Dive

> Discover how LangExtract grounds LLM extractions to source text. Learn about its three-stage pipeline using exact and fuzzy alignment for accurate results. Explore char_interval token_interval and alignment_status.

- Repository: [Google/langextract](https://github.com/google/langextract)
- Tags: deep-dive
- Published: 2026-02-16

---

**LangExtract grounds LLM extractions to source text by assigning character and token intervals through a three-stage pipeline that uses exact token matching with `difflib.SequenceMatcher` and optional fuzzy alignment, storing results in the `Extraction` object's `char_interval`, `token_interval`, and `alignment_status` fields.**

LangExtract is an open-source Python library developed by Google that transforms unstructured LLM outputs into structured data while maintaining traceability to the original source text. Understanding how LangExtract handles source grounding and text alignment is essential for building reliable extraction pipelines that can verify model outputs against their documentary origins.

## The Three-Stage Grounding Pipeline

LangExtract implements source grounding and text alignment through a systematic three-stage workflow defined in the `Resolver` class within [`langextract/resolver.py`](https://github.com/google/langextract/blob/main/langextract/resolver.py).

### Stage 1: Parsing Raw LLM Output

The process begins in the `Resolver.resolve()` method, which delegates to a `FormatHandler` to parse JSON or YAML strings into a list of `Extraction` objects. At this initial stage, extractions contain semantic content but lack positional metadata.

### Stage 2: Token-Level Alignment Strategies

The `Resolver.align()` method triggers the `WordAligner.align_extractions()` implementation, which performs source grounding and text alignment using two complementary strategies:

- **Exact Token Matching**: The aligner first attempts exact token-level matching using `difflib.SequenceMatcher` to compare token sequences between the extraction and source text.
- **Fuzzy Alignment Fallback**: When exact matching fails, the system optionally falls back to fuzzy alignment that scans sliding windows of source tokens, selecting the best overlap above a configurable threshold.

### Stage 3: Recording Character Intervals and Alignment Status

For every successful match, the aligner populates three critical fields on each `Extraction` object defined in [`langextract/core/data.py`](https://github.com/google/langextract/blob/main/langextract/core/data.py):

- `token_interval`: A `TokenInterval` containing start/end token indices relative to the chunk.
- `char_interval`: A `CharInterval` containing start/end character offsets in the original string.
- `alignment_status`: An `AlignmentStatus` enum value describing the match quality.

## Understanding Alignment Statuses

The `AlignmentStatus` enum in [`langextract/core/data.py`](https://github.com/google/langextract/blob/main/langextract/core/data.py) defines four possible outcomes for source grounding and text alignment:

- `MATCH_EXACT`: Tokens line up perfectly between extraction and source.
- `MATCH_GREATER`: Extraction matches a longer span than expected.
- `MATCH_LESSER`: Extraction is longer than the matched source span.
- `MATCH_FUZZY`: Overlap ratio meets or exceeds the configurable fuzzy threshold.

When no interval can be determined, both `char_interval` and `alignment_status` remain `None`, marking the extraction as **ungrounded**. This distinction is critical for downstream validation, as demonstrated in [`benchmarks/benchmark.py`](https://github.com/google/langextract/blob/main/benchmarks/benchmark.py) where grounded versus ungrounded entities are counted by checking `extraction.char_interval`.

## Practical Code Examples

### Example 1: Basic Grounding with Exact Matching

The following example demonstrates standard source grounding and text alignment using the default resolver:

```python
from langextract import Resolver, data

# Model-generated JSON (normally from an LLM)

model_output = """
{
  "person": "Ada Lovelace",
  "occupation": "mathematician"
}
"""

# Resolve to Extraction objects (no positions yet)

resolver = Resolver()
extractions: list[data.Extraction] = resolver.resolve(model_output)

# Align to the original source paragraph

source = "Ada Lovelace was a pioneering mathematician and writer."
aligned = list(
    resolver.align(
        extractions,
        source_text=source,
        token_offset=0,
        char_offset=0,
        enable_fuzzy_alignment=True,
    )
)

# Inspect grounding information

for ext in aligned:
    print(f"{ext.extraction_class!r}: {ext.extraction_text!r}")
    print(f"  Char interval: {ext.char_interval}")
    print(f"  Token interval: {ext.token_interval}")
    print(f"  Alignment status: {ext.alignment_status}\n")

```

**Typical output:**

```

'person': 'Ada Lovelace'
  Char interval: CharInterval(start_pos=0, end_pos=12)
  Token interval: TokenInterval(start_index=0, end_index=2)
  Alignment status: AlignmentStatus.MATCH_EXACT

'occupation': 'mathematician'
  Char interval: CharInterval(start_pos=27, end_pos=39)
  Token interval: TokenInterval(start_index=5, end_index=6)
  Alignment status: AlignmentStatus.MATCH_EXACT

```

### Example 2: Fuzzy Alignment for Sub-Phrases

When extractions contain partial matches, fuzzy alignment bridges the gap:

```python
source = "The quick brown fox jumps over the lazy dog."

# Extraction missing the article "the"

extractions = [
    data.Extraction(
        extraction_class="animal",
        extraction_text="lazy dog",
    )
]

resolver = Resolver()
aligned = list(
    resolver.align(
        extractions,
        source_text=source,
        token_offset=0,
        char_offset=0,
        enable_fuzzy_alignment=True,
        fuzzy_alignment_threshold=0.7,
    )
)

print(aligned[0].char_interval)      # CharInterval(start_pos=35, end_pos=43)

print(aligned[0].alignment_status)   # AlignmentStatus.MATCH_FUZZY

```

### Example 3: Detecting Ungrounded Extractions

Identify hallucinations or failed matches by checking for `None` intervals:

```python
from langextract import Resolver, data

source = "Paris is the capital of France."
extractions = [
    data.Extraction(extraction_class="city", extraction_text="Berlin"),
]

resolver = Resolver()
aligned = list(
    resolver.align(
        extractions,
        source_text=source,
        token_offset=0,
        char_offset=0,
        enable_fuzzy_alignment=False,
    )
)

print(aligned[0].char_interval)      # None

print(aligned[0].alignment_status)   # None (ungrounded)

```

## Key Implementation Files

The source grounding and text alignment system spans several critical files in the `google/langextract` repository:

| File | Purpose |
|------|---------|
| [`langextract/core/data.py`](https://github.com/google/langextract/blob/main/langextract/core/data.py) | Defines `Extraction`, `CharInterval`, `TokenInterval`, and the `AlignmentStatus` enum. |
| [`langextract/resolver.py`](https://github.com/google/langextract/blob/main/langextract/resolver.py) | Contains the `Resolver` class with the `align()` method and the inner `WordAligner` class implementing exact and fuzzy matching. |
| [`langextract/core/tokenizer.py`](https://github.com/google/langextract/blob/main/langextract/core/tokenizer.py) | Provides tokenization utilities and `TokenInterval` definitions used during alignment. |
| [`tests/resolver_test.py`](https://github.com/google/langextract/blob/main/tests/resolver_test.py) | Validates grounding accuracy across exact, lesser, fuzzy, and missing alignment scenarios. |
| [`benchmarks/benchmark.py`](https://github.com/google/langextract/blob/main/benchmarks/benchmark.py) | Demonstrates production usage by distinguishing grounded from ungrounded extractions. |

## Summary

LangExtract implements a robust source grounding and text alignment pipeline that bridges the gap between unstructured LLM outputs and traceable data extractions:

- **Three-stage process**: Parse model output into `Extraction` objects, align tokens using exact or fuzzy matching via `WordAligner`, and record positional metadata.
- **Dual alignment strategies**: Exact token matching with `difflib.SequenceMatcher` provides precision, while fuzzy sliding-window matching handles partial phrases and minor variations.
- **Comprehensive metadata**: Each extraction receives `char_interval`, `token_interval`, and `alignment_status` fields, enabling downstream validation and source slicing.
- **Ungrounded detection**: Failed alignments result in `None` values for intervals and status, allowing applications to filter hallucinations or mismatched extractions as demonstrated in the benchmark suite.

## Frequently Asked Questions

### What is the difference between exact and fuzzy alignment in LangExtract?

Exact alignment uses `difflib.SequenceMatcher` to find perfect token-level matches between the extraction text and source text, resulting in `MATCH_EXACT` status. Fuzzy alignment activates when exact matching fails, scanning sliding windows of source tokens to find the best partial overlap above a configurable threshold, resulting in `MATCH_FUZZY` status. Fuzzy alignment accommodates extractions that omit articles or minor words present in the source text.

### How does LangExtract represent ungrounded extractions?

When the `WordAligner` cannot locate the extraction text in the source through either exact or fuzzy matching, it leaves the `char_interval` and `token_interval` fields as `None` and sets `alignment_status` to `None`. These ungrounded extractions indicate potential hallucinations or entities drawn from the model's training data rather than the provided context, allowing downstream applications to filter or flag them accordingly.

### What files contain the core alignment logic?

The primary alignment implementation resides in [`langextract/resolver.py`](https://github.com/google/langextract/blob/main/langextract/resolver.py), specifically within the `Resolver.align()` method and the inner `WordAligner` class. Data structures supporting grounding—including `Extraction`, `CharInterval`, `TokenInterval`, and `AlignmentStatus`—are defined in [`langextract/core/data.py`](https://github.com/google/langextract/blob/main/langextract/core/data.py). Tokenization utilities used during alignment are located in [`langextract/core/tokenizer.py`](https://github.com/google/langextract/blob/main/langextract/core/tokenizer.py). Comprehensive tests validating these behaviors exist in [`tests/resolver_test.py`](https://github.com/google/langextract/blob/main/tests/resolver_test.py).

### Can I adjust the fuzzy alignment threshold?

Yes, the fuzzy alignment threshold is configurable through the `fuzzy_alignment_threshold` parameter in the `Resolver.align()` method. By default, LangExtract requires a high overlap ratio for fuzzy matches, but you can lower this threshold to accommodate more aggressive partial matching. However, reducing the threshold increases the risk of false positives where extractions align to semantically incorrect portions of the source text.