How LangExtract's Resolver Parses and Aligns Extractions: A Deep Dive into the Core Pipeline

LangExtract's resolver transforms raw LLM output into structured extraction objects through a two-stage pipeline that first parses formatted text into dictionaries, then aligns each extraction to its precise location in the original source text using exact and fuzzy matching algorithms.

LangExtract, developed by Google, provides a robust framework for extracting structured information from unstructured text using large language models. At the heart of this system lies the resolver, a sophisticated component responsible for bridging the gap between free-form model outputs and precisely annotated data. Understanding how LangExtract's resolver parses and aligns extractions is essential for implementing reliable information extraction pipelines.

Understanding the Parsing Pipeline

The parsing stage converts raw model output—often wrapped in markdown code fences—into validated Extraction objects. This process involves format detection, strict validation modes, and ordered extraction creation.

Format Handling and Fence Detection

When the resolver receives raw LLM output, it delegates initial processing to the FormatHandler class located in langextract/core/format_handler.py. This handler automatically detects and strips markdown code fences (json, yaml, etc.) to reveal the underlying structured data.

The handler supports multiple serialization formats and includes legacy compatibility layers for older model outputs. If the initial parse fails due to malformed fences or embedded HTML tags, the handler attempts sanitization retries before raising parsing errors.

Strict vs. Lax Validation Modes

The resolver operates in two distinct validation modes controlled by the strict parameter:

• Strict mode (strict=True): Requires a wrapper object with an "extractions" key containing the list of extraction dictionaries. Top-level lists are explicitly rejected, ensuring schema compliance.
• Lax mode (strict=False): Accepts top-level lists or single objects, providing tolerance for models that omit wrapper structures.

This distinction ensures compatibility with both well-behaved models and those producing irregular output formats.

Extraction Object Creation

Once parsed, the Resolver.extract_ordered_extractions method (in langextract/resolver.py) iterates over the resulting dictionaries to construct data.Extraction objects. This process includes:

1. Type validation of extraction fields against the expected schema
2. Index suffix application (_index) to ensure deterministic ordering when multiple extractions share identical content
3. Metadata preservation from the original parse for downstream alignment

The resulting list of Extraction objects proceeds to the alignment stage with positional information yet to be determined.

How LangExtract Aligns Extractions with Source Text

The alignment stage locates each extraction within the original source text using a multi-tiered approach: exact matching, lesser matching, and fuzzy fallback algorithms.

Tokenization and Exact Matching

Both the source text and extraction strings undergo tokenization using the Tokenizer class (langextract/core/tokenizer.py). This process applies lower-casing and optional stemming to normalize text for comparison.

The WordAligner implementation utilizes difflib.SequenceMatcher to identify matching blocks between tokenized source and extraction texts:

• MATCH_EXACT: Awarded when a matching block length equals the full tokenized extraction length, indicating perfect alignment
• MATCH_LESSER: Assigned when accept_match_lesser=True and the matching block is shorter than the full extraction but still exceeds the minimum overlap threshold

These statuses provide confidence metrics for downstream consumers regarding alignment precision.

Fuzzy Alignment Fallback

When exact matching fails, the resolver invokes _fuzzy_align_extraction to locate extractions through approximate matching:

1. Window sliding: The algorithm slides a window across the source token list, examining every possible position
2. Early pruning: Windows that cannot meet the required token overlap (calculated via extraction_counts & window_counts) are immediately discarded
3. Stemmed comparison: For remaining candidates, a secondary SequenceMatcher operates on stemmed tokens to compute similarity ratios
4. Threshold application: The best window exceeding the fuzzy_alignment_threshold (default 0.75) receives the extraction
5. Interval mapping: Token and character intervals are calculated, and the extraction status is set to MATCH_FUZZY

This fallback ensures robust extraction even when models paraphrase or slightly alter the original text wording.

Complete Code Example

The following example demonstrates the full pipeline from raw LLM output to aligned extractions:

from langextract import Resolver, data

# Raw LLM output containing fenced JSON

raw_output = """```json
{
  "extractions": [
    {"drug": "ibuprofen", "dose": "200mg"},
    {"drug": "acetaminophen", "dose": "500mg"}
  ]
}

```"""

# Initialize resolver and parse output

resolver = Resolver()
extractions = resolver.resolve(raw_output, strict=True)

# Original source text for alignment

source = "Patient was prescribed ibuprofen 200mg and acetaminophen 500mg."

# Align extractions to source positions

aligned = list(
    resolver.align(
        extractions,
        source_text=source,
        token_offset=0,
        char_offset=0,
        enable_fuzzy_alignment=True,
        fuzzy_alignment_threshold=0.75,
    )
)

# Display alignment results

for ext in aligned:
    print(f"{ext.extraction_class}: {ext.extraction_text}")
    print(f"  Tokens: {ext.token_interval}")
    print(f"  Status: {ext.alignment_status}")
    print()

This implementation leverages the Resolver class from langextract/resolver.py to handle both parsing and alignment, producing Extraction objects with precise positional metadata.

Summary

• Two-stage pipeline: LangExtract's resolver first parses raw LLM output using FormatHandler to strip fences and validate structure, then aligns extractions to source text using token-based matching algorithms.
• Flexible validation: The strict parameter controls whether the parser requires wrapper objects ({"extractions": [...]}) or accepts raw lists.
• Hierarchical alignment: The system attempts exact matching first via difflib.SequenceMatcher, falls back to lesser matching if enabled, and finally applies fuzzy alignment using stemmed token windows with a default threshold of 0.75.
• Positional metadata: Successful alignments populate TokenInterval and CharInterval fields, enabling precise source text referencing for downstream applications.

Frequently Asked Questions

How does LangExtract handle markdown code fences in model outputs?

The FormatHandler class in langextract/core/format_handler.py automatically detects and strips markdown code fences (such as json or yaml) from raw LLM output. If the initial parsing fails due to malformed fences or embedded HTML tags, the handler attempts sanitization retries before raising errors, ensuring robust extraction across different model formatting behaviors.

What is the difference between strict and lax parsing modes in LangExtract?

When strict=True (the default in many configurations), the resolver requires a wrapper object containing an "extractions" key with a list of extraction dictionaries, explicitly rejecting top-level lists. When strict=False, the parser operates in lax mode, accepting top-level lists or single objects without the wrapper structure. This flexibility accommodates models that may omit strict schema compliance.

How does LangExtract align extractions when exact text matching fails?

When exact matching via difflib.SequenceMatcher fails, LangExtract invokes _fuzzy_align_extraction which slides a window across the source token list. It prunes windows lacking sufficient token overlap, then applies a secondary SequenceMatcher on stemmed tokens to compute similarity ratios. The best window exceeding the fuzzy_alignment_threshold (default 0.75) receives the extraction with MATCH_FUZZY status, enabling alignment despite paraphrasing or minor text variations.

What alignment statuses can extractions receive in LangExtract?

Extractions can receive three distinct alignment statuses defined in langextract/core/data.py: MATCH_EXACT indicates perfect token-level alignment where the matching block equals the full extraction length; MATCH_LESSER indicates partial alignment when accept_match_lesser=True and the match is shorter than the full extraction; and MATCH_FUZZY indicates approximate alignment achieved through the fuzzy fallback algorithm when exact matching fails.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}