# LangExtract Prompt Validation and Alignment: How Google Validates Few-Shot Prompts

> LangExtract validates few-shot prompt alignment verifying extraction annotations match source text using exact and fuzzy matching. Configure validation levels for warnings or prompt alignment errors.

- Repository: [Google/langextract](https://github.com/google/langextract)
- Tags: how-to-guide
- Published: 2026-02-16

---

**LangExtract validates few-shot prompt alignment by verifying that extraction annotations exactly match their source text using exact and fuzzy matching algorithms, with configurable validation levels that can trigger warnings or raise `PromptAlignmentError` exceptions.**

LangExtract is an open-source extraction framework developed by Google that enables structured data extraction from unstructured text using few-shot prompting. A critical challenge in few-shot learning is ensuring that the annotated examples in your prompt actually align with the text they claim to extract. This article examines how LangExtract implements robust prompt validation and alignment verification to prevent runtime extraction errors.

## Core Components of LangExtract Prompt Validation

The validation system in [`langextract/prompt_validation.py`](https://github.com/google/langextract/blob/main/langextract/prompt_validation.py) centers around three primary components that work together to verify prompt integrity.

### PromptValidationLevel Enum

The `PromptValidationLevel` enum (lines 44-50 in [`langextract/prompt_validation.py`](https://github.com/google/langextract/blob/main/langextract/prompt_validation.py)) defines three validation behaviors:

- **`OFF`** – Disables validation entirely
- **`WARNING`** – Logs alignment issues but continues execution
- **`ERROR`** – Raises `PromptAlignmentError` when misalignments are detected

### validate_prompt_alignment Function

The `validate_prompt_alignment()` function (lines 22-39) serves as the entry point for validation. It accepts a list of `ExampleData` objects and iterates through each example, using a `WordAligner` instance to verify that every extraction in the example aligns with the example's source text.

The function returns a `ValidationReport` object containing detailed information about any alignment failures or non-exact matches.

### handle_alignment_report Function

The `handle_alignment_report()` function (lines 112-128) consumes the `ValidationReport` and implements the policy defined by `PromptValidationLevel`. When the level is set to `ERROR`, it raises `PromptAlignmentError` for failed alignments. If `strict_non_exact=True`, it also raises errors for fuzzy or partial matches that would otherwise be accepted.

## How LangExtract Aligns Extractions with Source Text

The alignment engine resides in [`langextract/resolver.py`](https://github.com/google/langextract/blob/main/langextract/resolver.py) and uses a hybrid approach combining exact token matching with fuzzy fallback mechanisms.

### Exact Matching with SequenceMatcher

The `WordAligner.align_extractions()` method (lines 663-728) first attempts exact matching using Python's `difflib.SequenceMatcher`. The algorithm:

1. Tokenizes the source text and extraction texts using `_tokenize_with_lowercase()`
2. Builds token sequences with delimiters separating multiple extractions
3. Identifies matching blocks where tokens are identical
4. Assigns `AlignmentStatus.MATCH_EXACT` when a block covers the full extraction
5. Assigns `AlignmentStatus.MATCH_LESSER` when `accept_match_lesser=True` and the block is shorter than the extraction

### Fuzzy Matching Algorithm

When exact matching fails, the aligner optionally executes fuzzy alignment (lines implemented in `_fuzzy_align_extraction`):

1. Normalizes tokens via `_normalize_token()` (lowercasing and light stemming)
2. Slides a window over the source tokens
3. Measures overlap using `difflib.SequenceMatcher.ratio()`
4. Assigns `AlignmentStatus.MATCH_FUZZY` if the best ratio exceeds the configurable threshold (default **0.75**)

### Alignment Status Classification

The `AlignmentStatus` enum in [`langextract/core/data.py`](https://github.com/google/langextract/blob/main/langextract/core/data.py) defines four states:

- **`MATCH_EXACT`** – Perfect token alignment
- **`MATCH_FUZZY`** – Fuzzy match above threshold
- **`MATCH_LESSER`** – Partial exact match (shorter than extraction)
- **`NO_MATCH`** – No alignment found

## Building and Acting on Validation Reports

The validation system generates structured reports that enable precise debugging of prompt issues.

### ValidationIssue and ValidationReport Structure

Each alignment problem is captured as a `ValidationIssue` (lines 59-71 in [`prompt_validation.py`](https://github.com/google/langextract/blob/main/prompt_validation.py)) containing:

- `example_index`: Position in the few-shot list
- `example_id`: Optional identifier for the example
- `extraction_class`: The entity type being extracted
- `extraction_text_preview`: Truncated text of the extraction
- `alignment_status`: The actual status returned by the aligner
- `issue_kind`: Either `FAILED` (no match) or `NON_EXACT` (fuzzy/partial)

The `ValidationReport` wrapper (lines 86-101) provides `has_failed` and `has_non_exact` boolean properties for quick policy checks.

### Error Handling Based on Validation Level

The `handle_alignment_report()` function implements a three-tier response system:

1. **Silent mode (`OFF`)**: Returns immediately without inspection
2. **Warning mode (`WARNING`)**: Iterates through all issues and logs warnings using the `short_msg()` method of each `ValidationIssue`
3. **Strict mode (`ERROR`)**: Raises `PromptAlignmentError` with a detailed message if any `FAILED` issues exist; additionally raises if `strict_non_exact=True` and `NON_EXACT` issues are present

## Configuring Alignment Policy

The `AlignmentPolicy` dataclass (referenced in [`prompt_validation.py`](https://github.com/google/langextract/blob/main/prompt_validation.py) lines 107-113) allows fine-tuning of the alignment behavior:

```python
@dataclass
class AlignmentPolicy:
    enable_fuzzy_alignment: bool = True
    fuzzy_alignment_threshold: float = 0.75
    accept_match_lesser: bool = False

```

- **`enable_fuzzy_alignment`**: Toggle fuzzy fallback when exact matching fails
- **`fuzzy_alignment_threshold`**: Minimum similarity ratio (0.0-1.0) for fuzzy acceptance
- **`accept_match_lesser`**: Whether to accept partial exact matches as valid

When omitted, `validate_prompt_alignment()` uses default `AlignmentPolicy()` values.

## Practical Implementation Examples

### Basic Validation Pipeline

This example demonstrates the complete validation workflow using the high-level API:

```python
from langextract import resolver, prompt_validation as pv

# List of few-shot examples

examples = [...]  # List[data.ExampleData]

# Generate validation report

report = pv.validate_prompt_alignment(
    examples,
    aligner=resolver.WordAligner(),
    policy=pv.AlignmentPolicy(),
)

# Handle based on severity level

pv.handle_alignment_report(
    report,
    level=pv.PromptValidationLevel.ERROR,
    strict_non_exact=True,
)

```

When `PromptValidationLevel.ERROR` is specified, the function raises `PromptAlignmentError` if any extraction fails alignment or produces a non-exact match.

### Custom Fuzzy Threshold Configuration

For applications requiring stricter alignment verification:

```python
policy = pv.AlignmentPolicy(
    enable_fuzzy_alignment=True,
    fuzzy_alignment_threshold=0.90,  # Require 90% token overlap

    accept_match_lesser=False,
)

report = pv.validate_prompt_alignment(examples, policy=policy)
pv.handle_alignment_report(report, level=pv.PromptValidationLevel.WARNING)

```

### Low-Level WordAligner Usage

Direct access to the alignment engine for custom validation logic:

```python
from langextract.resolver import WordAligner, data

aligner = WordAligner()
source_text = "Alice went to Paris in 2022."

extractions = [
    data.Extraction(
        extraction_class="PERSON",
        extraction_text="Alice",
    ),
    data.Extraction(
        extraction_class="LOCATION",
        extraction_text="Paris",
    ),
]

aligned = aligner.align_extractions(
    extraction_groups=[extractions],
    source_text=source_text,
    token_offset=0,
    char_offset=0,
)

# Each result contains token_interval, char_interval, and alignment_status

for group in aligned:
    for extraction in group:
        print(f"{extraction.extraction_class}: {extraction.alignment_status}")

```

## Summary

- **LangExtract** validates few-shot prompts through a three-tier system controlled by `PromptValidationLevel` (`OFF`, `WARNING`, `ERROR`).
- The **`validate_prompt_alignment()`** function in [`langextract/prompt_validation.py`](https://github.com/google/langextract/blob/main/langextract/prompt_validation.py) orchestrates validation by iterating through examples and using `WordAligner` to verify extraction alignment.
- **Alignment** uses hybrid exact/fuzzy matching via `difflib.SequenceMatcher`, with fuzzy thresholds defaulting to **0.75** and configurable through `AlignmentPolicy`.
- **Validation reports** aggregate issues as `ValidationIssue` objects containing example indices, extraction previews, and `AlignmentStatus` classifications (`MATCH_EXACT`, `MATCH_FUZZY`, `MATCH_LESSER`, `NO_MATCH`).
- **Error handling** via `handle_alignment_report()` raises `PromptAlignmentError` for failed alignments when configured strictly, or logs warnings based on the selected validation level.

## Frequently Asked Questions

### What happens when LangExtract detects a misaligned extraction in strict mode?

When `PromptValidationLevel.ERROR` is enabled and `handle_alignment_report()` encounters a `FAILED` alignment issue, it raises `PromptAlignmentError` with a detailed message indicating which example index and extraction class failed to align. If `strict_non_exact=True`, it also raises errors for fuzzy or partial (`MATCH_LESSER`) matches that would otherwise be accepted as valid alignments.

### How does LangExtract handle case sensitivity and pluralization during alignment?

The `WordAligner` in [`langextract/resolver.py`](https://github.com/google/langextract/blob/main/langextract/resolver.py) normalizes tokens using `_tokenize_with_lowercase()` and `_normalize_token()` helpers (lines 744-770 and 803-810). These functions convert tokens to lowercase and apply light stemming to handle pluralization, ensuring that "Paris" and "paris" or "companies" and "company" can match during the fuzzy alignment phase.

### Can I disable fuzzy matching and require only exact alignments?

Yes, you can disable fuzzy matching by setting `enable_fuzzy_alignment=False` in the `AlignmentPolicy` dataclass. When disabled, the `WordAligner` will only attempt exact matching using `difflib.SequenceMatcher`. If an extraction does not produce an exact match block, it will be marked with `AlignmentStatus.NO_MATCH` regardless of how similar the text is to the source.

### What is the difference between MATCH_LESSER and MATCH_FUZZY in LangExtract?

`MATCH_LESSER` indicates that an exact token match was found, but the matching block is shorter than the full extraction text (partial match), and this is only accepted if `accept_match_lesser=True` in the policy. `MATCH_FUZZY` indicates that no exact match was found, but a fuzzy search using normalized token sliding windows achieved a similarity ratio above the configured threshold (default 0.75), indicating the extraction text is present in the source with minor variations.