How to Handle Errors in Strict vs Non-Strict Mode in Sieves

In Sieves, the strict flag in ModelSettings determines whether parsing failures raise a RuntimeError to halt execution or return None to allow the pipeline to continue processing documents.

The mantisai/sieves library provides configurable error handling for AI-powered document processing pipelines. When integrating large language models, you must decide whether malformed outputs should crash your workflow or degrade gracefully. Understanding how to handle errors in strict vs non-strict mode in Sieves allows you to build resilient data pipelines that match your reliability requirements.

Understanding Strict Mode Configuration

The error handling behavior is controlled by the strict attribute in the ModelSettings class. Defined in sieves/model_wrappers/types.py, this boolean flag defaults to True, meaning the pipeline operates in strict mode unless explicitly configured otherwise. When strict=True, any exception during model inference or output parsing propagates immediately to the caller.

Error Handling Execution Path

The enforcement of strict mode occurs within the _infer method of the ModelWrapper base class in sieves/model_wrappers/core.py. This method orchestrates prompt generation, model execution, and result parsing within a comprehensive try/except block.

The _infer Method Core Logic

The _infer method processes batches of documents by building prompts, invoking the underlying model generator, and attempting to parse structured outputs. It captures all exceptions during this process to determine whether to fail fast or continue processing based on the strict configuration.

Strict Mode Failure Behavior

When self._strict evaluates to True and an exception occurs, the wrapper raises a RuntimeError with a descriptive message indicating the model and the specific failure. This immediately halts pipeline execution and prevents partial or corrupted results from propagating downstream.

Non-Strict Mode Fallback

When operating with strict=False, the except block swallows the exception and returns a placeholder tuple of (None, None, TokenUsage()) for each failed input. This allows the pipeline to continue processing subsequent documents while marking the failed ones with None values in their results dictionary.

Pipeline-Level Outcomes

The choice between strict and non-strict mode determines how downstream tasks receive data. In strict mode, a single parsing failure aborts the entire batch, ensuring data integrity at the cost of reliability. In non-strict mode, failed documents retain their place in the output list but contain None for the failed task key, enabling partial success patterns and error recovery workflows.

Practical Implementation Examples

Configure error handling by passing a ModelSettings instance to any predictive task. The following examples demonstrate both behaviors using the classification task with the Outlines model wrapper.

from sieves import Doc, Pipeline, ModelSettings
from sieves.model_wrappers import ModelType
from sieves.tasks.predictive import classification

# 1️⃣  Strict mode – pipeline will raise on parsing errors

strict_pipe = Pipeline(
    [
        classification.Classification(
            label_enum=MyLabels,
            model=ModelType.outlines,               # any supported wrapper

            model_settings=ModelSettings(strict=True),
        )
    ]
)

try:
    list(strict_pipe([Doc(text="corrupt response")]))
except RuntimeError as e:
    print("Pipeline stopped:", e)

# 2️⃣  Non‑strict mode – pipeline continues, result is None

lenient_pipe = Pipeline(
    [
        classification.Classification(
            label_enum=MyLabels,
            model=ModelType.outlines,
            model_settings=ModelSettings(strict=False),
        )
    ]
)

docs = list(lenient_pipe([Doc(text="corrupt response")]))
print(docs[0].results["Classification"])   # → None

Validating Behavior with Tests

The test suite in sieves/tests/test_strict_mode.py validates both execution paths across multiple model backends. The tests confirm that strict mode raises exceptions and returns empty document lists, while non-strict mode tolerates failures and returns documents with None results, ensuring consistent behavior regardless of the underlying model provider.

Summary

• The strict flag in ModelSettings controls whether parsing failures raise RuntimeError or return None.
• Strict mode (strict=True) aborts pipeline execution immediately upon any parsing error.
• Non-strict mode (strict=False) allows the pipeline to continue, marking failed documents with None values and returning (None, None, TokenUsage()) from the wrapper.
• Configure error handling per task by passing ModelSettings(strict=...) to any predictive task constructor.

Frequently Asked Questions

What is the default strict mode setting in Sieves?

By default, ModelSettings.strict is set to True in sieves/model_wrappers/types.py, meaning the pipeline operates in strict mode and will raise a RuntimeError on any parsing failure unless explicitly configured otherwise.

How do I enable non-strict mode for a specific task?

Instantiate ModelSettings with strict=False and pass it to the task's model_settings parameter, such as Classification(model_settings=ModelSettings(strict=False)). This configuration applies only to that specific task within the pipeline.

What happens to failed documents in non-strict mode?

In non-strict mode, documents that fail parsing receive a None value in their results dictionary for that specific task key, while the pipeline continues processing remaining documents. The wrapper returns (None, None, TokenUsage()) for each failed input.

Can I mix strict and non-strict modes in the same pipeline?

Yes, each task in a Sieves pipeline can have its own ModelSettings instance, allowing you to configure strict error handling for critical tasks and lenient handling for optional tasks within the same workflow. This granular control lets you balance data integrity requirements against pipeline reliability.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}