How to Use LangExtract with Local Ollama Models: A Complete Guide

LangExtract treats Ollama as a standard language model provider, allowing you to extract structured data using any local model by passing the model ID to lx.extract() and configuring the Ollama-specific format handler.

LangExtract is Google's open-source library for extracting structured information from unstructured text using large language models. When you want to run extractions entirely on your own hardware without sending data to external APIs, you can use LangExtract with local Ollama models through the dedicated provider interface implemented in langextract/providers/ollama.py.

How LangExtract Integrates with Ollama

The integration follows LangExtract's provider architecture defined in langextract/core/base_model.py. The OllamaLanguageModel class implements the abstract BaseLanguageModel interface, enabling the router in langextract/providers/router.py to automatically select the Ollama backend when it detects an Ollama model ID pattern.

When you invoke lx.extract() with an Ollama model, the library:

1. Routes to the Ollama provider based on the model ID string (e.g., gemma2:2b)
2. Constructs the API request targeting http://localhost:11434/api/generate by default
3. Merges runtime parameters such as temperature and timeout via BaseLanguageModel.merge_kwargs
4. Handles JSON mode through the OLLAMA_FORMAT_HANDLER to ensure responses conform to expected extraction schemas

Basic Usage of LangExtract with Local Ollama Models

One-Line Extraction Example

The simplest way to use LangExtract with Ollama requires only the model ID and the format handler:

import langextract as lx

result = lx.extract(
    text_or_documents="Marie Curie was a physicist and chemist who conducted pioneering research on radioactivity.",
    model_id="gemma2:2b",
    prompt_description="Extract the person's name and profession.",
    resolver_params={"format_handler": lx.providers.ollama.OLLAMA_FORMAT_HANDLER},
)

print(result.extractions)

This example automatically instantiates the OllamaLanguageModel provider, sends the request to your local Ollama server, and returns a list of Extraction objects containing the structured data.

Using a Pre-Instantiated Provider

For scenarios requiring custom configuration—such as non-default ports, authentication headers, or specific timeout values—instantiate the provider directly:

from langextract.providers.ollama import OllamaLanguageModel, OLLAMA_FORMAT_HANDLER
import langextract as lx

ollama_model = OllamaLanguageModel(
    model_id="gemma2:2b",
    model_url="http://localhost:11434",
    timeout=180,
    keep_alive=300,
)

result = lx.extract(
    text_or_documents="The patient was prescribed 500mg amoxicillin three times daily for seven days.",
    model=ollama_model,
    prompt_description="Extract medication name, dosage, and duration.",
    resolver_params={"format_handler": OLLAMA_FORMAT_HANDLER},
)

The model parameter accepts the pre-configured instance, bypassing the automatic router selection while preserving all extraction functionality.

Complete Working Examples

Running the Official Demo

The repository includes a comprehensive demonstration script at examples/ollama/demo_ollama.py that showcases real-world usage patterns:

# Install LangExtract with visualization dependencies

pip install "langextract[all]"

# Ensure Ollama is running locally

ollama pull gemma2:2b
ollama serve

# Execute the demonstration

python examples/ollama/demo_ollama.py

This demo script illustrates four distinct extraction scenarios, displays progress bars via show_progress=True, and generates both JSONL output and interactive HTML visualizations using lx.visualize.

Advanced Configuration with Custom Parameters

For deployments behind proxies or with authentication requirements, configure the provider with additional headers and custom endpoints:

ollama_model = OllamaLanguageModel(
    model_id="llama3.1:8b",
    model_url="http://my-proxy:11434",
    api_key="sk-proxy-token",
    timeout=300,
)

result = lx.extract(
    text_or_documents="Apple Inc. was founded by Steve Jobs, Steve Wozniak, and Ronald Wayne in 1976.",
    model=ollama_model,
    prompt_description="Extract company name, founders, and founding year.",
    resolver_params={"format_handler": OLLAMA_FORMAT_HANDLER},
)

All optional parameters are forwarded to the underlying HTTP request layer, allowing full control over the network configuration while maintaining the abstraction provided by BaseLanguageModel.

Understanding the Ollama Provider Architecture

The OllamaLanguageModel class in langextract/providers/ollama.py implements several critical methods defined in BaseLanguageModel:

• _ollama_query – Constructs the JSON payload for the /api/generate endpoint, adds the format field for JSON mode, and handles HTTP communication using the requests library
• merge_kwargs – Inherited from BaseLanguageModel, combines runtime arguments (temperature, timeout) with stored configuration
• parse_output – Converts raw string responses into Python structures, defaulting to JSON parsing

The OLLAMA_FORMAT_HANDLER constant provides a pre-configured FormatHandler instance that sets use_wrapper=True, instructing the resolver to expect a JSON dictionary root rather than a plain list. This aligns with Ollama's JSON mode behavior, which always returns a JSON object.

Error handling wraps standard HTTP exceptions into LangExtract-specific types: InferenceRuntimeError for request failures and InferenceConfigError for configuration issues, both defined in the core exception hierarchy.

Summary

• LangExtract integrates with Ollama through the OllamaLanguageModel provider in langextract/providers/ollama.py, which implements the BaseLanguageModel interface
• Use lx.extract() with any Ollama model ID (e.g., gemma2:2b, llama3.1:8b) and include resolver_params={"format_handler": lx.providers.ollama.OLLAMA_FORMAT_HANDLER} for correct JSON parsing
• For custom configurations, instantiate OllamaLanguageModel directly with parameters like model_url, timeout, api_key, and keep_alive
• The provider communicates with Ollama's /api/generate endpoint at http://localhost:11434 by default, handling JSON mode and error wrapping automatically

Frequently Asked Questions

Do I need to modify LangExtract source code to use Ollama models?

No. LangExtract supports Ollama natively through the provider system. Simply install LangExtract, ensure Ollama is running locally, and pass an Ollama model ID (like gemma2:2b) to lx.extract(). The router automatically selects the OllamaLanguageModel provider based on the model ID pattern.

Why do I need to specify the OLLAMA_FORMAT_HANDLER?

Ollama's JSON mode always returns a JSON object (dictionary) rather than a raw list. The OLLAMA_FORMAT_HANDLER (defined in langextract/providers/ollama.py) tells the resolver to expect this wrapped format with use_wrapper=True. Without this handler, the parser may fail to interpret the Ollama response structure correctly.

Can I use LangExtract with Ollama running on a different machine or port?

Yes. Instantiate the OllamaLanguageModel class directly with the model_url parameter pointing to your custom endpoint. For example: OllamaLanguageModel(model_id="llama3.1:8b", model_url="http://192.168.1.100:11434"). You can also add authentication headers via the api_key parameter if your Ollama instance sits behind a proxy.

What happens if my Ollama server is not running or returns an error?

LangExtract wraps HTTP errors and Ollama-specific failures in InferenceRuntimeError or InferenceConfigError exceptions. If the server is unreachable, you'll receive an InferenceRuntimeError indicating the connection failure. If the model ID is invalid or the response format is unexpected, an InferenceConfigError provides details about the configuration issue.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}