# How ModelManager Selects the Best AI Model Based on Context Size in Open Notebook

> Discover how Open Notebook's ModelManager intelligently selects AI models based on context size, routing large inputs to specialized models and handling smaller workloads efficiently.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-12

---

**Open Notebook's ModelManager automatically routes inputs exceeding 105,000 tokens to a specialized large-context AI model while respecting explicit model overrides and standard defaults for smaller workloads.**

The `lfnovo/open-notebook` repository implements an intelligent model selection pipeline that dynamically chooses the appropriate AI model based on input token count. The system relies on a hard-coded threshold and hierarchical decision logic to prevent context window overflow and ensure optimal performance across varying content sizes.

## Token Counting and Threshold Detection

The selection process begins with the `token_count` helper function in [`open_notebook/utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils.py), which calculates the exact number of tokens in the input content. This count drives the decision logic located in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) (lines 10-35).

The system uses a hard-coded threshold of **105,000 tokens**. When content exceeds this limit, the system categorizes the request as requiring "large_context" handling. This threshold represents the boundary where standard models risk context window exhaustion, automatically triggering the promotion to specialized high-capacity models configured for massive inputs.

## The Selection Logic in provision_langchain_model

The `provision_langchain_model` function implements a three-tier decision hierarchy that determines which model configuration to instantiate:

1. **Large Context Override**: If `tokens > 105_000`, the function logs "large_context" as the `selection_reason` and calls `await model_manager.get_default_model("large_context", **kwargs)`
2. **Explicit Model ID**: If the caller provides a specific `model_id`, the system bypasses automatic selection entirely and uses `await model_manager.get_model(model_id, **kwargs)`
3. **Standard Default**: For normal token counts without explicit overrides, the system uses `await model_manager.get_default_model(default_type, **kwargs)` where `default_type` typically equals "chat" or "embedding"

The `selection_reason` variable captures the rationale for debugging purposes, while the actual model instantiation occurs through the ModelManager's async methods.

## Resolving Large Context Models via ModelManager

When handling large context requests, `ModelManager.get_default_model` (defined in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py), lines 221-247) queries the `DefaultModels` record to retrieve the configured large-context model ID.

The system checks the `large_context_model` field defined in the `DefaultModels` dataclass (lines 62-68). If configured, the manager fetches the corresponding `Model` record, constructs the **Esperanto** configuration, and returns a `LanguageModel` instance ready for LangChain integration via the `to_langchain()` method.

If no `large_context_model` is configured in Settings, `get_default_model` returns `None`, causing `provision_langchain_model` to raise a `ConfigurationError` directing users to configure the large context model in the Settings UI.

## Practical Implementation Examples

The following examples demonstrate how to leverage the automatic selection system in application code:

```python
from open_notebook.ai.provision import provision_langchain_model

async def process_document(text: str):
    # Automatically selects large_context model if text > 105k tokens

    model = await provision_langchain_model(
        content=text,
        model_id=None,
        default_type="chat",
        temperature=0.7,
    )
    # Returns a LangChain BaseChatModel ready for .invoke()

    return model

```

For direct access to the large-context model without token counting:

```python
from open_notebook.ai.models import model_manager

async def get_high_capacity_model():
    model = await model_manager.get_default_model("large_context")
    if model is None:
        raise RuntimeError("Configure large_context_model in Settings")
    return model.to_langchain()

```

To bypass automatic selection and force a specific model regardless of token count:

```python

# Explicit model_id overrides the 105k threshold check

model = await provision_langchain_model(
    content=large_text,
    model_id="open_notebook:model:1234",
    default_type="chat"
)

```

## Summary

- **Token Threshold**: Inputs exceeding 105,000 tokens automatically trigger the large-context branch in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py)
- **Hierarchical Selection**: The system prioritizes explicit model IDs, then large-context detection, then standard defaults
- **Configuration Dependency**: Large-context handling requires setting `large_context_model` in the `DefaultModels` record via the Settings UI
- **LangChain Integration**: All paths return LangChain-compatible models through the `to_langchain()` method
- **Debug Visibility**: The `selection_reason` variable logs the rationale for model selection to aid troubleshooting

## Frequently Asked Questions

### What happens if no large-context model is configured?

If `DefaultModels.large_context_model` is unset when content exceeds 105,000 tokens, `ModelManager.get_default_model` returns `None`, causing `provision_langchain_model` to raise a `ConfigurationError` with instructions to configure the model in Settings.

### Can I force a specific model even for large inputs?

Yes. Providing an explicit `model_id` parameter to `provision_langchain_model` bypasses the automatic token-count check entirely, allowing any configured model to handle the content regardless of size.

### Where is the 105,000 token threshold defined?

The threshold is hard-coded in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) within the `provision_langchain_model` function (lines 10-35) as the constant `105_000`.

### How does ModelManager instantiate the selected model?

`ModelManager.get_default_model` retrieves the model ID from `DefaultModels`, fetches the corresponding database record, builds an Esperanto configuration object, and wraps it as a LangChain-compatible model through the `to_langchain()` method.