# How Context Window Size Affects Model Selection in Open Notebook

> Discover how context window size impacts model selection in Open Notebook. Learn how large inputs trigger dedicated models for optimal performance.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: deep-dive
- Published: 2026-06-30

---

**Open Notebook automatically routes requests to a dedicated large-context model when input exceeds 105,000 tokens, while standard requests use the default chat or transformation model configured in the system.**

When building AI applications that process varying document lengths, managing context window limitations is critical for both performance and cost. In the `lfnovo/open-notebook` repository, the framework implements an intelligent provisioning system that dynamically selects the appropriate language model based on the size of the incoming content. This mechanism ensures that normal conversations use cheaper, standard models while automatically upgrading to expensive, high-capacity models only when processing very long documents.

## The 105,000 Token Threshold

The decision point for context-window sizing is hard-coded at **105,000 tokens** in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py). This threshold acts as a gatekeeper between standard and large-context processing paths.

When the `provision_langchain_model` function receives content, it immediately calculates the token count:

```python
tokens = token_count(content)  # line 19

```

If this value exceeds 105,000, the system logs the selection reason and triggers the large-context branch:

```python
if tokens > 105_000:  # lines 23-28

    selection_reason = f"large_context (content has {tokens} tokens)"
    logger.debug(...)
    model = await model_manager.get_default_model("large_context", **kwargs)

```

Requests below this threshold bypass the large-context logic and proceed to standard model selection paths.

## Token Counting Mechanism

Before model selection occurs, the framework measures the payload using the `token_count` helper imported from [`open_notebook/utils/__init__.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/__init__.py). This utility analyzes the raw content string and returns an integer representing the estimated token count.

The provisioning function stores this count and uses it as the primary determinant for the subsequent routing logic. According to the source analysis, this counting happens at line 19 of [`provision.py`](https://github.com/lfnovo/open-notebook/blob/main/provision.py), making it the first operation in the model selection pipeline.

## Model Selection Logic in provision.py

The `provision_langchain_model` function implements a three-tiered decision tree in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) (lines 10-38):

1. **Large-context branch**: Activated when `tokens > 105_000`, fetching the model configured for `"large_context"` type
2. **Explicit override branch**: Activated when a `model_id` parameter is provided, bypassing token checks entirely
3. **Default type branch**: Fallback for standard requests, using the `default_type` parameter (e.g., `"chat"` or `"transformation"`)

This structure ensures that token size is evaluated before any other selection criteria, preventing context window overflow errors that would occur if a standard model attempted to process 105,000+ tokens.

## Configuring the Large-Context Default

The identity of the large-context model is stored in the **`DefaultModels`** dataclass defined in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py). Specifically, line 66 defines the optional field:

```python
large_context_model: Optional[str] = None

```

When the token threshold is exceeded, `ModelManager.get_default_model` (lines 46-48) reads this field and retrieves the corresponding model configuration. If `large_context_model` is not set in the database, the manager returns `None`, which subsequently triggers a `ConfigurationError` during the provisioning flow.

Administrators can configure this default through the API layer exposed in [`api/routers/models.py`](https://github.com/lfnovo/open-notebook/blob/main/api/routers/models.py) or programmatically via the `model_manager` interface.

## Explicit Model Overrides

Users and developers can bypass the automatic context-window logic entirely by supplying a `model_id` parameter. When provided, the provisioning function skips the token count check and loads the specified model directly:

```python
elif model_id:  # lines 29-32

    selection_reason = f"explicit model_id={model_id}"
    model = await model_manager.get_model(model_id, **kwargs)

```

This override mechanism is useful for testing specific models or forcing smaller, cheaper models for cost control even when processing long documents.

## Code Examples

### Automatic Selection Based on Content Size

The following example demonstrates how Open Notebook automatically handles varying document lengths without manual intervention:

```python
from open_notebook.ai.provision import provision_langchain_model

async def process_document(content: str):
    # Let Open Notebook decide based on token count

    llm = await provision_langchain_model(
        content=content,
        model_id=None,
        default_type="chat",
        temperature=0.7,
    )
    # If content > 105,000 tokens, uses large_context_model

    # Otherwise uses the default chat model

    response = await llm.ainvoke({"messages": [{"role": "user", "content": content}]})
    return response

```

### Forcing a Specific Model

To override the automatic selection and force a specific model regardless of document length:

```python
llm = await provision_langchain_model(
    content=very_long_text,
    model_id="open_notebook:model:anthropic:claude-2",
    default_type="chat",
)

```

### Configuring the Large-Context Default

Administrators can set the large-context model programmatically:

```python
from open_notebook.ai.models import model_manager, DefaultModels

async def configure_large_context():
    defaults = await model_manager.get_defaults()
    defaults.large_context_model = "open_notebook:model:openai:gpt-4-32k"
    await defaults.save()

```

After execution, any request exceeding 105,000 tokens will automatically route to `gpt-4-32k` or the configured equivalent.

## Summary

- Open Notebook uses a **hard-coded threshold of 105,000 tokens** in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) to determine when to switch to large-context models.
- The **`token_count`** function from [`open_notebook/utils/__init__.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/__init__.py) measures input size before model selection occurs.
- When the threshold is exceeded, the system retrieves the model ID stored in **`DefaultModels.large_context_model`** via `ModelManager.get_default_model`.
- Providing an explicit **`model_id`** parameter bypasses the token check and uses the specified model regardless of content size.
- If no large-context model is configured, the system emits a warning and raises a `ConfigurationError` for oversized requests.

## Frequently Asked Questions

### What happens if no large-context model is configured?

If the token count exceeds 105,000 but `DefaultModels.large_context_model` is `None`, `ModelManager.get_default_model` returns `None`, causing `provision_langchain_model` to raise a `ConfigurationError`. Administrators must configure the large-context default in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) before processing very long documents.

### Can I force a specific model for a request regardless of token count?

Yes. By passing the `model_id` parameter to `provision_langchain_model`, you bypass the token threshold check entirely. The function will use `ModelManager.get_model` to load the specified model directly, skipping the 105,000 token evaluation logic.

### Where is the 105,000 token threshold defined?

The threshold is hard-coded at line 23 of [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) as the integer literal `105_000`. This value is not configurable at runtime and requires modifying the source code to change.

### How does Open Notebook count tokens?

The framework uses the `token_count` utility imported from [`open_notebook/utils/__init__.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/__init__.py). This helper analyzes the content string and returns the token count, which `provision_langchain_model` then compares against the 105,000 token limit to determine the appropriate model selection path.