# How ModelManager Selects Models Based on Context Size: The 105,000 Token Threshold Explained

> Learn how the ModelManager selects models by context size with its 105,000 token threshold. Discover how Open Notebook routes requests based on token counts for optimal performance.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: deep-dive
- Published: 2026-06-28

---

**The ModelManager in Open Notebook automatically selects a large-context model when content exceeds 105,000 tokens, using the `provision_langchain_model` function in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) to check token counts and route requests accordingly.**

The Open Notebook project (lfnovo/open-notebook) handles large document processing by dynamically switching between standard and large-context language models based on content length. Understanding how the **ModelManager** selects models based on context size is essential for optimizing AI operations when working with extensive texts. This mechanism relies on a specific 105,000 token threshold defined in the provisioning layer to determine when to escalate to models with larger context windows.

## The Two-Step Selection Process

The Open Notebook backend decides which language model to use through a coordinated two-step process involving token counting and conditional model provisioning.

### Step 1: Token Counting with tiktoken

First, the system calculates the token count of incoming content using the `open_notebook.utils.token_count` helper. This utility, located in [`open_notebook/utils/token_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/token_utils.py), employs the `tiktoken` library with the "o200k_base" encoder to return an integer representing the total tokens in the supplied `content`.

### Step 2: The 105,000 Token Threshold

Next, the `provision_langchain_model` function in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) evaluates the token count against the threshold:

```python
tokens = token_count(content)

if tokens > 105_000:                     # <-- 105k-token threshold

    # use the "large-context" model configured in the defaults

    model = await model_manager.get_default_model("large_context", **kwargs)
elif model_id:
    # an explicit model was requested in the UI/config

    model = await model_manager.get_model(model_id, **kwargs)
else:
    # fall back to the default for the requested type (chat, embedding, …)

    model = await model_manager.get_default_model(default_type, **kwargs)

```

When the token count exceeds **105,000**, the function logs the decision (`large_context (content has {tokens} tokens)`) and requests the **large-context** default model from the ModelManager. If the count falls below this threshold, the function either respects an explicit `model_id` or returns the default model for the requested `default_type`.

## Inside the ModelManager Facade

The **ModelManager** class, defined in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py), serves as a thin façade over the model retrieval system. According to the source code, it retrieves model records from SurrealDB, resolves linked credentials, and constructs the appropriate Esperanto factory call using methods like `AIFactory.create_language` or `create_embedding`.

Importantly, the large-context decision logic resides entirely within the `provision_langchain_model` function, not inside the ModelManager itself. The ModelManager simply executes the retrieval when asked for the `large_context` default via `get_default_model("large_context", **kwargs)`.

## Implementation Examples

### Automatic Large-Context Selection

When processing documents exceeding the threshold, the system automatically selects the appropriate model:

```python

# Example: provisioning a chat model for a 120k-token document

from open_notebook.ai.provision import provision_langchain_model

content = "…" * 150_000  # a very long string

model = await provision_langchain_model(
    content=content,
    model_id=None,            # no explicit model chosen by the user

    default_type="chat",      # we want a chat-type model

)

# → the function returns the large-context language model

```

### Bypassing the Threshold with Explicit Model IDs

You can force a specific model regardless of content size by providing a `model_id`:

```python

# Example: forcing a specific model, bypassing the token-threshold logic

model = await provision_langchain_model(
    content="short text",
    model_id="open_notebook:model:anthropic Claude-3.5-sonnet",  # explicit ID

    default_type="chat",
)

# → the explicit model is used even if the content were > 105k tokens

```

### Direct ModelManager Usage

For advanced use cases, interact with the ModelManager directly:

```python

# Direct use of ModelManager (rarely needed outside provisioning)

from open_notebook.ai.models import ModelManager

mgr = ModelManager()
large_ctx = await mgr.get_default_model("large_context")

# large_ctx is the LanguageModel instance configured as the large-context default

```

## Summary

- **Token counting** occurs in [`open_notebook/utils/token_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/token_utils.py) using the `tiktoken` "o200k_base" encoder.
- The **105,000 token threshold** is hardcoded in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) within the `provision_langchain_model` function.
- When content exceeds **105,000 tokens**, the system automatically requests the `large_context` default model from the ModelManager.
- **Explicit model IDs** bypass the threshold logic entirely, allowing forced model selection regardless of content size.
- The **ModelManager** in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) handles model instantiation but does not contain the threshold logic—it receives instructions from the provisioning layer.

## Frequently Asked Questions

### What happens if my content is exactly 105,000 tokens?

Content with exactly 105,000 tokens uses the standard default model, not the large-context variant. The threshold check uses a strict greater-than comparison (`tokens > 105_000`), so only content exceeding 105,000 tokens triggers the large-context selection.

### Can I force a specific model even if my content exceeds 105,000 tokens?

Yes. Providing an explicit `model_id` parameter to `provision_langchain_model` bypasses the token threshold logic entirely. The function checks for explicit IDs before evaluating the token count, ensuring your specified model handles the content regardless of size.

### Where is the 105,000 token threshold configured?

The threshold is hardcoded as the integer literal `105_000` in the `provision_langchain_model` function within [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py). It is not defined as a configuration variable or environment setting in the current implementation.

### Which tokenizer does Open Notebook use for counting tokens?

Open Notebook uses the `tiktoken` library with the "o200k_base" encoder, implemented in [`open_notebook/utils/token_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/token_utils.py). This provides accurate token counts compatible with OpenAI's model standards, ensuring reliable threshold calculations across different content types.