# How the Open Notebook API Handles the Large Context Model Selection Threshold of 105,000 Tokens

> The Open Notebook API automatically selects large context models for inputs over 105000 tokens, preventing overflow and data loss. Learn how this threshold prevents context window issues.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: deep-dive
- Published: 2026-06-12

---

**The Open Notebook API automatically provisions a large-context language model when input content exceeds 105,000 tokens, using a hard-coded threshold in the `provision_langchain_model` function to prevent context window overflow and data truncation.**

The Open Notebook repository (`lfnovo/open-notebook`) implements intelligent model routing that examines payload size before selecting a language model. When processing extensive documents or lengthy conversations, the API evaluates token counts against a specific 105,000 token boundary to determine whether to invoke high-capacity models capable of handling massive contexts without truncation.

## Token Counting with [`open_notebook/utils/token_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/token_utils.py)

Before the API can apply the 105,000 token threshold, it must accurately calculate the input size. The `token_count` function in [`open_notebook/utils/token_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/token_utils.py) handles this calculation using the **o200k_base** tokenizer from the `tiktoken` library.

This function encodes the input text using OpenAI's o200k_base encoding scheme, which provides precise token counts compatible with modern large language models. If `tiktoken` is unavailable, the utility falls back to a word-estimate heuristic, though the primary path relies on exact tokenization.

## The 105,000 Token Threshold Decision Logic

The core provisioning logic resides in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) within the `provision_langchain_model` function. This function implements a conditional check that compares the calculated token count against the hard-coded limit of **105,000 tokens**.

### When Content Exceeds 105,000 Tokens

When `tokens > 105_000`, the function logs a debug message at line 26 and immediately routes the request to a large-context capable model. Specifically, it calls `model_manager.get_default_model("large_context")` to retrieve a model configured for extensive context windows—typically variants like Claude-3.5-Sonnet or GPT-4-Turbo-128k that support 128k+ token contexts.

This automatic failover ensures that lengthy inputs, such as full-length PDF conversions or extensive codebases, remain intact without the truncation that would occur with standard context models.

### Normal Selection Path for Smaller Inputs

If the token count falls at or below 105,000, the function follows the standard provisioning path. It either uses an explicitly provided `model_id` or falls back to the `default_type` parameter (typically `"chat"`). This optimization prevents unnecessary resource allocation for routine queries while reserving high-capacity models for genuinely extensive content.

## Configuration Error Handling

If the API detects content exceeding 105,000 tokens but no `"large_context"` model is configured in the system, the provisioner raises a `ConfigurationError`. This exception includes a descriptive message directing users to the **Settings → Models** configuration page to set up a compatible large-context provider.

This safeguard prevents runtime failures by ensuring that the automatic threshold logic always has a valid target model available when triggered.

## Practical Implementation Examples

### Triggering Large-Context Model Selection

To observe the threshold behavior in action, provide content that exceeds 105,000 tokens without specifying an explicit model ID:

```python
from open_notebook.ai.provision import provision_langchain_model

# Simulate extensive content (e.g., a full PDF converted to text)

long_text = "word " * 200_000  # Generates >105,000 tokens

# Provision without explicit model_id to trigger threshold logic

model = await provision_langchain_model(
    content=long_text,
    model_id=None,          # Forces threshold evaluation

    default_type="chat",    # Fallback type when under threshold

)

print("Provisioned model:", model)  # Returns large-context model

```

### Verifying Token Counts Directly

You can inspect the token calculation independently using the utility function:

```python
from open_notebook.utils.token_utils import token_count

content = "Your extensive document text here..."
count = token_count(content)

print(f"Token count: {count}")
if count > 105_000:
    print("Content exceeds large-context threshold")

```

## Summary

- The Open Notebook API uses a **hard-coded threshold of 105,000 tokens** in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) to determine model selection.
- Token counting relies on the **o200k_base** tokenizer via [`open_notebook/utils/token_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/token_utils.py) for accurate measurement.
- Content exceeding the threshold automatically triggers `model_manager.get_default_model("large_context")` to prevent truncation.
- Missing large-context configurations raise a `ConfigurationError` with instructions to configure Settings → Models.
- The 105,000 token limit aligns with high-capacity models like Claude-3.5-Sonnet and GPT-4-Turbo-128k.

## Frequently Asked Questions

### What happens if my content is exactly 105,000 tokens?

Content exactly at 105,000 tokens follows the **normal selection path** rather than triggering the large-context model. The threshold check uses a strict greater-than comparison (`tokens > 105_000`), so only inputs exceeding this boundary activate the large-context provisioning logic.

### Why is the threshold set at 105,000 instead of a round number like 100,000?

The 105,000 token threshold provides a **safety margin** below the 128k context windows supported by models like GPT-4-Turbo and Claude-3.5-Sonnet. This buffer accommodates system prompts, message formatting overhead, and response generation while ensuring the full input remains within the model's actual capacity.

### Can I configure the 105,000 token threshold to a different value?

Currently, the 105,000 limit is **hard-coded** in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) at line 26. There is no runtime configuration option to adjust this threshold; modifying it requires changing the source code constant and redeploying the application.

### Which models are typically used for the "large_context" type?

The specific model returned by `model_manager.get_default_model("large_context")` depends on your [`open_notebook/config.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/config.py) settings. Common configurations include **Claude-3.5-Sonnet**, **GPT-4-Turbo-128k**, or other variants supporting 128,000+ token contexts, ensuring sufficient headroom beyond the 105,000 token trigger point.