How the 105,000 Token Threshold Triggers Large Context Model Upgrades in Open Notebook

Open Notebook automatically upgrades to a large-context language model when content exceeds 105,000 tokens, using the provision_langchain_model() function in open_notebook/ai/provision.py to switch from standard to high-capacity configurations.

Managing large documents in AI-powered notebooks requires intelligent context handling. The open-notebook repository implements an automatic model escalation system that monitors token counts and switches to high-capacity models when content exceeds the 105,000 token threshold. This mechanism ensures that extensive documents are processed by models capable of handling massive context windows without manual user intervention.

Token Counting with tiktoken

Before the threshold check occurs, Open Notebook quantifies content length using the token_count() function defined in open_notebook/utils/token_utils.py. This utility leverages the tiktoken library with the "o200k_base" encoder to calculate precise token counts for the text that will be sent to the LLM.

If tiktoken is unavailable, the system falls back to a word-count estimation method. The resulting token count determines whether the standard model can handle the request or if the system must escalate to a large-context alternative.

The 105,000 Token Threshold Check

The core escalation logic resides in provision_langchain_model() within open_notebook/ai/provision.py. This async function receives raw content and immediately evaluates its size against the hard-coded limit:

if tokens > 105_000:
    # select the large-context model

When the token count exceeds 105,000 tokens, the function logs the decision and initiates the large-context model selection process. This threshold represents the maximum capacity for standard models in the default configuration, ensuring that any overflow is handled by models with expanded context windows.

Large Context Model Selection

Upon crossing the 105,000 token threshold, the system queries the DefaultModels record defined in open_notebook/ai/models.py. Specifically, it retrieves the model ID stored in the large_context_model field using model_manager.get_default_model("large_context").

If a large-context model is properly configured (such as GPT-4-Turbo-1106k or Claude-200k variants), the system instantiates this alternative. However, if no model is configured in Settings → Models, the function raises a ConfigurationError, prompting the user to set a valid large-context model before proceeding.

Practical Implementation Example

The following code demonstrates how token counting and automatic model provisioning work together in practice:

from open_notebook.utils import token_count
from open_notebook.ai.provision import provision_langchain_model

# 1. Count tokens in a document

text = "Your long notebook content ..."
num_tokens = token_count(text)
print(f"Document contains {num_tokens:,} tokens")

# 2. Provision a model - large-context upgrade happens automatically

#    If num_tokens exceeds 105,000, large_context_model will be used.

model = await provision_langchain_model(
    content=text,
    model_id=None,            # No explicit model forced

    default_type="chat",     # Normal default type

)

print(f"Using model: {model.id}")

When num_tokens exceeds 105,000, the model.id returned reflects the large-context configuration rather than the standard chat model, transparently handling the capacity upgrade.

Summary

• Hard threshold: The system checks if tokens > 105_000 in open_notebook/ai/provision.py to determine when content exceeds standard model capacity.
• Automatic escalation: Cross-references the large_context_model field from DefaultModels in open_notebook/ai/models.py to load high-capacity alternatives.
• Configuration requirement: Requires explicit setup of a large-context model in Settings, or raises ConfigurationError.
• Token calculation: Uses tiktoken "o200k_base" encoder via token_count() in open_notebook/utils/token_utils.py with word-count fallback.

Frequently Asked Questions

What happens if no large context model is configured?

If the token count exceeds 105,000 but no model is defined in the large_context_model field, the system raises a ConfigurationError. This error prompts you to navigate to Settings → Models and configure a valid large-context model (such as GPT-4-Turbo or Claude-200k) before the operation can proceed.

How does Open Notebook count tokens when tiktoken is unavailable?

When the tiktoken library is not installed or accessible, the token_count() function in open_notebook/utils/token_utils.py falls back to a word-count estimation method. While less precise than the "o200k_base" encoder, this fallback ensures the 105,000 token threshold check can still function in constrained environments.

Can I manually override the 105,000 token threshold?

The 105,000 token limit is hard-coded in open_notebook/ai/provision.py within the provision_langchain_model() function. To modify this behavior, you would need to edit the source code where the if tokens > 105_000: check occurs, as the threshold is not exposed as a configurable runtime parameter in the current implementation.

Which models are typically used as large context alternatives?

According to the open-notebook source code, suitable replacements include GPT-4-Turbo-1106k and Claude-200k variants. These models offer expanded context windows capable of handling the content that exceeds the 105,000 token limit, and are stored in the large_context_model configuration field of the DefaultModels record.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}