# How Open Notebook Automatically Switches to Large Context AI Models

> Learn how Open Notebook automatically switches to large context AI models when token counts exceed 105,000. Discover efficient AI model management for your projects.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-27

---

**Open Notebook automatically routes content exceeding 105,000 tokens to a dedicated large-context model by comparing token counts in the provisioning layer and requesting the `large_context` default type from the model manager.**

Open Notebook is an open-source note-taking application that intelligently handles extensive documents by automatically switching to large context AI models when inputs exceed specific thresholds. This mechanism ensures that processing of very long texts remains seamless without requiring manual model selection by the user. The implementation spans token utilities, model provisioning logic, and default model configuration across the `lfnovo/open-notebook` repository.

## Token Counting and Threshold Detection

The automatic switching mechanism begins with **token counting** in [`open_notebook/utils/token_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/token_utils.py). The `token_count()` function processes raw content using the `o200k_base` tiktoken encoding to calculate the exact number of tokens. If tiktoken is unavailable, it falls back to a word-count estimate.

This token count is then compared against a **hard-coded threshold of 105,000 tokens**. When the input exceeds this limit, the system recognizes that standard context windows may be insufficient and triggers the large-context model selection pathway.

## The Model Provision Logic

The core switching logic resides in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) inside the `provision_langchain_model()` function. This async function evaluates the token count before initializing any model:

- **If `tokens > 105_000`**: The function logs the decision and calls `model_manager.get_default_model("large_context")` to retrieve the appropriate high-capacity model.
- **Otherwise**: The function respects an explicitly requested `model_id` or falls back to the default model for the requested type (chat, embedding, etc.).

This branching ensures that large inputs automatically receive a model capable of processing them, while normal inputs use standard, cost-effective models.

## Default Model Resolution

When a large-context model is requested, `ModelManager.get_default_model()` in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) queries the `DefaultModels` record stored in the database. Specifically, it reads the `large_context_model` field, which users populate through the Settings → Models UI (labeled as **"Large Context Model"**).

If a valid model ID exists in this field, the system fetches the corresponding `Model` record and returns a fully-initialized `LanguageModel` instance via Esperanto. The model is then converted to a LangChain-compatible object using `model.to_langchain()` and handed to the downstream LangGraph workflow for processing.

## Practical Implementation Examples

### Automatic Model Selection in LangGraph Nodes

When building custom nodes, you can rely on the automatic switching by calling `provision_langchain_model()`:

```python
from open_notebook.ai.provision import provision_langchain_model

async def chat_node(content: str, model_id: str | None = None):
    # Automatically picks a large-context model if content exceeds 105k tokens

    llm = await provision_langchain_model(
        content=content,
        model_id=model_id,
        default_type="chat",
        temperature=0.7,
    )
    response = await llm.ainvoke({"messages": [{"role": "user", "content": content}]})
    return response

```

### Manual Token Count Verification

To check whether content will trigger the large-context switch:

```python
from open_notebook.utils.token_utils import token_count

text = "..."  # Your large document

count = token_count(text)
print(f"Token count: {count}")

# If count > 105000, the provision logic will switch to the large-context model

```

### Configuring the Large Context Model via API

You can set the default large-context model programmatically:

```bash
curl -X PATCH http://localhost:5055/api/models/defaults \
  -H "Content-Type: application/json" \
  -d '{"large_context_model": "gemini-1.5-flash"}'

```

## Summary

- **Token counting** uses `o200k_base` encoding in [`open_notebook/utils/token_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/token_utils.py) to calculate input size.
- **Threshold detection** occurs at **105,000 tokens** within `provision_langchain_model()` in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py).
- **Automatic switching** requests the `large_context` model type from `ModelManager.get_default_model()` when thresholds are exceeded.
- **Configuration** happens via the `large_context_model` field in the `DefaultModels` database record, set through the Settings UI.
- **Transparency** ensures users never manually select models; the system handles extensive documents seamlessly.

## Frequently Asked Questions

### What is the exact token threshold for switching to a large context model?

Open Notebook uses a hard-coded threshold of **105,000 tokens**. Any input exceeding this count automatically triggers the large-context model selection in `provision_langchain_model()`.

### How does Open Notebook count tokens when processing content?

The system uses the `token_count()` function from [`open_notebook/utils/token_utils.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/utils/token_utils.py), which utilizes the `o200k_base` tiktoken encoding for accurate OpenAI-compatible token counting. If tiktoken is unavailable, it falls back to a word-count estimation method.

### Can users configure which model is used for large context processing?

Yes. Users configure the large-context model through the Settings → Models UI by setting the **"Large Context Model"** field, which corresponds to the `large_context_model` column in the `DefaultModels` database record. This can also be updated via the REST API at `/api/models/defaults`.

### Is the large context model switch transparent to the end user?

Yes. The switch is completely automatic and transparent. Users only need to ensure a suitable large-context model (such as Gemini 1.5 Flash or Claude 3 Opus) is configured in the Settings page. The provisioning layer handles all detection and model initialization without requiring manual intervention.