# How ContextBuilder Injects Relevant Source Content into AI Prompts in Open Notebook

> Learn how ContextBuilder injects relevant source content into AI prompts within Open Notebook using LangGraph state machines and LangChain for efficient token management.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-07-05

---

**Open Notebook injects source content into AI prompts through a LangGraph state machine where `ai_prompter.Prompter` assembles system prompts by combining notebook context with chat history, while `provision_langchain_model` ensures token limits are respected before model invocation.**

Open Notebook is a privacy-first research assistant that dynamically injects source material into AI conversations. The context building pipeline leverages **LangGraph** state management, **FastAPI** endpoints, and a sophisticated **Prompter** class to ensure relevant source content reaches the language model without exceeding token constraints.

## Context Storage in ThreadState

The foundation of context injection lies in the `ThreadState` class defined in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py). This Pydantic state object holds the complete conversation context before processing:

- **Message list**: The accumulated chat history
- **Notebook reference**: Links to source materials and research context
- **Context field**: Explicit source content to be injected into the prompt
- **Model override**: Optional specific model selection for this request

When a user initiates a chat, the system populates `ThreadState` with relevant source content from the notebook, making it available for the prompt construction phase.

## Building System Prompts with ai_prompter.Prompter

The actual injection mechanism resides in the `call_model_with_messages` function within [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py). This orchestrator uses `ai_prompter.Prompter` to construct the final prompt:

```python

# Conceptual flow based on the architecture

def call_model_with_messages(state: ThreadState):
    # Build system prompt with injected source context

    system_prompt = ai_prompter.Prompter.build_system_prompt(
        context=state.context,
        notebook_ref=state.notebook_id
    )
    # Concatenate with chat history

    messages = [system_prompt] + state.messages
    return messages

```

The **Prompter** class performs the critical task of formatting raw source content (PDFs, web pages, audio transcripts) into a structured system message that instructs the AI on how to use the provided context.

## Token-Aware Context Management

Before invoking the model, `provision_langchain_model` in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) validates that the injected context fits within model constraints:

1. **Token counting**: The `token_count(content)` function calculates the total tokens of the full prompt including injected source material
2. **Threshold detection**: If the count exceeds **105,000 tokens**, the system automatically switches to the *large-context* default model
3. **Model resolution**: If a specific `model_id` is forced in the state, that model is used regardless of token count
4. **Fallback handling**: If no suitable model exists or the retrieved model is not a `LanguageModel`, a `ConfigurationError` is raised with a UI-friendly message

This ensures that context injection never causes token overflow errors during inference.

## Model Provisioning and Credential Injection

The `ModelManager` class in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) supports context injection by providing the appropriate AI client:

- **`ModelManager.get_model`**: Retrieves the model by SurrealDB ID and builds an **Esperanto** wrapper via `AIFactory.create_language`
- **`provision_provider_keys`**: Called from [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py), this function sets environment variables (e.g., `OPENAI_API_KEY`) just-in-time before model invocation, ensuring the context-rich prompt can actually reach the AI provider
- **Cache-free operation**: Every call hits the database directly, ensuring that model changes in the UI reflect immediately in context injection workflows

## The Complete Injection Workflow

The end-to-end context injection follows this pipeline in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py):

1. **State initialization**: `ThreadState` captures user query and relevant source documents
2. **Prompt construction**: `ai_prompter.Prompter` builds the system prompt incorporating source content
3. **Token validation**: `provision_langchain_model` counts tokens and selects the appropriate model (standard or large-context)
4. **Credential provisioning**: `provision_provider_keys` prepares API credentials via [`open_notebook/ai/key_provider.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/key_provider.py)
5. **Async execution**: Because FastAPI handlers are synchronous, the workflow spins up a new asyncio loop (or thread pool) to call the async provisioning code safely
6. **Model invocation**: The Esperanto wrapper's `invoke` method processes the context-rich prompt and returns an `AIMessage`

## Summary

- **ThreadState** in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) serves as the container for source content and conversation history
- **ai_prompter.Prompter** builds system prompts by dynamically injecting notebook context into AI instructions
- **Token counting** in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) prevents context overflow by switching to large-context models when prompts exceed 105,000 tokens
- **ModelManager** and **key_provider.py** ensure the correct model and credentials are provisioned just-in-time for context-heavy requests
- The architecture maintains **cache-free** database operations, ensuring real-time updates to models and context sources

## Frequently Asked Questions

### What is ThreadState in Open Notebook?

**ThreadState** is a Pydantic model defined in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) that serves as the LangGraph state container. It holds the message list, notebook reference, source context, and optional model overrides. This state object travels through the LangGraph workflow, ensuring that relevant source content extracted from your research materials accompanies every AI interaction.

### How does Open Notebook handle token limits for large contexts?

The system uses `token_count(content)` in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) to calculate prompt size before sending requests to AI providers. If the injected source content pushes the total above **105,000 tokens**, the `provision_langchain_model` function automatically routes the request to the *large-context* default model. If no appropriate model is configured, it raises a `ConfigurationError` with guidance for the user.

### Where is the prompt building logic located?

The core prompt construction happens in [`open_notebook/graphs/chat.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/graphs/chat.py) within the `call_model_with_messages` function. This component uses `ai_prompter.Prompter` to build the system prompt by formatting source content and concatenating it with the chat history. The resulting message array is then passed to the provisioned language model for inference.

### How does the system choose which AI model processes the context?

Model selection occurs in [`open_notebook/ai/provision.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/provision.py) through the `provision_langchain_model` function. The logic checks token count first, then checks for a forced `model_id` in the request state, and finally falls back to the default model for the requested type (`chat`, `embedding`, etc.). The `ModelManager` class in [`open_notebook/ai/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/models.py) retrieves the actual model configuration from SurrealDB and wraps it with the Esperanto client for unified API access.