How ContextBuilder Injects Relevant Source Content into AI Prompts in Open Notebook

Open Notebook injects source content into AI prompts through a LangGraph state machine where ai_prompter.Prompter assembles system prompts by combining notebook context with chat history, while provision_langchain_model ensures token limits are respected before model invocation.

Open Notebook is a privacy-first research assistant that dynamically injects source material into AI conversations. The context building pipeline leverages LangGraph state management, FastAPI endpoints, and a sophisticated Prompter class to ensure relevant source content reaches the language model without exceeding token constraints.

Context Storage in ThreadState

The foundation of context injection lies in the ThreadState class defined in open_notebook/graphs/chat.py. This Pydantic state object holds the complete conversation context before processing:

• Message list: The accumulated chat history
• Notebook reference: Links to source materials and research context
• Context field: Explicit source content to be injected into the prompt
• Model override: Optional specific model selection for this request

When a user initiates a chat, the system populates ThreadState with relevant source content from the notebook, making it available for the prompt construction phase.

Building System Prompts with ai_prompter.Prompter

The actual injection mechanism resides in the call_model_with_messages function within open_notebook/graphs/chat.py. This orchestrator uses ai_prompter.Prompter to construct the final prompt:

# Conceptual flow based on the architecture

def call_model_with_messages(state: ThreadState):
    # Build system prompt with injected source context

    system_prompt = ai_prompter.Prompter.build_system_prompt(
        context=state.context,
        notebook_ref=state.notebook_id
    )
    # Concatenate with chat history

    messages = [system_prompt] + state.messages
    return messages

The Prompter class performs the critical task of formatting raw source content (PDFs, web pages, audio transcripts) into a structured system message that instructs the AI on how to use the provided context.

Token-Aware Context Management

Before invoking the model, provision_langchain_model in open_notebook/ai/provision.py validates that the injected context fits within model constraints:

1. Token counting: The token_count(content) function calculates the total tokens of the full prompt including injected source material
2. Threshold detection: If the count exceeds 105,000 tokens, the system automatically switches to the large-context default model
3. Model resolution: If a specific model_id is forced in the state, that model is used regardless of token count
4. Fallback handling: If no suitable model exists or the retrieved model is not a LanguageModel, a ConfigurationError is raised with a UI-friendly message

This ensures that context injection never causes token overflow errors during inference.

Model Provisioning and Credential Injection

The ModelManager class in open_notebook/ai/models.py supports context injection by providing the appropriate AI client:

• ModelManager.get_model: Retrieves the model by SurrealDB ID and builds an Esperanto wrapper via AIFactory.create_language
• provision_provider_keys: Called from open_notebook/ai/key_provider.py, this function sets environment variables (e.g., OPENAI_API_KEY) just-in-time before model invocation, ensuring the context-rich prompt can actually reach the AI provider
• Cache-free operation: Every call hits the database directly, ensuring that model changes in the UI reflect immediately in context injection workflows

The Complete Injection Workflow

The end-to-end context injection follows this pipeline in open_notebook/graphs/chat.py:

1. State initialization: ThreadState captures user query and relevant source documents
2. Prompt construction: ai_prompter.Prompter builds the system prompt incorporating source content
3. Token validation: provision_langchain_model counts tokens and selects the appropriate model (standard or large-context)
4. Credential provisioning: provision_provider_keys prepares API credentials via open_notebook/ai/key_provider.py
5. Async execution: Because FastAPI handlers are synchronous, the workflow spins up a new asyncio loop (or thread pool) to call the async provisioning code safely
6. Model invocation: The Esperanto wrapper's invoke method processes the context-rich prompt and returns an AIMessage

Summary

• ThreadState in open_notebook/graphs/chat.py serves as the container for source content and conversation history
• ai_prompter.Prompter builds system prompts by dynamically injecting notebook context into AI instructions
• Token counting in open_notebook/ai/provision.py prevents context overflow by switching to large-context models when prompts exceed 105,000 tokens
• ModelManager and key_provider.py ensure the correct model and credentials are provisioned just-in-time for context-heavy requests
• The architecture maintains cache-free database operations, ensuring real-time updates to models and context sources

Frequently Asked Questions

What is ThreadState in Open Notebook?

ThreadState is a Pydantic model defined in open_notebook/graphs/chat.py that serves as the LangGraph state container. It holds the message list, notebook reference, source context, and optional model overrides. This state object travels through the LangGraph workflow, ensuring that relevant source content extracted from your research materials accompanies every AI interaction.

How does Open Notebook handle token limits for large contexts?

The system uses token_count(content) in open_notebook/ai/provision.py to calculate prompt size before sending requests to AI providers. If the injected source content pushes the total above 105,000 tokens, the provision_langchain_model function automatically routes the request to the large-context default model. If no appropriate model is configured, it raises a ConfigurationError with guidance for the user.

Where is the prompt building logic located?

The core prompt construction happens in open_notebook/graphs/chat.py within the call_model_with_messages function. This component uses ai_prompter.Prompter to build the system prompt by formatting source content and concatenating it with the chat history. The resulting message array is then passed to the provisioned language model for inference.

How does the system choose which AI model processes the context?

Model selection occurs in open_notebook/ai/provision.py through the provision_langchain_model function. The logic checks token count first, then checks for a forced model_id in the request state, and finally falls back to the default model for the requested type (chat, embedding, etc.). The ModelManager class in open_notebook/ai/models.py retrieves the actual model configuration from SurrealDB and wraps it with the Esperanto client for unified API access.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}