How to Configure Open-Notebook: Environment Variables and Core Settings Guide

Configure Open-Notebook by creating a .env file from the provided .env.example template, setting your SurrealDB connection credentials and AI provider API keys, while optionally customizing filesystem paths in open_notebook/config.py before starting the FastAPI server.

Open-Notebook is a multi-tier, self-hosted AI research assistant that relies on environment variables and source-level constants for all runtime configuration. To successfully configure Open-Notebook for local development or production deployment, you must ensure the SurrealDB database is reachable, AI provider credentials are accessible via the KeyProvider class, and the Next.js frontend knows where to find the backend API.

Core Filesystem Configuration in open_notebook/config.py

The open_notebook/config.py module defines the filesystem layout and creates required folders at startup using os.makedirs(..., exist_ok=True).

Data Folders and Cache Paths

You can customize these constants before building the application:

• DATA_FOLDER – Root directory for all runtime data. Defaults to "./data" and stores uploaded files, checkpoints, and logs.
• LANGGRAPH_CHECKPOINT_FILE – SQLite database path used by LangGraph to persist workflow state.
• UPLOADS_FOLDER – Destination for uploaded PDFs, audio, video, and other source materials.
• TIKTOKEN_CACHE_DIR – Cache location for the tiktoken tokenizer; override this via the TIKTOKEN_CACHE_DIR environment variable when running inside read-only containers.

If you change DATA_FOLDER, ensure the process has write permissions to the new location.

Environment Variables Setup

Open-Notebook reads every configuration option via os.getenv. Copy the repository’s .env.example file to .env at the project root and populate the values specific to your deployment.

Mandatory Variables for Database and AI

These variables must be set for the application to start:

• SURREAL_URL – Full URL of the SurrealDB instance including port (e.g., http://localhost:8000). Consumed by open_notebook/database/repository.py.
• SURREAL_USER – Username for SurrealDB authentication (typically root).
• SURREAL_PASS – Password for SurrealDB authentication.
• OPENAI_API_KEY or ANTHROPIC_API_KEY – API key for your chosen AI provider. The KeyProvider class in open_notebook/ai/key_provider.py supports eight different providers and reads the specific key based on the provider name.
• MODEL_NAME – Default LLM identifier (e.g., gpt-4o, claude-3-sonnet-20240229) used by ModelManager in open_notebook/ai/models.py.
• BASE_API_URL – Base URL for the FastAPI server (e.g., http://localhost:5055). The frontend reads this via its own frontend/.env.local file.

Optional Advanced Settings

Fine-tune performance and security with these variables:

• TIKTOKEN_CACHE_DIR – Redirects the tokenizer cache location (essential for Docker deployments).
• LOG_LEVEL – Controls verbosity in api/main.py (DEBUG, INFO, WARNING, etc.).
• MAX_WORKERS – Thread count for background job queues (podcast generation, embedding rebuilds) defined in api/podcast_service.py.
• CORS_ORIGINS – Comma-separated list of allowed origins for development CORS in api/main.py.
• AUTH_PASSWORD – Simple password for the development authentication middleware in api/auth.py.

AI Provider and Model Configuration

Managing API Keys with KeyProvider

The KeyProvider class in open_notebook/ai/key_provider.py abstracts credential lookup. It first checks environment variables following the pattern {PROVIDER_UPPER}_API_KEY, then falls back to credentials stored in SurrealDB.

# From open_notebook/ai/key_provider.py

class KeyProvider:
    @staticmethod
    def get_key(provider: str) -> str | None:
        env_var = f"{provider.upper()}_API_KEY"
        return os.getenv(env_var)  # Reads from .env

You only need to set one provider-specific key; the rest are ignored.

Selecting Models via ModelManager

The ModelManager class in open_notebook/ai/models.py instantiates an Esperanto model using your MODEL_NAME variable. If a request exceeds a provider’s context limit, the manager automatically falls back to a long-context variant (e.g., gpt-4-turbo-preview).

Database Migration Settings

On API startup, the AsyncMigrationManager in open_notebook/database/async_migrate.py executes pending SurrealQL migrations found in the migrations/ folder. Control this behavior with:

• SKIP_MIGRATIONS – Set to 1 to skip migrations (useful for read-only containers).
• MIGRATION_LOG – Path to the migration output log (defaults to ./data/migration.log).

Frontend Connection Settings

The React/Next.js UI requires its own environment file at frontend/.env.local. Set the NEXT_PUBLIC_API_URL variable to match your FastAPI server address:

NEXT_PUBLIC_API_URL=http://localhost:5055

Update this value when running the API on a non-default port or behind a reverse proxy.

Deployment Configuration Examples

Minimal Local Development Configuration

Create a .env file at the project root:

# Database

SURREAL_URL=http://localhost:8000
SURREAL_USER=root
SURREAL_PASS=mysupersecret

# AI provider (choose one)

OPENAI_API_KEY=sk-****************************************

# Model

MODEL_NAME=gpt-4o

# Optional – change data locations

DATA_FOLDER=./my_data
TIKTOKEN_CACHE_DIR=./my_data/tiktoken-cache

Docker Configuration with Volume Mounts

When running in a container, mount a volume and set the cache directory:

docker run -e TIKTOKEN_CACHE_DIR=/cache/tiktoken \
           -e OPENAI_API_KEY=$OPENAI_API_KEY \
           -e SURREAL_URL=http://host.docker.internal:8000 \
           -v $(pwd)/data:/cache \
           lfnovo/open-notebook:latest

Programmatic Model Access

Access the configured model directly in Python:

from open_notebook.ai.models import ModelManager

model = ModelManager.get_model()  # Respects MODEL_NAME and provider keys

response = model.chat(messages=[{"role": "user", "content": "Explain LangGraph."}])
print(response.content)

Summary

• Copy .env.example to .env and fill in mandatory SurrealDB credentials (SURREAL_URL, SURREAL_USER, SURREAL_PASS) and at least one AI provider key (OPENAI_API_KEY or ANTHROPIC_API_KEY).
• Customize data paths in open_notebook/config.py or via the DATA_FOLDER and TIKTOKEN_CACHE_DIR environment variables.
• Set MODEL_NAME to define your default LLM, managed by ModelManager in open_notebook/ai/models.py.
• Configure the frontend separately in frontend/.env.local with NEXT_PUBLIC_API_URL pointing to your FastAPI instance.
• Control database migrations using SKIP_MIGRATIONS if you need to prevent automatic SurrealQL execution on startup.

Frequently Asked Questions

Where do I set my OpenAI API key in Open-Notebook?

Set your OpenAI API key (or any supported provider key) in the .env file at the project root using the variable OPENAI_API_KEY. The KeyProvider class in open_notebook/ai/key_provider.py reads this value at runtime using os.getenv("OPENAI_API_KEY").

Can I use Anthropic Claude instead of OpenAI?

Yes. Open-Notebook supports multiple providers through the Esperanto library. Simply set ANTHROPIC_API_KEY in your .env file and update MODEL_NAME to a Claude identifier (e.g., claude-3-sonnet-20240229). The KeyProvider automatically detects the correct credential based on the provider name.

How do I change the data storage location?

Modify the DATA_FOLDER constant in open_notebook/config.py or set the DATA_FOLDER environment variable before starting the server. The application creates the directory automatically using os.makedirs(..., exist_ok=True), so ensure the process has write permissions to the target path.

Do I need to run database migrations manually?

No. The AsyncMigrationManager in open_notebook/database/async_migrate.py runs pending SurrealQL migrations automatically when the FastAPI server starts. You only need to intervene if you set SKIP_MIGRATIONS=1 to disable this behavior for read-only containers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}