How Configuration is Handled in Open Notebook: Environment Variables, Secrets, and Runtime Settings

Open Notebook centralizes configuration through environment variables loaded via python-dotenv, validates required secrets like OPEN_NOTEBOOK_ENCRYPTION_KEY at startup, and supports Docker secrets while exposing runtime status through a dedicated /api/config endpoint.

The lfnovo/open-notebook repository implements a flexible, environment-driven configuration architecture that separates secrets from code. By leveraging Python's os.getenv with fallback mechanisms and Docker secrets support, the application ensures that sensitive credentials remain encrypted while allowing deployment-specific tuning of database connections, CORS policies, and AI provider timeouts.

Loading Environment Variables at Startup

Configuration initialization begins in api/main.py where the application invokes load_dotenv() from the python-dotenv library. This call reads a .env file from the project root if present, making all variables available to subsequent os.getenv calls throughout the codebase.

# api/main.py

from dotenv import load_dotenv
load_dotenv()

After loading, the FastAPI application validates critical security parameters immediately during the lifespan startup. The OPEN_NOTEBOOK_ENCRYPTION_KEY is checked first, as it protects API credentials stored in SurrealDB.

# api/main.py (lifespan startup)

if not get_secret_from_env("OPEN_NOTEBOOK_ENCRYPTION_KEY"):
    logger.warning(
        "OPEN_NOTEBOOK_ENCRYPTION_KEY not set. "
        "API key encryption will fail until this is configured."
    )

Encryption and Secret Management

All sensitive values use Fernet symmetric encryption implemented in open_notebook/utils/encryption.py. The get_secret_from_env function supports both standard environment variables and Docker secrets via the _FILE suffix convention.

# open_notebook/utils/encryption.py

def get_secret_from_env(var_name: str) -> Optional[str]:
    # Checks VAR_FILE first, then VAR

    file_path = os.environ.get(f"{var_name}_FILE")
    if file_path and os.path.exists(file_path):
        with open(file_path, 'r') as f:
            return f.read().strip()
    return os.environ.get(var_name)

The encryption key undergoes transformation through _ensure_fernet_key before creating the Fernet instance used by encrypt_value and decrypt_value. This ensures that API keys for LLM providers remain encrypted at rest in SurrealDB while remaining decryptable during runtime.

Core Configuration Variables

Open Notebook recognizes several environment variable categories:

API and Security Settings

• OPEN_NOTEBOOK_ENCRYPTION_KEY: Required secret for credential encryption (no default)
• OPEN_NOTEBOOK_PASSWORD: Optional UI-level password protection
• API_URL: External-facing API endpoint (auto-detected if unset)
• INTERNAL_API_URL: Server-side proxy address (defaults to http://localhost:5055)
• CORS_ORIGINS: Comma-separated allowed origins for cross-origin requests (defaults to *)

Database Configuration

• SURREAL_URL: WebSocket endpoint for SurrealDB (e.g., ws://localhost:8000/rpc)
• SURREAL_USER, SURREAL_PASSWORD, SURREAL_NAMESPACE, SURREAL_DATABASE: Required connection parameters

AI Provider Tuning

• ESPERANTO_LLM_TIMEOUT: LLM inference timeout in seconds (default: 60)
• OPEN_NOTEBOOK_EMBEDDING_BATCH_SIZE: Texts processed per embedding batch (default: 50)
• TTS_BATCH_SIZE: Parallel text-to-speech requests (default: 5)

The complete authoritative list resides in docs/5-CONFIGURATION/environment-reference.md.

Runtime Configuration Endpoint

The application exposes current system status through the /api/config endpoint defined in api/routers/config.py. This router aggregates version information, checks GitHub for updates, and verifies database connectivity.

# api/routers/config.py

@router.get("/config")
async def get_config(request: Request):
    current_version = get_version()
    latest_version, has_update = await get_latest_version_cached(current_version)
    db_health = await check_database_health()
    
    return {
        "version": current_version,
        "latestVersion": latest_version,
        "hasUpdate": has_update,
        "dbStatus": db_health["status"],
    }

The database health check executes a lightweight SurrealDB query with a strict two-second timeout:

# api/routers/config.py

result = await asyncio.wait_for(repo_query("RETURN 1"), timeout=2.0)

File System and Cache Configuration

Static data paths for LangGraph checkpoints, file uploads, and tiktoken caches are defined in open_notebook/config.py. This module creates directories automatically at import time using os.makedirs.

# open_notebook/config.py

DATA_FOLDER = "./data"
sqlite_folder = f"{DATA_FOLDER}/sqlite-db"
os.makedirs(sqlite_folder, exist_ok=True)

TIKTOKEN_CACHE_DIR = os.environ.get("TIKTOKEN_CACHE_DIR", "").strip() \
    or f"{DATA_FOLDER}/tiktoken-cache"
os.makedirs(TIKTOKEN_CACHE_DIR, exist_ok=True)

Override the tiktoken cache location by setting the environment variable before import:

export TIKTOKEN_CACHE_DIR=/opt/tiktoken-cache

Practical Configuration Examples

Example 1: Complete .env Configuration for Docker Deployment

# .env

OPEN_NOTEBOOK_ENCRYPTION_KEY=my-super-secret-key
SURREAL_URL=ws://surrealdb:8000/rpc
SURREAL_USER=root
SURREAL_PASSWORD=strong-password
SURREAL_NAMESPACE=open_notebook
SURREAL_DATABASE=default
CORS_ORIGINS=https://notebook.example.com
API_URL=https://notebook.example.com
ESPERANTO_LLM_TIMEOUT=120

Example 2: Docker Secrets Support

Instead of exposing the encryption key directly, use Docker secrets:

# docker-compose.yml

secrets:
  encryption_key:
    file: ./secrets/encryption_key.txt
services:
  api:
    secrets:
      - encryption_key
    environment:
      - OPEN_NOTEBOOK_ENCRYPTION_KEY_FILE=/run/secrets/encryption_key

Example 3: Encrypting Credentials Programmatically

from open_notebook.utils.encryption import encrypt_value, decrypt_value

plain_key = "sk-my-openai-key"
encrypted = encrypt_value(plain_key)  # Stores in SurrealDB

decrypted = decrypt_value(encrypted)  # Returns original key

Example 4: Frontend Configuration Retrieval

// Next.js frontend
const resp = await fetch(`${process.env.NEXT_PUBLIC_API_URL}/api/config`);
const cfg = await resp.json();

if (cfg.hasUpdate) {
  console.log(`Version ${cfg.latestVersion} available`);
}

Summary

• Environment variables drive all configuration through load_dotenv() in api/main.py, supporting both .env files and container runtime injection.
• Encryption requires OPEN_NOTEBOOK_ENCRYPTION_KEY at startup, utilizing Fernet symmetric encryption via open_notebook/utils/encryption.py with Docker secrets support.
• Database connections rely on SURREAL_* variables for WebSocket endpoints and authentication.
• Runtime introspection is available through the /api/config endpoint in api/routers/config.py, providing version checks and database health status.
• Data folders for checkpoints and caches are auto-created at import based on DATA_FOLDER in open_notebook/config.py.

Frequently Asked Questions

How does Open Notebook handle sensitive configuration like API keys?

Open Notebook encrypts all provider API keys using Fernet symmetric encryption before storing them in SurrealDB. The encryption key must be provided via the OPEN_NOTEBOOK_ENCRYPTION_KEY environment variable, which the system reads through get_secret_from_env in open_notebook/utils/encryption.py. This function also supports Docker secrets by checking for a _FILE suffix, allowing you to mount secrets as files rather than exposing them in environment variables.

What happens if the encryption key is not configured?

If OPEN_NOTEBOOK_ENCRYPTION_KEY is missing during startup, the application logs a warning in api/main.py but continues to run. However, any operation attempting to encrypt or decrypt credentials will fail until the key is configured, as the Fernet instance cannot be initialized without it.

Can I use a .env file instead of setting environment variables directly?

Yes. The application explicitly calls load_dotenv() at the beginning of api/main.py, which reads variables from a .env file in the project root. Variables defined in this file are available to all subsequent os.getenv calls, making it ideal for local development while keeping sensitive values out of version control.

Where can I find the complete list of supported environment variables?

The authoritative reference document is located at docs/5-CONFIGURATION/environment-reference.md in the repository. This markdown file contains descriptions, default values, and requirements for all variables including embedding batch sizes, TTS configurations, proxy settings, and LangChain tracing options.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}