Open Notebook Configuration System and Pydantic v2 Environment Variable Validation

Open Notebook does not use Pydantic v2 for environment variable validation; instead, it reads configuration directly from os.environ with fallback defaults in open_notebook/config.py.

When exploring the lfnovo/open-notebook repository for Pydantic v2 environment variable validation patterns, you'll discover that the project takes a deliberately minimal approach to configuration management. While the codebase extensively leverages Pydantic v2 for API request validation and domain entities, environment variables are handled through native Python os.environ calls with sensible defaults. This architecture keeps the configuration layer lightweight while reserving strict schema validation for application data.

How Configuration Works in Open Notebook

The configuration system resides entirely within open_notebook/config.py, a lightweight module that resolves paths and reads environment variables without external validation libraries. This file defines the root data directory and handles optional environment overrides for specific paths.

Directory Structure and Defaults

The module establishes three primary directories under a root ./data folder:

• SQLite checkpoints: ./data/sqlite-db/checkpoints.sqlite
• Uploads storage: ./data/uploads
• Tiktoken cache: Configurable via TIKTOKEN_CACHE_DIR environment variable

# open_notebook/config.py

import os

# ROOT DATA FOLDER

DATA_FOLDER = "./data"

# LANGGRAPH CHECKPOINT FILE

sqlite_folder = f"{DATA_FOLDER}/sqlite-db"
os.makedirs(sqlite_folder, exist_ok=True)
LANGGRAPH_CHECKPOINT_FILE = f"{sqlite_folder}/checkpoints.sqlite"

# UPLOADS FOLDER

UPLOADS_FOLDER = f"{DATA_FOLDER}/uploads"
os.makedirs(UPLOADS_FOLDER, exist_ok=True)

# TIKTOKEN CACHE FOLDER

TIKTOKEN_CACHE_DIR = os.environ.get("TIKTOKEN_CACHE_DIR", "").strip() \
    or f"{DATA_FOLDER}/tiktoken-cache"
os.makedirs(TIKTOKEN_CACHE_DIR, exist_ok=True)

The code uses os.environ.get("TIKTOKEN_CACHE_DIR", "") to read the optional environment variable, falling back to a default subdirectory when the variable is missing or empty. No type conversion, validation, or schema enforcement occurs during this process.

Pydantic v2 Usage in Open Notebook

While Pydantic v2 powers extensive validation throughout the application, it validates API payloads and domain entities—not environment configuration.

Where Pydantic v2 Is Actually Used

open_notebook/domain/base.py defines RecordModel, a Pydantic v2 BaseModel subclass that serves as the foundation for database entities. Similarly, open_notebook/domain/content_settings.py contains ContentSettings, a Pydantic v2 model that validates user-editable settings stored in the database. The api/models.py file contains numerous Pydantic v2 request and response models such as SettingsResponse and SettingsUpdate.

None of these models utilize BaseSettings or SettingsConfigDict from Pydantic-Settings to handle environment variables. The strict separation keeps configuration (file paths, directories) distinct from validated application data.

Why Native os.environ Is Preferred

The project avoids Pydantic v2 environment variable validation for configuration due to the simplicity of its requirements. The TIKTOKEN_CACHE_DIR variable requires only string detection and default substitution—functionality that os.environ.get provides without additional dependencies. This approach eliminates the need for complex validation schemas when the configuration surface remains small and the data types remain simple strings.

Working with the Configuration System

Accessing configuration values requires importing the module directly and reading its module-level constants.

Reading the Cache Directory

from open_notebook import config

def get_tiktoken_cache_path() -> str:
    """Return the absolute path to the tiktoken cache directory."""
    return config.TIKTOKEN_CACHE_DIR

Overriding via Environment Variables

To redirect the tiktoken cache outside the default ./data directory—particularly useful in Docker containers where /data is volume-mounted—set the variable before starting the application:

export TIKTOKEN_CACHE_DIR=/tmp/tiktoken-cache
uvicorn api.main:app --reload

The server picks up the new path on startup, creating the directory automatically via the os.makedirs call in config.py.

Verifying Directory Creation

The configuration module ensures directories exist at import time, making them immediately available to application code:

import os
from open_notebook import config

assert os.path.isdir(config.UPLOADS_FOLDER)
assert os.path.isdir(config.TIKTOKEN_CACHE_DIR)

Summary

• Open Notebook uses os.environ.get in open_notebook/config.py to read environment variables, not Pydantic v2.
• No validation schema is applied to environment variables; the code falls back to hardcoded defaults when variables are missing.
• Pydantic v2 validates domain entities in open_notebook/domain/base.py and API models in api/models.py, but never handles configuration.
• Directory creation happens automatically at module import via os.makedirs with exist_ok=True.
• TIKTOKEN_CACHE_DIR represents the only environment-configurable path, enabling Docker-specific cache location overrides.

Frequently Asked Questions

Does Open Notebook use Pydantic BaseSettings for configuration?

No, Open Notebook does not implement a BaseSettings subclass or use pydantic-settings for environment variable validation. The open_notebook/config.py module reads values directly using os.environ.get with string defaults, bypassing the type conversion and validation features found in Pydantic v2 settings models.

How do I change the tiktoken cache location in Open Notebook?

Set the TIKTOKEN_CACHE_DIR environment variable to your desired path before launching the application. The configuration module in open_notebook/config.py checks this variable at startup and defaults to ./data/tiktoken-cache only when the variable is unset or empty. This design supports Docker deployments where the cache must reside outside volume-mounted directories.

Where is Pydantic v2 used in the Open Notebook codebase?

Pydantic v2 validates data structures throughout the application, including RecordModel in open_notebook/domain/base.py, ContentSettings in open_notebook/domain/content_settings.py, and various request/response models in api/models.py. These classes inherit from pydantic.BaseModel and enforce strict typing on API payloads and database entities, but they do not handle environment configuration.

Can I add Pydantic v2 validation to Open Notebook's configuration?

Yes, you could introduce a Pydantic v2 BaseSettings subclass to replace the direct os.environ calls in open_notebook/config.py. However, the current implementation intentionally avoids this dependency for configuration due to the simplicity of the requirements—only the TIKTOKEN_CACHE_DIR variable requires external configuration, and the default string handling suffices for the project's current needs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}