Open Notebook Build Scripts and Processes: Complete Developer Guide

Open Notebook provides a comprehensive build system using Make, Docker, and Python scripts to compile the FastAPI backend, bundle the Next.js frontend, and orchestrate multi-service development environments.

The lfnovo/open-notebook repository includes a complete build pipeline that handles everything from local development to multi-architecture container distribution. This guide covers the available build scripts, their specific functions, and how to use them effectively.

Understanding the Build System Architecture

The build system centers on a Makefile located in the repository root, which orchestrates Docker builds, service lifecycle management, and utility tasks. The architecture supports two primary deployment modes: a multi-container setup (SurrealDB + API + Worker + Frontend) and a single-container image that bundles the API and compiled Next.js UI.

Key files in the build system include:

• Makefile – Central build orchestration defining all targets
• Dockerfile – Multi-stage build for the full application stack
• Dockerfile.single – Lightweight single-container configuration
• run_api.py – FastAPI entry point for standalone API execution
• docker-compose.yml – Pre-wired composition for quick local deployment
• scripts/export_docs.py – Documentation generation utility

Docker Build Processes

Local Development Builds

For local testing and development, the docker-build-local target compiles the Python backend, installs Node dependencies, and produces a production-ready container tagged with the current version from pyproject.toml.

# Build the image locally for your current platform

make docker-build-local

After building, you can run the container directly:

docker run -p 5055:5055 -p 8502:8502 lfnovo/open_notebook:$(git describe --tags --abbrev=0)

This exposes the API on port 5055 and the UI on port 8502.

Multi-Platform Production Builds

For distribution, the docker-push target uses Docker Buildx to create both AMD64 and ARM64 images simultaneously, pushing them to Docker Hub and GitHub Container Registry (GHCR).

# Build and push multi-arch images to registries

make docker-push

This command generates:

• lfnovo/open_notebook:<version> → Docker Hub
• ghcr.io/lfnovo/open-notebook:<version> → GitHub Container Registry

To update the latest tag after a release, use:

make docker-push-latest

For a complete release workflow that builds, versions, and tags:

make docker-release

Single-Container Image Builds

The build system supports a "single-container" variant defined in Dockerfile.single, which bundles the API and compiled Next.js UI into a minimal image. This is triggered automatically during the docker-push and docker-push-latest targets via docker-buildx steps.

Development Workflow Automation

Starting the Full Stack

The start-all target initializes the complete development environment, launching SurrealDB, the FastAPI server, a background worker, and the Next.js development server simultaneously.

make start-all

This makes the UI available at http://localhost:3000 and the API at http://localhost:5055.

Running Individual Services

For backend debugging, the api target launches the FastAPI server directly without containerization:

make api

Alternatively, invoke the Python script directly:

python run_api.py

The background worker (which handles SurrealDB commands and embeddings) can be managed independently:

make worker-start    # Start the background worker

make worker-stop     # Stop the background worker

make worker-restart  # Restart the background worker

Utility and Maintenance Scripts

Documentation Export

The export-docs target renders Markdown documentation into a bundled format for the project website:

make export-docs

This executes scripts/export_docs.py and generates static files under docs/_site.

Cache Cleanup

To remove Python and tool caches (__pycache__, .ruff_cache, etc.):

make clean-cache

Docker Compose Deployment

For immediate local deployment without manually building images, the repository provides a docker-compose.yml that wires SurrealDB and Open Notebook together:

# Download and run the prepared composition

curl -O https://raw.githubusercontent.com/lfnovo/open-notebook/main/docker-compose.yml
docker compose up -d

This exposes the UI at http://localhost:8502 and API documentation at http://localhost:5055/docs.

Summary

• The Makefile serves as the central build orchestrator, providing targets for Docker builds, service management, and utilities.
• Multi-platform builds support both AMD64 and ARM64 architectures, publishing to Docker Hub and GHCR via make docker-push.
• Development workflows use make start-all for full-stack startup or make api for isolated backend debugging.
• Single-container images are built using Dockerfile.single for lightweight deployments.
• Background workers support independent lifecycle management through worker-start, worker-stop, and worker-restart targets.
• Docker Compose offers a zero-build option for quick local testing with pre-configured SurrealDB integration.

Frequently Asked Questions

How do I build Open Notebook for only my local machine architecture?

Run make docker-build-local from the repository root. This target compiles the Python backend and Node frontend into a single image tagged with your current Git version, optimized for your local platform only.

What is the difference between docker-push and docker-release?

The docker-push target builds multi-platform images and pushes them to registries with versioned tags, while docker-release additionally updates the latest tag. Specifically, docker-release calls docker-push-latest, which ensures both the versioned and latest tags point to the current build.

Can I run just the API without building Docker containers?

Yes. Use make api to launch the FastAPI server directly via the run_api.py script. This is useful for debugging the backend without container overhead, though you must ensure SurrealDB is running separately for full functionality.

Where are the background worker commands defined?

The worker lifecycle targets in the Makefile manage the Surreal-commands background worker, which processes tasks defined in the commands/ directory (such as embedding_commands.py). Use make worker-start, make worker-stop, or make worker-restart to control this process independently of the API server.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}