How to Integrate lfnovo/open-notebook with Other Systems: REST API and Python Client Guide

Open Notebook exposes a comprehensive REST API built on FastAPI that enables seamless integration with external applications, CI pipelines, and AI workflows through HTTP requests, a native Python client, or direct database access.

The lfnovo/open-notebook repository provides a self-hosted, privacy-first research assistant designed for extensibility. Its architecture exposes every major function—from notebook management to semantic search and podcast generation—via stateless HTTP endpoints. This design allows developers to integrate Open Notebook's capabilities into existing toolchains or build custom interfaces on top of its core engine.

Core Architectural Integration Points

Open Notebook provides multiple integration layers, each optimized for different use cases. All components are stateless and communicate through the FastAPI server defined in api/main.py.

FastAPI HTTP Server

The primary integration surface is the FastAPI HTTP server, which exposes all public endpoints under /api/*. These endpoints handle notebooks, sources, notes, search, chat, models, and podcasts. You can interact with these using standard HTTP verbs (GET, POST, PUT, DELETE) from any programming language.

Open Notebook Python Client

For Python projects, the api/client.py file provides the OpenNotebookClient class. This thin wrapper handles URL construction, authentication headers, and response deserialization, offering methods like create_source(), search(), and chat_send() that map 1:1 to the REST API.

Command Service and Job Queue

Asynchronous operations—such as source ingestion and podcast generation—run through the Surreal-Commands queue. The api/command_service.py file implements the CommandService class, which manages background job execution. External systems can enqueue jobs by posting to /api/commands or by using the internal service class directly.

AI Model Abstraction

The Esperanto library, located in open_notebook/ai/models.py, unifies 18+ LLM, embedding, TTS, and STT providers behind a single interface. Credentials can be configured via environment variables or database records, allowing the API to route calls to the selected provider automatically.

Database Layer

For direct data access, open_notebook/database/repository.py exposes an async repo_query helper. SurrealDB stores all notebooks, sources, embeddings, and command metadata, enabling bulk migrations or custom reporting queries from external services.

Typical Integration Workflows

Integrating with Open Notebook follows a predictable pattern of creation, processing, and retrieval.

1. Create or import a source by uploading PDFs, URLs, or raw text through /api/sources.
2. Trigger async processing by setting async_processing=true. The background job extracts text, generates embeddings, and runs transformations.
3. Search and retrieve using /api/search for semantic results or /api/notebooks/{id} for structured content.
4. Initiate chat sessions via /api/chat to create conversational AI experiences grounded in notebook context.
5. Generate podcasts by calling /api/podcasts, which starts multi-speaker TTS jobs tracked through the command endpoint.

Implementation Examples

Using the Built-in Python Client

The OpenNotebookClient class in api/client.py provides the most straightforward integration for Python applications.

from api.client import OpenNotebookClient
import time

# Initialize with your deployment URL and API key

client = OpenNotebookClient(base_url="http://localhost:5055", api_key="my-secret-token")

# 1. Create a new notebook

notebook = client.create_notebook(name="Research Project", description="AI-augmented notes")
print("Notebook ID:", notebook.id)

# 2. Upload a PDF source with async processing

source = client.create_source(
    type="upload",
    title="Important Paper",
    notebooks=[notebook.id],
    async_processing=True,
    file_path="/path/to/paper.pdf",
)

print("Source queued, command ID:", source.command_id)

# 3. Poll the command until completion

while True:
    cmd = client.get_command(source.command_id)
    if cmd.status in ("completed", "failed"):
        break
    time.sleep(2)

# 4. Search the notebook once embeddings are ready

results = client.search(query="neural scaling laws", notebook_id=notebook.id)
for hit in results:
    print(hit.title, hit.score)

# 5. Chat with the notebook

chat = client.start_chat(notebook_id=notebook.id)
response = client.chat_send(chat_id=chat.id, message="Summarise the key findings.")
print(response.content)

All client methods correspond directly to routes defined in api/routers/*, such as client.create_source mapping to POST /api/sources.

Raw HTTP Integration

For non-Python systems, use standard HTTP requests. This cURL example creates a source and performs semantic search:

# Create a source with file upload

curl -X POST http://localhost:5055/api/sources \
  -F "type=upload" \
  -F "title=Demo PDF" \
  -F "notebooks=[\"$NOTEBOOK_ID\"]" \
  -F "async_processing=true" \
  -F "file=@/tmp/demo.pdf"

# Poll command status (replace command:xyz with actual ID from response)

curl http://localhost:5055/api/commands/command:xyz

# Execute semantic search

curl -G http://localhost:5055/api/search \
  --data-urlencode "query=quantum entanglement" \
  --data-urlencode "notebook_id=$NOTEBOOK_ID"

Enqueuing Custom Background Jobs

External worker services can trigger custom transformations through the command queue:

import requests

payload = {
    "source_id": "source:123",
    "transformation_id": "transformation:abc",
}

resp = requests.post(
    "http://localhost:5055/api/commands",
    json={"command_name": "run_transformation", "payload": payload},
    headers={"Authorization": "Bearer my-token"},
)

command_id = resp.json()["command_id"]
print("Queued command:", command_id)

The command is stored in SurrealDB and processed by the job queue defined in api/command_service.py. Check status via GET /api/commands/{command_id}.

Key Source Files for Integration

When building custom integrations, reference these core files:

• api/main.py: FastAPI entry point handling CORS, authentication middleware, and router registration.
• api/routers/*: Individual endpoint implementations for models, sources, search, chat, and podcasts.
• api/client.py: Python wrapper class providing typed methods for all API operations.
• api/command_service.py: Background job queueing and status retrieval logic.
• open_notebook/ai/models.py: Unified AI provider interface (Esperanto) for LLM and embedding operations.
• open_notebook/database/repository.py: Low-level async SurrealDB query helper.
• open_notebook/config.py: Environment variable and path configuration.
• open_notebook/domain/notebook.py: Domain models backing the API.

Security and Deployment Considerations

Authentication defaults to PasswordAuthMiddleware in api/auth.py. For production integrations, replace this with JWT or API-key middleware.

Configure CORS origins via the CORS_ORIGINS environment variable to prevent cross-origin blocks when calling from browser-based external systems.

Store model credentials either as environment variables (e.g., OPENAI_API_KEY) or as Credential records in the database. The endpoint GET /api/models/providers returns currently active providers.

Summary

• Open Notebook exposes a complete REST API via FastAPI in api/main.py, enabling integration with any HTTP-capable system.
• The OpenNotebookClient class in api/client.py provides a type-safe Python interface for all API operations.
• Async workflows use the Surreal-Commands queue managed by api/command_service.py, requiring polling for completion status.
• Direct database access is available through open_notebook/database/repository.py for bulk operations or custom queries.
• Authentication is middleware-based and swappable, while CORS settings control cross-origin access from external UIs.

Frequently Asked Questions

Does Open Notebook require authentication for API access?

By default, the API runs behind PasswordAuthMiddleware as defined in api/auth.py. However, this can be replaced with JWT or API-key middleware for production environments, allowing seamless machine-to-machine authentication.

Can I integrate Open Notebook with non-Python applications?

Yes. The FastAPI server in api/main.py exposes standard HTTP endpoints that accept JSON and multipart requests. Any language capable of HTTP requests—JavaScript, Go, Rust, or shell scripts—can interact with the service directly without using the Python client.

How does async processing work when creating sources?

When you set async_processing=true in a source creation request, the API returns immediately with a command_id. The actual processing—text extraction, embedding generation, and transformations—runs as a background job in the Surreal-Commands queue. You poll GET /api/commands/{command_id} to track progress until status shows completed or failed.

What database does Open Notebook use for storing notebooks and sources?

Open Notebook uses SurrealDB as its primary datastore. The open_notebook/database/repository.py file provides an async repo_query helper for direct database access, while most integrations should use the REST API to ensure data consistency and validation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}