Running Local AI with Ollama for Open Notebook: Complete Setup Guide

Configure Open Notebook to use locally-hosted Ollama models by setting the base URL in Settings → API Keys and ensuring exact model name matches from ollama list.

Open Notebook is a three-tier application combining a Next.js frontend, a FastAPI backend, and a SurrealDB graph database. When you want to run completely local AI without external API calls, you integrate Ollama by adding a credential with the provider name "ollama" and pointing it at your local Ollama server URL. This guide covers the architecture, network configurations, and exact steps to get local language and embedding models working.

How Ollama Integrates with Open Notebook Architecture

The Three-Tier System

According to the repository architecture defined in CLAUDE.md, Open Notebook separates concerns across three layers:

• Frontend (frontend/): The React-based UI for notebooks, sources, and chat that communicates with the backend via http://localhost:5055
• API Layer (api/ and open_notebook/): FastAPI endpoints (/chat, /ask, /transformations) and LangGraph workflows that handle extraction, embedding, and generation
• Database (surrealdb/): Stores notebooks, sources, embeddings, and credentials including the Credential records for Ollama

Where Ollama Fits In

The backend uses the Esperanto library to abstract LLM providers. When you configure an Ollama credential, the system in open_notebook/ai/model_manager.py maps requests to Ollama's HTTP API endpoints (/api/generate, /api/tags). SurrealDB stores the model name strings exactly as they appear in ollama list, with no transformation or rewriting performed by the application code.

Network Configuration for Ollama

Because Ollama binds to localhost by default, you must configure the correct base URL based on your deployment topology:

Deployment Scenario	URL for Settings → API Keys
Ollama and Open Notebook on same host (no Docker)	http://localhost:11434
Open Notebook in Docker, Ollama on host (Linux)	http://host.docker.internal:11434 (requires extra_hosts: ["host.docker.internal:host-gateway"] in docker-compose.yml)
Both services in same Docker Compose stack	http://ollama:11434
Remote Ollama server	http://<IP-ADDRESS>:11434

Critical: When running Open Notebook inside a container, localhost refers to the container itself, not the host. You must either expose Ollama on all interfaces using OLLAMA_HOST=0.0.0.0:11434 or use Docker's internal DNS resolution.

Model Naming Requirements

Open Notebook performs no model name translation. The exact string returned by the ollama list command must be used when registering models in the UI. For example:

• qwen3:latest (not "qwen3")
• mxbai-embed-large:latest (not "mxbai-embed")

A mismatch between the registered name and Ollama's internal name causes the generic "Failed to send message" error. This validation occurs in the credential handling logic referenced in open_notebook/domain/models.py.

Docker Compose Setup for Local AI

The repository provides examples/docker-compose-ollama.yml for a complete local deployment. This configuration starts SurrealDB, Ollama, and Open Notebook with persistent volumes:

services:
  surrealdb:
    image: surrealdb/surrealdb:v2
    command: start --log info --user root --pass root rocksdb:/mydata/mydatabase.db
    ports: ["8000:8000"]
    volumes: ["./surreal_data:/mydata"]
    restart: always

  ollama:
    image: ollama/ollama:latest
    ports: ["11434:11434"]
    volumes: ["ollama_models:/root/.ollama"]
    restart: always

  open_notebook:
    image: lfnovo/open_notebook:v1-latest
    ports: ["8502:8502", "5055:5055"]
    environment:
      - OPEN_NOTEBOOK_ENCRYPTION_KEY=change-me-to-a-secret-string
      - SURREAL_URL=ws://surrealdb:8000/rpc
      - SURREAL_USER=root
      - SURREAL_PASSWORD=root
      - SURREAL_NAMESPACE=open_notebook
      - SURREAL_DATABASE=open_notebook
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes: ["./notebook_data:/app/data"]
    depends_on: [surrealdb, ollama]
    restart: always

volumes:
  ollama_models:

Save this as docker-compose.yml, modify the OPEN_NOTEBOOK_ENCRYPTION_KEY, and run docker compose up -d. The OLLAMA_BASE_URL environment variable ensures the backend can reach Ollama using the internal Docker network hostname.

Configuring Ollama Credentials in Open Notebook

After containers are healthy, complete the setup through the UI:

1. Pull models into the Ollama container:

   docker exec <ollama-container-name> ollama pull qwen3
   docker exec <ollama-container-name> ollama pull mxbai-embed-large

2. Add the credential in the UI at Settings → API Keys:

   • Select provider: ollama
   • Base URL: http://ollama:11434 (or your configured URL)
   • No API key required for local instances

3. Register language models at Settings → Models → Add Model:

   • Enter exact name: qwen3:latest
   • Select provider: ollama
   • Mark as default chat model

4. Register embedding models at Settings → Models → Add Model:

   • Enter exact name: mxbai-embed-large:latest
   • Set as default search embedding model

The FastAPI layer in api/main.py now routes generation requests to your local Ollama instance instead of remote APIs.

Troubleshooting and Verification

Health Check with curl

Verify Ollama is reachable from your Open Notebook host:

curl -s http://localhost:11434/api/tags | jq '.models[].name'

If running from inside the Open Notebook container, replace localhost with the appropriate hostname (host.docker.internal or ollama).

Python Client Example

For debugging or custom scripts, interact directly with Ollama's HTTP API:

import os
import requests

OLLAMA_BASE = os.getenv("OLLAMA_API_BASE", "http://localhost:11434")

# List available models

resp = requests.get(f"{OLLAMA_BASE}/api/tags")
print("Available models:", [m["name"] for m in resp.json()["models"]])

# Test generation

payload = {
    "model": "qwen3:latest",
    "prompt": "Explain why local AI is useful",
    "stream": False,
}
generated = requests.post(
    f"{OLLAMA_BASE}/api/generate", 
    json=payload
).json()
print(generated["response"])

Fixing Model Name Mismatches

If you encounter "Failed to send message" errors, verify the exact model name:

# Get the precise identifier from Ollama

ollama list

# Use the full string including tags (e.g., ":latest") in the Open Notebook UI

Common errors occur when users enter qwen3 instead of qwen3:latest or omit the repository prefix for custom models.

Summary

• Architecture: Open Notebook uses Esperanto in the FastAPI backend (open_notebook/ai/model_manager.py) to route requests to Ollama's HTTP API, with SurrealDB storing exact model name strings.
• Networking: Use http://localhost:11434 for native installs, http://host.docker.internal:11434 for Docker-on-host setups, or http://ollama:11434 for shared Compose networks.
• Naming: Register models using the exact output from ollama list to avoid provider errors.
• Deployment: Use examples/docker-compose-ollama.yml for a complete local stack with persistent storage for both notebooks and model weights.

Frequently Asked Questions

Why does Open Notebook say "Failed to send message" when using Ollama?

This error typically indicates a model name mismatch. Open Notebook does not modify the model identifier you enter; it must match the exact string shown by ollama list including tags like :latest. Additionally, verify that the Ollama base URL in your credential settings is reachable from the Open Notebook container (test with curl from inside the container if using Docker).

Can I use Ollama running on a different machine than Open Notebook?

Yes. In Settings → API Keys, set the Ollama base URL to http://<IP-ADDRESS>:11434 where the IP address is accessible from your Open Notebook host. Ensure Ollama is started with OLLAMA_HOST=0.0.0.0:11434 to bind to all interfaces, not just localhost. Be aware that Ollama does not implement authentication by default, so secure the network appropriately.

How do I switch between local Ollama and cloud providers like OpenAI?

Open Notebook supports multiple concurrent credentials. You can add both an Ollama credential and an OpenAI API key in Settings → API Keys. When adding or editing models in Settings → Models, select the desired provider from the dropdown. You can set different default models for chat and embeddings, allowing you to mix local embeddings (Ollama) with cloud generation (OpenAI) or vice versa.

Where are my Ollama models stored when using Docker Compose?

The provided docker-compose-ollama.yml defines a named volume ollama_models mounted at /root/.ollama inside the Ollama container. This persists model weights across container restarts. If you want to use existing host models, modify the volume mapping to mount your host's ~/.ollama directory into the container instead of using the named volume.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}