Setting up Ollama for Local AI Inference with Open Notebook

Open Notebook integrates with Ollama as a drop-in replacement for cloud LLM providers by connecting to its HTTP API at http://host:11434, enabling fully local chat and embedding inference without external API keys.

Setting up Ollama for local AI inference allows you to run large language models and embedding models entirely on your own hardware, eliminating reliance on cloud providers and associated costs. The lfnovo/open-notebook repository supports Ollama through a dedicated credential system that communicates with Ollama's REST endpoints. This guide walks through the complete configuration process, from installation to model registration, as documented in docs/5-CONFIGURATION/ollama.md.

Installation and Model Setup

Begin by installing Ollama on your host machine using the official installer. According to docs/5-CONFIGURATION/ollama.md lines 18-25, Linux and macOS users can run the installation script, while Windows users should use the dedicated installer.

# Install Ollama (Linux/macOS)

curl -fsSL https://ollama.ai/install.sh | sh

After installation, pull the required models for both chat completion and semantic search. As specified in docs/5-CONFIGURATION/ollama.md lines 28-38, you need at least one language model and one embedding model:

ollama pull qwen3               # General-purpose LLM

ollama pull mxbai-embed-large   # Embedding model for search

Network Configuration Scenarios

The base URL configuration depends on your deployment architecture. The docs/5-CONFIGURATION/ollama.md file lines 60-76 outlines several network scenarios.

Local Installation (Same Machine)

When both services run directly on the host machine without containers, use the localhost endpoint:

http://localhost:11434

Docker Host with External Ollama

If Open Notebook runs inside Docker while Ollama runs on the host, use the Docker internal gateway. On macOS and Windows, the URL is http://host.docker.internal:11434. However, Linux requires an additional extra_hosts entry in your docker-compose.yml as documented in lines 83-95:

services:
  open-notebook:
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - OLLAMA_API_BASE=http://host.docker.internal:11434

Docker Compose Stack (Integrated)

When both services run in the same Docker Compose stack, reference the service name directly. Lines 124-152 in docs/5-CONFIGURATION/ollama.md provide a complete reference configuration:

services:
  open-notebook:
    image: lfnovo/open_notebook:v1-latest
    ports:
      - "8502:8502"
      - "5055:5055"
    environment:
      - OPEN_NOTEBOOK_ENCRYPTION_KEY=change-me-to-a-secret-string
    volumes:
      - ./notebook_data:/app/data
    depends_on:
      - ollama

  ollama:
    image: ollama/ollama:latest
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

volumes:
  ollama_data:

Enabling External Access

By default, Ollama binds only to 127.0.0.1, which blocks Docker containers from connecting. As noted in lines 76-80, you must set OLLAMA_HOST before starting the server:

export OLLAMA_HOST=0.0.0.0:11434
ollama serve

Configuring Open Notebook Credentials

Once Ollama is accessible, add the credential in the Open Notebook interface. Navigate to Settings → API Keys → Add Credential → select Ollama, and enter the appropriate base URL (as documented in lines 41-46).

The system also supports the legacy OLLAMA_API_BASE environment variable, though this method is now deprecated in favor of the Settings UI.

Registering Models in Open Notebook

After establishing the connection, register specific models in Settings → Models → Add Model. You must use the exact names shown by ollama list (e.g., gemma3:12b or qwen3), as detailed in lines 48-74.

Open Notebook will route chat requests to Ollama's /api/generate endpoint and embedding requests to /api/embeddings, treating the local server exactly like any cloud provider.

Verification and Testing

Verify connectivity using the health-check script from docs/5-CONFIGURATION/ollama.md lines 60-73:

#!/usr/bin/env bash
OLLAMA_API_BASE=${OLLAMA_API_BASE:-http://localhost:11434}
if curl -s "${OLLAMA_API_BASE}/api/tags" > /dev/null; then
  echo "✅ Ollama is running"
  curl -s "${OLLAMA_API_BASE}/api/tags" | jq -r '.models[].name'
else
  echo "❌ Ollama is not reachable at ${OLLAMA_API_BASE}"
  exit 1
fi

For a minimal Python test (lines 40-58), use:

import os, requests
base = os.getenv("OLLAMA_API_BASE", "http://localhost:11434")
print("Models:", requests.get(f"{base}/api/tags").json()["models"])
payload = {"model": "qwen3", "prompt": "Explain quantum computing", "stream": False}
print("Response:", requests.post(f"{base}/api/generate", json=payload).json()["response"])

Summary

• Install Ollama using the official script and pull required models (qwen3, mxbai-embed-large) before connecting to Open Notebook.
• Configure network access based on your deployment: use http://localhost:11434 for local setups, http://host.docker.internal:11434 for Docker-on-host (with Linux extra_hosts), or http://ollama:11434 for integrated Compose stacks.
• Enable external connections by setting OLLAMA_HOST=0.0.0.0:11434 before starting Ollama, as the default 127.0.0.1 binding blocks container access.
• Add credentials in Settings → API Keys using the Ollama provider type, then register exact model names from ollama list in Settings → Models.
• Reference files: docs/5-CONFIGURATION/ollama.md contains the complete setup guide, while examples/docker-compose-ollama.yml provides a working reference stack.

Frequently Asked Questions

What is the correct Ollama base URL when running Open Notebook in Docker?

The correct URL depends on where Ollama runs relative to the Open Notebook container. Use http://host.docker.internal:11434 when Ollama runs on the host machine (macOS/Windows), http://localhost:11434 when both run on the bare host, and http://ollama:11434 when both services share a Docker Compose network according to docs/5-CONFIGURATION/ollama.md.

Why can't Open Notebook connect to Ollama on Linux Docker?

Linux Docker requires explicit host gateway configuration. You must add extra_hosts: ["host.docker.internal:host-gateway"] to your docker-compose.yml for the Open Notebook service, as the internal DNS name does not resolve automatically on Linux systems unlike macOS or Windows.

Which environment variables does Ollama require for Open Notebook integration?

Ollama requires OLLAMA_HOST=0.0.0.0:11434 to accept connections from outside the host machine, which is necessary for Docker-based Open Notebook deployments. The deprecated OLLAMA_API_BASE variable in Open Notebook is superseded by the Settings UI credential system.

How do I verify that Ollama is running correctly before adding credentials?

Query the /api/tags endpoint to list available models. If curl http://localhost:11434/api/tags returns a JSON model list, the server is operational. The Open Notebook documentation includes a bash health-check script that automates this verification and pretty-prints installed models.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}