# Running Local AI with Ollama for Open Notebook: Complete Setup Guide

> Learn to run local AI with Ollama for Open Notebook. Configure API keys and set the base URL for seamless local model integration. Full setup guide.

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: how-to-guide
- Published: 2026-06-17

---

**Configure Open Notebook to use locally-hosted Ollama models by setting the base URL in Settings → API Keys and ensuring exact model name matches from `ollama list`.**

Open Notebook is a three-tier application combining a **Next.js frontend**, a **FastAPI backend**, and a **SurrealDB graph database**. When you want to run completely local AI without external API calls, you integrate Ollama by adding a credential with the provider name "ollama" and pointing it at your local Ollama server URL. This guide covers the architecture, network configurations, and exact steps to get local language and embedding models working.

## How Ollama Integrates with Open Notebook Architecture

### The Three-Tier System

According to the repository architecture defined in [`CLAUDE.md`](https://github.com/lfnovo/open-notebook/blob/main/CLAUDE.md), Open Notebook separates concerns across three layers:

- **Frontend** (`frontend/`): The React-based UI for notebooks, sources, and chat that communicates with the backend via `http://localhost:5055`
- **API Layer** (`api/` and `open_notebook/`): FastAPI endpoints (`/chat`, `/ask`, `/transformations`) and LangGraph workflows that handle extraction, embedding, and generation
- **Database** (`surrealdb/`): Stores notebooks, sources, embeddings, and credentials including the `Credential` records for Ollama

### Where Ollama Fits In

The backend uses the **Esperanto** library to abstract LLM providers. When you configure an Ollama credential, the system in [`open_notebook/ai/model_manager.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/model_manager.py) maps requests to Ollama's HTTP API endpoints (`/api/generate`, `/api/tags`). SurrealDB stores the model name strings exactly as they appear in `ollama list`, with no transformation or rewriting performed by the application code.

## Network Configuration for Ollama

Because Ollama binds to `localhost` by default, you must configure the correct base URL based on your deployment topology:

| Deployment Scenario | URL for Settings → API Keys |
|---|---|
| Ollama and Open Notebook on same host (no Docker) | `http://localhost:11434` |
| Open Notebook in Docker, Ollama on host (Linux) | `http://host.docker.internal:11434` (requires `extra_hosts: ["host.docker.internal:host-gateway"]` in [`docker-compose.yml`](https://github.com/lfnovo/open-notebook/blob/main/docker-compose.yml)) |
| Both services in same Docker Compose stack | `http://ollama:11434` |
| Remote Ollama server | `http://<IP-ADDRESS>:11434` |

**Critical:** When running Open Notebook inside a container, `localhost` refers to the container itself, not the host. You must either expose Ollama on all interfaces using `OLLAMA_HOST=0.0.0.0:11434` or use Docker's internal DNS resolution.

## Model Naming Requirements

Open Notebook performs **no model name translation**. The exact string returned by the `ollama list` command must be used when registering models in the UI. For example:

- `qwen3:latest` (not "qwen3")
- `mxbai-embed-large:latest` (not "mxbai-embed")

A mismatch between the registered name and Ollama's internal name causes the generic "Failed to send message" error. This validation occurs in the credential handling logic referenced in [`open_notebook/domain/models.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/domain/models.py).

## Docker Compose Setup for Local AI

The repository provides [`examples/docker-compose-ollama.yml`](https://github.com/lfnovo/open-notebook/blob/main/examples/docker-compose-ollama.yml) for a complete local deployment. This configuration starts SurrealDB, Ollama, and Open Notebook with persistent volumes:

```yaml
services:
  surrealdb:
    image: surrealdb/surrealdb:v2
    command: start --log info --user root --pass root rocksdb:/mydata/mydatabase.db
    ports: ["8000:8000"]
    volumes: ["./surreal_data:/mydata"]
    restart: always

  ollama:
    image: ollama/ollama:latest
    ports: ["11434:11434"]
    volumes: ["ollama_models:/root/.ollama"]
    restart: always

  open_notebook:
    image: lfnovo/open_notebook:v1-latest
    ports: ["8502:8502", "5055:5055"]
    environment:
      - OPEN_NOTEBOOK_ENCRYPTION_KEY=change-me-to-a-secret-string
      - SURREAL_URL=ws://surrealdb:8000/rpc
      - SURREAL_USER=root
      - SURREAL_PASSWORD=root
      - SURREAL_NAMESPACE=open_notebook
      - SURREAL_DATABASE=open_notebook
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes: ["./notebook_data:/app/data"]
    depends_on: [surrealdb, ollama]
    restart: always

volumes:
  ollama_models:

```

Save this as [`docker-compose.yml`](https://github.com/lfnovo/open-notebook/blob/main/docker-compose.yml), modify the `OPEN_NOTEBOOK_ENCRYPTION_KEY`, and run `docker compose up -d`. The `OLLAMA_BASE_URL` environment variable ensures the backend can reach Ollama using the internal Docker network hostname.

## Configuring Ollama Credentials in Open Notebook

After containers are healthy, complete the setup through the UI:

1. **Pull models** into the Ollama container:
   ```bash
   docker exec <ollama-container-name> ollama pull qwen3
   docker exec <ollama-container-name> ollama pull mxbai-embed-large
   ```

2. **Add the credential** in the UI at **Settings → API Keys**:
   - Select provider: **ollama**
   - Base URL: `http://ollama:11434` (or your configured URL)
   - No API key required for local instances

3. **Register language models** at **Settings → Models → Add Model**:
   - Enter exact name: `qwen3:latest`
   - Select provider: ollama
   - Mark as default chat model

4. **Register embedding models** at **Settings → Models → Add Model**:
   - Enter exact name: `mxbai-embed-large:latest`
   - Set as default search embedding model

The FastAPI layer in [`api/main.py`](https://github.com/lfnovo/open-notebook/blob/main/api/main.py) now routes generation requests to your local Ollama instance instead of remote APIs.

## Troubleshooting and Verification

### Health Check with curl

Verify Ollama is reachable from your Open Notebook host:

```bash
curl -s http://localhost:11434/api/tags | jq '.models[].name'

```

If running from inside the Open Notebook container, replace `localhost` with the appropriate hostname (`host.docker.internal` or `ollama`).

### Python Client Example

For debugging or custom scripts, interact directly with Ollama's HTTP API:

```python
import os
import requests

OLLAMA_BASE = os.getenv("OLLAMA_API_BASE", "http://localhost:11434")

# List available models

resp = requests.get(f"{OLLAMA_BASE}/api/tags")
print("Available models:", [m["name"] for m in resp.json()["models"]])

# Test generation

payload = {
    "model": "qwen3:latest",
    "prompt": "Explain why local AI is useful",
    "stream": False,
}
generated = requests.post(
    f"{OLLAMA_BASE}/api/generate", 
    json=payload
).json()
print(generated["response"])

```

### Fixing Model Name Mismatches

If you encounter "Failed to send message" errors, verify the exact model name:

```bash

# Get the precise identifier from Ollama

ollama list

# Use the full string including tags (e.g., ":latest") in the Open Notebook UI

```

Common errors occur when users enter `qwen3` instead of `qwen3:latest` or omit the repository prefix for custom models.

## Summary

- **Architecture**: Open Notebook uses Esperanto in the FastAPI backend ([`open_notebook/ai/model_manager.py`](https://github.com/lfnovo/open-notebook/blob/main/open_notebook/ai/model_manager.py)) to route requests to Ollama's HTTP API, with SurrealDB storing exact model name strings.
- **Networking**: Use `http://localhost:11434` for native installs, `http://host.docker.internal:11434` for Docker-on-host setups, or `http://ollama:11434` for shared Compose networks.
- **Naming**: Register models using the exact output from `ollama list` to avoid provider errors.
- **Deployment**: Use [`examples/docker-compose-ollama.yml`](https://github.com/lfnovo/open-notebook/blob/main/examples/docker-compose-ollama.yml) for a complete local stack with persistent storage for both notebooks and model weights.

## Frequently Asked Questions

### Why does Open Notebook say "Failed to send message" when using Ollama?

This error typically indicates a **model name mismatch**. Open Notebook does not modify the model identifier you enter; it must match the exact string shown by `ollama list` including tags like `:latest`. Additionally, verify that the Ollama base URL in your credential settings is reachable from the Open Notebook container (test with `curl` from inside the container if using Docker).

### Can I use Ollama running on a different machine than Open Notebook?

Yes. In **Settings → API Keys**, set the Ollama base URL to `http://<IP-ADDRESS>:11434` where the IP address is accessible from your Open Notebook host. Ensure Ollama is started with `OLLAMA_HOST=0.0.0.0:11434` to bind to all interfaces, not just localhost. Be aware that Ollama does not implement authentication by default, so secure the network appropriately.

### How do I switch between local Ollama and cloud providers like OpenAI?

Open Notebook supports multiple concurrent credentials. You can add both an Ollama credential and an OpenAI API key in **Settings → API Keys**. When adding or editing models in **Settings → Models**, select the desired provider from the dropdown. You can set different default models for chat and embeddings, allowing you to mix local embeddings (Ollama) with cloud generation (OpenAI) or vice versa.

### Where are my Ollama models stored when using Docker Compose?

The provided [`docker-compose-ollama.yml`](https://github.com/lfnovo/open-notebook/blob/main/docker-compose-ollama.yml) defines a named volume `ollama_models` mounted at `/root/.ollama` inside the Ollama container. This persists model weights across container restarts. If you want to use existing host models, modify the volume mapping to mount your host's `~/.ollama` directory into the container instead of using the named volume.