# Setting up Ollama for Local AI Inference with Open Notebook

> Set up Ollama for local AI inference with Open Notebook Connect seamlessly to Ollama's API for private chat and embedding without cloud reliance

- Repository: [Luis Novo/open-notebook](https://github.com/lfnovo/open-notebook)
- Tags: tutorial
- Published: 2026-06-30

---

**Open Notebook integrates with Ollama as a drop-in replacement for cloud LLM providers by connecting to its HTTP API at `http://host:11434`, enabling fully local chat and embedding inference without external API keys.**

Setting up Ollama for local AI inference allows you to run large language models and embedding models entirely on your own hardware, eliminating reliance on cloud providers and associated costs. The `lfnovo/open-notebook` repository supports Ollama through a dedicated credential system that communicates with Ollama's REST endpoints. This guide walks through the complete configuration process, from installation to model registration, as documented in [`docs/5-CONFIGURATION/ollama.md`](https://github.com/lfnovo/open-notebook/blob/main/docs/5-CONFIGURATION/ollama.md).

## Installation and Model Setup

Begin by installing Ollama on your host machine using the official installer. According to [`docs/5-CONFIGURATION/ollama.md`](https://github.com/lfnovo/open-notebook/blob/main/docs/5-CONFIGURATION/ollama.md) lines 18-25, Linux and macOS users can run the installation script, while Windows users should use the dedicated installer.

```bash

# Install Ollama (Linux/macOS)

curl -fsSL https://ollama.ai/install.sh | sh

```

After installation, pull the required models for both chat completion and semantic search. As specified in [`docs/5-CONFIGURATION/ollama.md`](https://github.com/lfnovo/open-notebook/blob/main/docs/5-CONFIGURATION/ollama.md) lines 28-38, you need at least one language model and one embedding model:

```bash
ollama pull qwen3               # General-purpose LLM

ollama pull mxbai-embed-large   # Embedding model for search

```

## Network Configuration Scenarios

The base URL configuration depends on your deployment architecture. The [`docs/5-CONFIGURATION/ollama.md`](https://github.com/lfnovo/open-notebook/blob/main/docs/5-CONFIGURATION/ollama.md) file lines 60-76 outlines several network scenarios.

### Local Installation (Same Machine)

When both services run directly on the host machine without containers, use the localhost endpoint:

```

http://localhost:11434

```

### Docker Host with External Ollama

If Open Notebook runs inside Docker while Ollama runs on the host, use the Docker internal gateway. On macOS and Windows, the URL is `http://host.docker.internal:11434`. However, Linux requires an additional `extra_hosts` entry in your [`docker-compose.yml`](https://github.com/lfnovo/open-notebook/blob/main/docker-compose.yml) as documented in lines 83-95:

```yaml
services:
  open-notebook:
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - OLLAMA_API_BASE=http://host.docker.internal:11434

```

### Docker Compose Stack (Integrated)

When both services run in the same Docker Compose stack, reference the service name directly. Lines 124-152 in [`docs/5-CONFIGURATION/ollama.md`](https://github.com/lfnovo/open-notebook/blob/main/docs/5-CONFIGURATION/ollama.md) provide a complete reference configuration:

```yaml
services:
  open-notebook:
    image: lfnovo/open_notebook:v1-latest
    ports:
      - "8502:8502"
      - "5055:5055"
    environment:
      - OPEN_NOTEBOOK_ENCRYPTION_KEY=change-me-to-a-secret-string
    volumes:
      - ./notebook_data:/app/data
    depends_on:
      - ollama

  ollama:
    image: ollama/ollama:latest
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

volumes:
  ollama_data:

```

### Enabling External Access

By default, Ollama binds only to `127.0.0.1`, which blocks Docker containers from connecting. As noted in lines 76-80, you must set `OLLAMA_HOST` before starting the server:

```bash
export OLLAMA_HOST=0.0.0.0:11434
ollama serve

```

## Configuring Open Notebook Credentials

Once Ollama is accessible, add the credential in the Open Notebook interface. Navigate to Settings → API Keys → Add Credential → select **Ollama**, and enter the appropriate base URL (as documented in lines 41-46).

The system also supports the legacy `OLLAMA_API_BASE` environment variable, though this method is now deprecated in favor of the Settings UI.

## Registering Models in Open Notebook

After establishing the connection, register specific models in Settings → Models → Add Model. You must use the **exact** names shown by `ollama list` (e.g., `gemma3:12b` or `qwen3`), as detailed in lines 48-74.

Open Notebook will route chat requests to Ollama's `/api/generate` endpoint and embedding requests to `/api/embeddings`, treating the local server exactly like any cloud provider.

## Verification and Testing

Verify connectivity using the health-check script from [`docs/5-CONFIGURATION/ollama.md`](https://github.com/lfnovo/open-notebook/blob/main/docs/5-CONFIGURATION/ollama.md) lines 60-73:

```bash
#!/usr/bin/env bash
OLLAMA_API_BASE=${OLLAMA_API_BASE:-http://localhost:11434}
if curl -s "${OLLAMA_API_BASE}/api/tags" > /dev/null; then
  echo "✅ Ollama is running"
  curl -s "${OLLAMA_API_BASE}/api/tags" | jq -r '.models[].name'
else
  echo "❌ Ollama is not reachable at ${OLLAMA_API_BASE}"
  exit 1
fi

```

For a minimal Python test (lines 40-58), use:

```python
import os, requests
base = os.getenv("OLLAMA_API_BASE", "http://localhost:11434")
print("Models:", requests.get(f"{base}/api/tags").json()["models"])
payload = {"model": "qwen3", "prompt": "Explain quantum computing", "stream": False}
print("Response:", requests.post(f"{base}/api/generate", json=payload).json()["response"])

```

## Summary

- **Install Ollama** using the official script and pull required models (`qwen3`, `mxbai-embed-large`) before connecting to Open Notebook.
- **Configure network access** based on your deployment: use `http://localhost:11434` for local setups, `http://host.docker.internal:11434` for Docker-on-host (with Linux `extra_hosts`), or `http://ollama:11434` for integrated Compose stacks.
- **Enable external connections** by setting `OLLAMA_HOST=0.0.0.0:11434` before starting Ollama, as the default `127.0.0.1` binding blocks container access.
- **Add credentials** in Settings → API Keys using the Ollama provider type, then register exact model names from `ollama list` in Settings → Models.
- **Reference files**: [`docs/5-CONFIGURATION/ollama.md`](https://github.com/lfnovo/open-notebook/blob/main/docs/5-CONFIGURATION/ollama.md) contains the complete setup guide, while [`examples/docker-compose-ollama.yml`](https://github.com/lfnovo/open-notebook/blob/main/examples/docker-compose-ollama.yml) provides a working reference stack.

## Frequently Asked Questions

### What is the correct Ollama base URL when running Open Notebook in Docker?

The correct URL depends on where Ollama runs relative to the Open Notebook container. Use `http://host.docker.internal:11434` when Ollama runs on the host machine (macOS/Windows), `http://localhost:11434` when both run on the bare host, and `http://ollama:11434` when both services share a Docker Compose network according to [`docs/5-CONFIGURATION/ollama.md`](https://github.com/lfnovo/open-notebook/blob/main/docs/5-CONFIGURATION/ollama.md).

### Why can't Open Notebook connect to Ollama on Linux Docker?

Linux Docker requires explicit host gateway configuration. You must add `extra_hosts: ["host.docker.internal:host-gateway"]` to your [`docker-compose.yml`](https://github.com/lfnovo/open-notebook/blob/main/docker-compose.yml) for the Open Notebook service, as the internal DNS name does not resolve automatically on Linux systems unlike macOS or Windows.

### Which environment variables does Ollama require for Open Notebook integration?

Ollama requires `OLLAMA_HOST=0.0.0.0:11434` to accept connections from outside the host machine, which is necessary for Docker-based Open Notebook deployments. The deprecated `OLLAMA_API_BASE` variable in Open Notebook is superseded by the Settings UI credential system.

### How do I verify that Ollama is running correctly before adding credentials?

Query the `/api/tags` endpoint to list available models. If `curl http://localhost:11434/api/tags` returns a JSON model list, the server is operational. The Open Notebook documentation includes a bash health-check script that automates this verification and pretty-prints installed models.