How to Run Neuro SAN with Ollama Models Locally or in Docker

Neuro SAN supports locally-hosted Ollama models by configuring the llm_config block with a model_name and optional base_url, enabling fully offline agent networks with tool-calling capabilities.

Running large language models locally is essential for privacy-sensitive agent networks. The cognizant-ai-lab/neuro-san-studio repository provides native integration with Ollama, allowing you to run Neuro SAN with local models instead of cloud APIs. This guide covers the exact configuration steps, file paths, and Docker deployment patterns needed to connect your agent networks to Ollama's HTTP server.

Architecture and Prerequisites

Neuro SAN integrates with Ollama through LangChain's ChatOllama interface. The Ollama server exposes a lightweight HTTP API on port 11434 by default, serving quantized models that support tool-calling.

Only models advertising the tool capability can execute Neuro SAN's coded tools. Verify compatibility using Ollama's searchable tool-capable model registry before proceeding.

Local Setup Guide

Install and Pull Ollama Models

First, install Ollama and download a tool-capable model:

curl -fsSL https://ollama.com/install.sh | sh
ollama run qwen3:8b

Verify the installation:

ollama list

Register Models in default_llm_info.hocon

Neuro SAN maintains a registry of known LLM configurations in neuro_san/internals/run_context/langchain/llms/default_llm_info.hocon. Add your Ollama model if not present:

"qwen3:8b": {
    "class": "ollama",
    "max_output_tokens": 8192
}

The class field shortcuts the full module path, instantiating ChatOllama internally.

Configure Agent Network HOCON

Create or modify your agent-network configuration file (e.g., registries/music_nerd.hocon). The llm_config block at the root level determines which model all agents use unless overridden:

{
    "llm_config": {
        "model_name": "qwen3:8b"
    },
    "agents": [
        {
            "name": "MusicNerd",
            "toolbox": "search",
            "instructions": "You are a music expert..."
        }
    ]
}

For models not in the default registry, explicitly specify the provider:

"llm_config": {
    "class": "ollama",
    "model_name": "llama3.1:8b"
}

Start the Neuro SAN Server

Launch the gRPC server using the main loop entry point:

python -m neuro_san.service.main_loop.server_main_loop --port 30011

Alternatively, use the convenience wrapper:

python -m run

The server loads the manifest from registries/manifest.hocon and instantiates ChatOllama instances for each configured network.

Docker and Remote Deployment

When Ollama runs in a separate container or remote host, update the base_url field in your HOCON configuration.

Container Network Configuration

Set base_url to the reachable container name or IP address. The URL must include the protocol (http:// or https://) or Neuro SAN falls back to http://127.0.0.1:11434:

"llm_config": {
    "model_name": "qwen3:8b",
    "base_url": "http://ollama:11434"
}

Docker Compose Example

Create a docker-compose.yml in the repository root:

version: "3.8"
services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    restart: unless-stopped

  neuro-san:
    build: .
    environment:
      - AGENT_MANIFEST_FILE=./registries/manifest.hocon
    depends_on:
      - ollama
    ports:
      - "30011:30011"

Deploy with:

docker compose up -d

Verify Ollama health before starting Neuro SAN:

curl http://localhost:11434/api/version

Summary

• Tool-calling requirement: Only Ollama models with tool capability work with Neuro SAN's coded tools.
• Configuration location: Add model metadata to neuro_san/internals/run_context/langchain/llms/default_llm_info.hocon if not already present.
• HOCON syntax: Use model_name in llm_config; add class: "ollama" for unregistered models or base_url for remote endpoints.
• Docker networking: Set base_url to the container service name (e.g., http://ollama:11434) when running in compose stacks.
• Server startup: Use python -m neuro_san.service.main_loop.server_main_loop or python -m run to start the agent network.

Frequently Asked Questions

Which Ollama models work with Neuro SAN?

Only models advertising tool-calling support in their Ollama metadata function correctly with Neuro SAN's coded tools. Check the model's capabilities on Ollama's search page or pull a verified tool-capable model like qwen3:8b or llama3.1:8b.

Where does Neuro SAN look for Ollama configuration?

The runtime reads the llm_config block from your agent-network HOCON file. It cross-references model_name against neuro_san/internals/run_context/langchain/llms/default_llm_info.hocon to determine the provider class and parameters, instantiating a LangChain ChatOllama object for each agent.

Can I run Neuro SAN and Ollama on different machines?

Yes. Set the base_url parameter in llm_config to the remote Ollama server's address (e.g., http://192.168.1.10:11434). Ensure the URL includes the http:// or https:// protocol prefix; otherwise, Neuro SAN defaults to http://127.0.0.1:11434.

How do I verify the integration is working?

After starting both services, check that the Neuro SAN server logs show successful model instantiation without connection errors to port 11434. Run a sample agent network like docs/examples/music_nerd_pro_local.md and verify that tool calls execute against your local model rather than returning cloud API errors.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}