How to Set Up Continue with Self-Hosted Models Like Ollama

To set up Continue with self-hosted models like Ollama, install the Ollama daemon locally, pull your desired model, and configure Continue to use the ollama provider pointing to http://127.0.0.1:11434.

Continue supports fully local LLM inference through direct integration with Ollama, an open-source tool for running large language models on your own hardware. This setup routes all code completions and chat requests through a local HTTP server instead of cloud APIs, ensuring complete data privacy and zero latency from network overhead. The integration leverages Continue's native Ollama adapter to translate between Continue's internal message format and Ollama's OpenAI-compatible API surface.

Architecture Overview

The self-hosted workflow consists of three integrated components:

1. Ollama Server – A lightweight daemon installed via the official installer at ollama.ai/download that serves model files from a local cache. It listens on http://127.0.0.1:11434 by default and exposes endpoints including /api/chat, /api/show, and /api/list.

2. Continue's Ollama Adapter – Located in core/llm/llms/Ollama.ts, the Ollama class extends BaseLLM to translate Continue's ChatMessage objects into Ollama-compatible payloads. This adapter handles model installation checks, error recovery, and feature detection such as tool support.

3. Configuration Interface – The AddNewModel page in gui/src/pages/AddNewModel/configs/providers.ts renders Ollama-specific setup instructions, while the underlying configuration stores connection details in config.json.

Prerequisites and Installation

Before configuring Continue, install and initialize the Ollama server on your machine.

# Install Ollama (macOS/Linux)

curl -fsSL https://ollama.ai/install.sh | sh

# Pull a code-optimized model (e.g., CodeLlama 7B)

ollama pull codellama:7b-instruct

# Verify the server is running

ollama serve &

The server must remain accessible at http://127.0.0.1:11434 (or your custom baseUrl) for Continue to establish connectivity.

Configuration Methods

You can configure Ollama either through Continue's graphical interface or by manually editing the configuration file.

Using config.json

Add an Ollama entry to your config.json with the ollama provider. The apiBase field defaults to http://127.0.0.1:11434 if omitted.

{
  "models": [
    {
      "provider": "ollama",
      "model": "codellama:7b-instruct",
      "apiBase": "http://127.0.0.1:11434"
    }
  ]
}

The model string must exactly match the name used in the Ollama CLI (e.g., codellama:7b-instruct, not just codellama).

Using the GUI

When adding a model through the Continue UI, the AddNewModel page displays an Ollama-specific help block that guides you through downloading Ollama, running a model, and setting the model name. This interface writes the same JSON configuration to config.json automatically.

Starting the Ollama Server

Continue offers multiple pathways to ensure the Ollama daemon is running.

Command Line Setup

Start the server manually before launching Continue:

ollama serve

This command binds to port 11434 and loads any previously pulled models into memory on first request.

Automatic Startup from Continue

If Continue detects that Ollama is not running, core/util/ollamaHelper.ts emits a friendly error message and offers a Start Ollama command. This triggers startLocalOllama() which executes:

• open -a Ollama.app on macOS
• ~/.config/ollama/start.sh on Linux

The VS Code extension exposes this through the continue.startLocalOllama command defined in extensions/vscode/src/commands.ts.

Key Implementation Details

Understanding the internal mechanics helps troubleshoot connection issues.

The Ollama Adapter Class

In core/llm/llms/Ollama.ts, the Ollama class extends BaseLLM to manage the communication protocol. It constructs payloads matching the OllamaChatOptions interface and sends them to /api/chat. Internally, the adapter can be instantiated directly as shown below, though most users interact with it through the configuration file:

import Ollama from "core/llm/llms/Ollama";

const ollama = new Ollama({
  model: "codellama:7b-instruct",
  apiBase: "http://127.0.0.1:11434",
});

const response = await ollama.chat({
  messages: [{ role: "user", content: "Explain the quicksort algorithm." }],
});

console.log(response.choices[0].message.content);

This class also maintains modelsBeingInstalled to prevent concurrent installation attempts of the same model.

Error Handling and Model Management

When a requested model is not cached locally, the adapter returns a specific error indicating the model is not found in the Ollama registry. The core/util/ollamaHelper.ts utility functions handle platform-specific path detection and startup script execution.

For tool calling, Continue first checks if the Ollama version supports native tool parsing. If not, it falls back to Continue's own parser implemented in gui/src/util/errorAnalysis.ts, ensuring feature compatibility across Ollama versions.

Summary

• Install Ollama via the official script and pull models using ollama pull <model>.
• Configure Continue by adding an ollama provider entry in config.json with the correct model name and apiBase.
• Start the server using ollama serve or trigger automatic startup through Continue's UI when prompted.
• Verify connectivity by checking that http://127.0.0.1:11434 responds before launching Continue features.

Frequently Asked Questions

How do I verify that Continue is actually using my local Ollama instance?

Check that requests appear in the Ollama server logs when you send a message in Continue. The adapter in core/llm/llms/Ollama.ts sends POST requests to /api/chat, which Ollama logs to stdout. If you see inference requests logging locally rather than network activity to external APIs, Continue is routing through your self-hosted model.

Can I use a remote Ollama server instead of localhost?

Yes. Change the apiBase field in your config.json from http://127.0.0.1:11434 to your remote server's URL (e.g., http://192.168.1.100:11434). Ensure the remote server is accessible and that any firewalls allow traffic on port 11434.

What happens if I request a model I haven't downloaded yet?

Continue will display an error stating the model is not found in the Ollama registry. You must first run ollama pull <model_name> from the command line, or use the Ollama CLI to download the model files before Continue can initialize the chat session. The Ollama.modelsBeingInstalled set prevents duplicate pull attempts.

Does Continue support tool calling with Ollama models?

Yes, but with a fallback mechanism. Continue attempts to use Ollama's native tool-call parser first. If the local Ollama version does not support tools (or returns a parsing error), Continue automatically falls back to its own tool parser in gui/src/util/errorAnalysis.ts, allowing tool use even on older Ollama versions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}