How to Integrate Forge with an Ollama Backend: Complete Configuration Guide

Forge integrates with Ollama by instantiating an OllamaClient that implements the standard LLMClient interface, enabling both synchronous and streaming inference with automatic context window management.

Forge treats Ollama as a first-class LLM backend, allowing you to run local models through a unified workflow interface. Whether you use the command-line interface or embed the client programmatically, the integration handles model-specific sampling defaults, hardware-aware context sizing, and optional reasoning modes automatically.

Configuring the Ollama Client

The OllamaClient class in src/forge/clients/ollama.py provides the primary interface for Ollama integration. It accepts the model identifier, endpoint URL, sampling parameters, and specialized flags for reasoning modes.

Client Construction and Parameters

Instantiate the client with your target model and Ollama server address:

from forge.clients.ollama import OllamaClient

client = OllamaClient(
    model="llama3:8b",
    base_url="http://localhost:11434",
    temperature=0.7,
    think=True,
    timeout=300
)

The constructor automatically loads per-model sampling defaults via apply_sampling_defaults from forge.clients.sampling_defaults, ensuring optimal parameters for specific models.

Think Mode Auto-Detection

The client automatically enables thinking mode for models containing "reason" or "think" in their names unless explicitly disabled. According to lines 36-40 in src/forge/clients/ollama.py, attempting to force thinking on incompatible models raises ThinkingNotSupportedError.

Backend Selection and Server Integration

CLI Configuration

When launching a Forge server, specify Ollama as the backend using the --backend flag. In src/forge/server.py (lines 312-332), the setup_backend function detects this flag and initializes the OllamaClient:

forge run \
  --backend ollama \
  --model mistral:7b \
  --think auto

The server builder wires the client into the workflow runner, making it available for all inference requests.

Hardware-Aware Context Management

Ollama limits context windows based on available GPU VRAM. Forge's hardware manager detects your GPU tier and automatically adjusts the num_ctx parameter. In src/forge/server.py (lines 483-492), the system calls client.set_num_ctx to optimize the token budget without manual configuration.

Streaming and Tool Calling

The OllamaClient supports both synchronous and asynchronous request patterns through the send and send_stream methods. These methods forward requests to Ollama's /api/chat endpoint, attaching tool specifications via the tools field when function calling is required.

In src/forge/clients/ollama.py (lines 137-156), the client parses the NDJSON response stream, extracts token usage statistics, and returns either an LLMResponse or specialized TextResponse/ToolCall objects for tool-enabled conversations.

Programmatic Integration Example

For custom applications, manually inject the client into a WorkflowRunner from src/forge/core/runner.py:

from forge.clients.ollama import OllamaClient
from forge.core.runner import WorkflowRunner
from forge.core.workflow import Workflow

# Initialize the Ollama backend

client = OllamaClient(
    model="llama3:8b",
    base_url="http://localhost:11434",
    think=True
)

# Define your workflow

workflow = Workflow(
    messages=[{"role": "user", "content": "Explain quantum entanglement"}]
)

# Execute with streaming support

runner = WorkflowRunner(client=client, workflow=workflow)
result = await runner.run()
print(result.text)

This pattern gives you direct control over the model configuration while leveraging Forge's workflow orchestration.

Summary

• Client Location: The OllamaClient class in src/forge/clients/ollama.py implements the standard interface for Ollama communication, handling model parameters and reasoning modes.
• CLI Integration: Use --backend ollama with forge run to automatically configure the client via setup_backend in src/forge/server.py.
• Context Optimization: Forge automatically adjusts num_ctx based on GPU VRAM tier detection (lines 483-492) to prevent OOM errors.
• Streaming Support: Both send and send_stream methods handle NDJSON responses from the /api/chat endpoint, supporting tool calls via ToolCall objects and token tracking.
• Reasoning Modes: The think parameter auto-detects reasoning model capabilities (lines 36-40), with automatic fallback for unsupported models.

Frequently Asked Questions

What is the default Ollama endpoint URL?

By default, the OllamaClient connects to http://localhost:11434. You can override this by passing the base_url parameter during client initialization.

How does Forge handle Ollama models with limited VRAM?

Forge's hardware manager detects your GPU's VRAM tier in src/forge/server.py (lines 483-492) and automatically calls client.set_num_ctx to adjust the context window. This prevents out-of-memory errors while maximizing available token budget for your specific hardware configuration.

Can I use tool calling with Ollama in Forge?

Yes. The OllamaClient.send and OllamaClient.send_stream methods support tool specifications via the tools field in the request payload. Responses are parsed into ToolCall objects when the model generates function calls, allowing seamless integration with Forge's workflow system.

How do I disable thinking mode for reasoning models?

Pass think=False when constructing the OllamaClient. If you use the CLI, omit the --think flag or set it to false. The client only auto-enables thinking for models with "reason" or "think" in their names when the parameter is left in its default auto state.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}