# How to Integrate Forge with an Ollama Backend: Complete Configuration Guide

> Integrate Forge with Ollama using the OllamaClient for seamless synchronous and streaming inference. Configure your project effortlessly with this comprehensive guide.

- Repository: [Antoine/forge](https://github.com/antoinezambelli/forge)
- Tags: how-to-guide
- Published: 2026-05-22

---

**Forge integrates with Ollama by instantiating an `OllamaClient` that implements the standard `LLMClient` interface, enabling both synchronous and streaming inference with automatic context window management.**

Forge treats Ollama as a first-class LLM backend, allowing you to run local models through a unified workflow interface. Whether you use the command-line interface or embed the client programmatically, the integration handles model-specific sampling defaults, hardware-aware context sizing, and optional reasoning modes automatically.

## Configuring the Ollama Client

The `OllamaClient` class in [`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py) provides the primary interface for Ollama integration. It accepts the model identifier, endpoint URL, sampling parameters, and specialized flags for reasoning modes.

### Client Construction and Parameters

Instantiate the client with your target model and Ollama server address:

```python
from forge.clients.ollama import OllamaClient

client = OllamaClient(
    model="llama3:8b",
    base_url="http://localhost:11434",
    temperature=0.7,
    think=True,
    timeout=300
)

```

The constructor automatically loads per-model sampling defaults via `apply_sampling_defaults` from `forge.clients.sampling_defaults`, ensuring optimal parameters for specific models.

### Think Mode Auto-Detection

The client automatically enables thinking mode for models containing "reason" or "think" in their names unless explicitly disabled. According to lines 36-40 in [`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py), attempting to force thinking on incompatible models raises `ThinkingNotSupportedError`.

## Backend Selection and Server Integration

### CLI Configuration

When launching a Forge server, specify Ollama as the backend using the `--backend` flag. In [`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py) (lines 312-332), the `setup_backend` function detects this flag and initializes the `OllamaClient`:

```bash
forge run \
  --backend ollama \
  --model mistral:7b \
  --think auto

```

The server builder wires the client into the workflow runner, making it available for all inference requests.

### Hardware-Aware Context Management

Ollama limits context windows based on available GPU VRAM. Forge's hardware manager detects your GPU tier and automatically adjusts the `num_ctx` parameter. In [`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py) (lines 483-492), the system calls `client.set_num_ctx` to optimize the token budget without manual configuration.

## Streaming and Tool Calling

The `OllamaClient` supports both synchronous and asynchronous request patterns through the `send` and `send_stream` methods. These methods forward requests to Ollama's `/api/chat` endpoint, attaching tool specifications via the `tools` field when function calling is required.

In [`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py) (lines 137-156), the client parses the NDJSON response stream, extracts token usage statistics, and returns either an `LLMResponse` or specialized `TextResponse`/`ToolCall` objects for tool-enabled conversations.

## Programmatic Integration Example

For custom applications, manually inject the client into a `WorkflowRunner` from [`src/forge/core/runner.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/runner.py):

```python
from forge.clients.ollama import OllamaClient
from forge.core.runner import WorkflowRunner
from forge.core.workflow import Workflow

# Initialize the Ollama backend

client = OllamaClient(
    model="llama3:8b",
    base_url="http://localhost:11434",
    think=True
)

# Define your workflow

workflow = Workflow(
    messages=[{"role": "user", "content": "Explain quantum entanglement"}]
)

# Execute with streaming support

runner = WorkflowRunner(client=client, workflow=workflow)
result = await runner.run()
print(result.text)

```

This pattern gives you direct control over the model configuration while leveraging Forge's workflow orchestration.

## Summary

- **Client Location**: The `OllamaClient` class in [`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py) implements the standard interface for Ollama communication, handling model parameters and reasoning modes.
- **CLI Integration**: Use `--backend ollama` with `forge run` to automatically configure the client via `setup_backend` in [`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py).
- **Context Optimization**: Forge automatically adjusts `num_ctx` based on GPU VRAM tier detection (lines 483-492) to prevent OOM errors.
- **Streaming Support**: Both `send` and `send_stream` methods handle NDJSON responses from the `/api/chat` endpoint, supporting tool calls via `ToolCall` objects and token tracking.
- **Reasoning Modes**: The `think` parameter auto-detects reasoning model capabilities (lines 36-40), with automatic fallback for unsupported models.

## Frequently Asked Questions

### What is the default Ollama endpoint URL?

By default, the `OllamaClient` connects to `http://localhost:11434`. You can override this by passing the `base_url` parameter during client initialization.

### How does Forge handle Ollama models with limited VRAM?

Forge's hardware manager detects your GPU's VRAM tier in [`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py) (lines 483-492) and automatically calls `client.set_num_ctx` to adjust the context window. This prevents out-of-memory errors while maximizing available token budget for your specific hardware configuration.

### Can I use tool calling with Ollama in Forge?

Yes. The `OllamaClient.send` and `OllamaClient.send_stream` methods support tool specifications via the `tools` field in the request payload. Responses are parsed into `ToolCall` objects when the model generates function calls, allowing seamless integration with Forge's workflow system.

### How do I disable thinking mode for reasoning models?

Pass `think=False` when constructing the `OllamaClient`. If you use the CLI, omit the `--think` flag or set it to `false`. The client only auto-enables thinking for models with "reason" or "think" in their names when the parameter is left in its default `auto` state.