# How to Configure Backend-Specific Native Function Calling in Forge

> Learn to configure backend-specific native function calling in Forge. Set mode native, use the --jinja flag for Llamafile, and pass ToolSpec objects for seamless integration.

- Repository: [Antoine/forge](https://github.com/antoinezambelli/forge)
- Tags: how-to-guide
- Published: 2026-05-22

---

**Configure backend-specific native function calling in Forge by setting `mode="native"` when initializing Ollama or Llamafile clients, ensuring Llamafile servers start with the `--jinja` flag, and passing `ToolSpec` objects to the `send()` method.**

Forge simplifies structured tool usage by supporting *native* function calling on backends that expose a **tools** API. This approach lets models return structured JSON tool calls rather than parsing plain text. The framework currently supports native function calling on **Ollama** and **Llamafile** backends, while falling back to prompt-injected techniques for other servers.

## Supported Backends and Native Capabilities

Forge distinguishes between backends that support native function calling and those that rely on prompt engineering.

**Native-capable backends** expose a formal `tools` field in their API:
- **Ollama**: Uses the `/api/chat` endpoint with a `tools` array; no special server flags required.
- **Llamafile**: Requires the `--jinja` flag to enable the native template that understands tool schemas.

**Prompt-only backends**:
- **Llama-Server (default)**: Does not support native function calling; Forge automatically injects tool descriptions into the system prompt when using `mode="prompt"` or `mode="auto"`.

In [`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py), the `ServerManager` class handles backend initialization, while specific client implementations in [`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py) and [`src/forge/clients/llamafile.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/llamafile.py) manage the native request formatting.

## Step-by-Step Configuration Workflow

Configuring native function calling follows a six-stage pipeline from server initialization to response parsing.

**1. Select the backend via `ServerManager`**
Instantiate the manager with your target backend. In [`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py), the `ServerManager.__init__` method accepts `backend="ollama"` or `backend="llamafile"`.

**2. Start the server in native mode**
For Llamafile, the `ServerManager.start` method automatically appends `--jinja` to the startup flags when `mode="native"` is specified. Ollama servers do not require additional flags for native support.

**3. Initialize the client with mode configuration**
Create an `OllamaClient` or `LlamafileClient` with `mode="native"`. In [`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py), the `send` method checks this mode to determine whether to include the `tools` field in the request body. Similarly, `LlamafileClient._send_native` in [`src/forge/clients/llamafile.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/llamafile.py) constructs the native payload.

**4. Define tool specifications**
Create `ToolSpec` objects describing your functions. The `format_tool` helper in [`src/forge/clients/base.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/base.py) serializes these into the JSON schema required by the backend.

**5. Execute the request**
Pass your tools to `client.send()`. The client transmits the request and parses the backend's structured response into `ToolCall` objects (or `TextResponse` if no tool is invoked).

**6. Handle auto-fallback (optional)**
When using `mode="auto"`, `LlamafileClient._resolve_and_send` probes the server with a native request first. If the backend returns a 400 error indicating tool calling is unsupported, the client automatically retries in prompt-injected mode.

## Backend-Specific Configuration Requirements

Each backend requires distinct configuration to enable native function calling.

### Ollama Configuration

Ollama supports native tools without server-side flags. Configure the client explicitly:

```python
from forge.clients.ollama import OllamaClient
from forge.core.workflow import ToolSpec

client = OllamaClient(
    model="mistral",
    base_url="http://localhost:11434",
    mode="native",  # Explicit native mode

    think=True,     # Optional: enable reasoning blocks

)

```

The `OllamaClient.send` method (lines 50-92 in [`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py)) constructs the request with a `tools` array and parses the `tool_calls` field from the JSON response.

### Llamafile Configuration

Llamafile requires the `--jinja` flag for native template support. Use `ServerManager` to ensure correct startup:

```python
from forge.server import ServerManager

sm = ServerManager(backend="llamafile", port=8080)
await sm.start(
    model="llama3",
    gguf_path="/models/llama3.gguf",
    mode="native",  # Enables --jinja flag automatically

)

```

The `ServerManager.start` method in [`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py) (lines 72-78) checks for `mode="native"` and appends `--jinja` to the command arguments. Without this flag, the server cannot process the `tools` field.

### Auto-Fallback Configuration

For deployment flexibility, use `mode="auto"` to let the client detect capabilities:

```python
from forge.clients.llamafile import LlamafileClient

client = LlamafileClient(
    gguf_path="/models/llama3.gguf",
    mode="auto",  # Try native first, fall back to prompt

)

response = await client.send(messages=[...], tools=[...])
print("Resolved mode:", client.resolved_mode)  # "native" or "prompt"

```

As implemented in [`src/forge/clients/llamafile.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/llamafile.py) (lines 84-89), this mode catches HTTP errors indicating unsupported native features and transparently switches to prompt-injected tool calling.

## Implementing Tool Specifications

Native function calling requires structured tool definitions using the `ToolSpec` class from [`src/forge/core/workflow.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/workflow.py):

```python
from forge.core.workflow import ToolSpec

search_tool = ToolSpec(
    name="web_search",
    description="Search the internet for current information",
    parameters={
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": "The search query string"
            }
        },
        "required": ["query"]
    },
)

# Use with any native-enabled client

response = await client.send(
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=[search_tool],
)

```

The `format_tool` function in [`src/forge/clients/base.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/base.py) (lines 71-80) marshals these specifications into the exact JSON schema expected by Ollama and Llamafile APIs.

## Handling Reasoning Blocks and Error Recovery

Both native clients support a `think` parameter for reasoning models. When `think=True`, the client requests the backend to include chain-of-thought reasoning. If the backend rejects this (detected via `_is_think_unsupported_error` in [`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py)), the client automatically retries without the reasoning flag.

For Ollama, the error detection logic in lines 19-27 of [`src/forge/clients/ollama.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/ollama.py) identifies unsupported thinking errors, triggering the retry mechanism in lines 60-66.

## Summary

- **Forge supports native function calling on Ollama and Llamafile backends**, while Llama-Server requires prompt-injected tools.
- **Set `mode="native"`** when creating clients or starting servers to enable structured tool APIs.
- **Llamafile requires the `--jinja` flag**, which `ServerManager.start` automatically adds when `mode="native"` is specified.
- **Use `ToolSpec` objects** to define tool schemas; the `format_tool` helper ensures correct JSON formatting for each backend.
- **`mode="auto"` enables resilient deployments** by probing for native support and falling back to prompt mode if unavailable.

## Frequently Asked Questions

### What is the difference between native and prompt mode in Forge?

**Native mode** uses the backend's structured API to send tool schemas and receive parsed JSON tool calls, while **prompt mode** injects tool descriptions into the system prompt and parses the model's text output. Native mode is more reliable and available on Ollama and Llamafile backends, whereas prompt mode works universally but requires careful output parsing.

### Why does Llamafile require the `--jinja` flag for function calling?

The `--jinja` flag enables the server's native Jinja template engine, which understands how to format tool instructions and parse tool calls according to the ChatML format. Without this flag, Llamafile operates in standard completion mode and cannot process the `tools` field in API requests, as implemented in `ServerManager.start` in [`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py).

### How does the auto-fallback mechanism work when using `mode="auto"`?

When `mode="auto"` is set, `LlamafileClient._resolve_and_send` first attempts a native request with the `tools` field. If the server responds with a 400 error or message indicating tool calling is unsupported, the client catches this error, switches to `mode="prompt"`, and retries the request with injected tool descriptions, storing the resolved mode in `client.resolved_mode`.

### Can I use the same `ToolSpec` definitions across different backends?

Yes. The `ToolSpec` class in [`src/forge/core/workflow.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/core/workflow.py) provides a backend-agnostic way to define tools. The `format_tool` helper in [`src/forge/clients/base.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/clients/base.py) automatically converts these specifications into the specific JSON schema required by Ollama's `/api/chat` endpoint or Llamafile's native API, ensuring portability across supported backends.