# Context Budget Resolution in Forge: How Token Allocation Works Across Ollama, Llama-Server, and Llamafile Backends

> Understand context budget resolution in Forge. Discover how token allocation works across Ollama, Llama-Server, and Llamafile backends using backend discovery and BudgetMode strategies.

- Repository: [Antoine/forge](https://github.com/antoinezambelli/forge)
- Tags: how-to-guide
- Published: 2026-05-22

---

**Context budget resolution determines the maximum tokens a model can process per request by combining backend-specific discovery mechanisms with configurable allocation strategies defined in the `BudgetMode` enum.**

The `forge` framework abstracts LLM interaction across multiple local inference backends, but each handles context window sizing differently. Understanding how `forge` unifies these disparate approaches reveals how to optimize memory usage and inference speed for your specific hardware configuration.

## The Two Dimensions of Context Budget

Context budget resolution operates along two axes defined in **[`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py)**: the **Backend** dimension (`ollama`, `llamaserver`, `llamafile`) and the **BudgetMode** dimension (`BACKEND`, `MANUAL`, `FORGE_FULL`, `FORGE_FAST`).

The `BudgetMode` enum (lines 26-33) declares four distinct strategies:

- **`BACKEND`** – Respect the backend's native default or auto-tuned value
- **`MANUAL`** – Enforce a specific token limit provided by the caller
- **`FORGE_FULL`** – Utilize the maximum available context for the hardware
- **`FORGE_FAST`** – Trade half the available context for improved generation speed

The core resolution logic lives in `ServerManager.resolve_budget` (lines 52-90) and `ServerManager.start_with_budget` (lines 91-124), which orchestrate server startup and context negotiation.

## Ollama Backend Resolution

For Ollama backends, `forge` relies on GPU VRAM detection rather than server process management. The `_ollama_vram_tier_budget` function (lines 84-94) maps detected hardware to default budgets: **4,096** tokens for low VRAM, **32,768** for mid-range, and **262,144** for high-end GPUs (≥48GB).

**BudgetMode behavior for Ollama:**

- **`BACKEND` / `FORGE_FULL`** – Returns the full VRAM-tier budget (32,768 on a 24GB GPU). No server process is started.
- **`FORGE_FAST`** – Returns half the full budget (`full // 2`), reducing KV cache memory pressure.
- **`MANUAL`** – Returns the exact `manual_tokens` value specified by the caller, bypassing hardware detection.

## Llama-Server and Llamafile Backend Resolution

Unlike Ollama, these backends expose a `/props` endpoint reporting the actual context size configured via the `-c` flag. `resolve_budget` uses `get_server_context()` (lines 85-89) to read this value after server startup.

### Manual Mode

When `budget_mode=BudgetMode.MANUAL`, the server starts with `-c <manual_tokens>`. The method then queries `/props` to confirm the server accepted the override, returning that verified value.

### Forge Full Mode

`FORGE_FULL` starts the server **without** specifying `-c`, allowing the binary to auto-tune to the maximum context supported by the model and hardware. The resolved budget equals the value reported by `/props`.

### Forge Fast Mode

`FORGE_FAST` implements a two-phase startup dance (lines 91-124):

1. **Phase 1** – Start without `-c` and read the full context from `/props`
2. **Phase 2** – Calculate `half_total = full // 2` (adjusting for slots if applicable), restart with `-c half_total`, and verify via `/props`

This yields a budget trading maximum sequence length for reduced memory usage and faster token generation.

### Backend Mode

For these servers, `BACKEND` behaves identically to `FORGE_FULL`, using the auto-tuned full context.

## Handling Multi-Slot Configurations

When running multiple parallel slots (`n_slots > 1`), context calculation differs based on KV cache architecture:

**Non-unified KV cache (`kv_unified=False`)** – The `/props` endpoint reports context **per-slot**. `start_with_budget` multiplies this by `n_slots` when calculating the total budget for `FORGE_FAST` halving (lines 61-64).

**Unified KV cache (`kv_unified=True`)** – `/props` reports the **total** shared context across all slots; no multiplication occurs.

The integer returned by `start_with_budget` always reflects the budget passed to `ContextManager`, representing either per-slot (default) or total context depending on the `kv_unified` flag (docstring lines 15-20).

## Integration with setup_backend

The high-level `setup_backend` function orchestrates the complete resolution flow:

1. Instantiates `ServerManager` with the selected backend
2. Calls `start_with_budget` to resolve the budget according to the rules above
3. For Ollama, wires the budget into the client via `client.set_num_ctx(budget)`
4. Constructs a `ContextManager` with `budget_tokens=budget` (docstring lines 58-81)

This provides a unified interface where downstream code receives a validated integer representing available tokens, regardless of backend complexity.

## Practical Examples

Resolve a full budget on Ollama using auto-detected VRAM tiers:

```python
from forge import BudgetMode, setup_backend
from forge.clients.ollama import OllamaClient

client = OllamaClient(model="llama3")
server, ctx = await setup_backend(
    backend="ollama",
    model="llama3",
    budget_mode=BudgetMode.FORGE_FULL,
    client=client,
)
print(f"Ollama full budget: {ctx.budget_tokens} tokens")

# Output: 32768 (on 24GB GPU) or 262144 (on 48GB+ GPU)

```

Force a specific context size on llama-server using manual mode:

```python
from forge import BudgetMode, setup_backend

server, ctx = await setup_backend(
    backend="llamaserver",
    gguf_path="/models/llama3.gguf",
    budget_mode=BudgetMode.MANUAL,
    manual_tokens=8192,
)
print(f"Manual budget: {ctx.budget_tokens} tokens")

# Output: 8192 (verified via /props endpoint)

```

Optimize for speed with multiple slots on llamafile:

```python
from forge import BudgetMode, setup_backend

server, ctx = await setup_backend(
    backend="llamafile",
    gguf_path="/models/phi3.gguf",
    n_slots=4,
    kv_unified=False,
    budget_mode=BudgetMode.FORGE_FAST,
)
print(f"Fast budget per slot: {ctx.budget_tokens} tokens")

# Calculation: (full_context * 4 slots) // 2

```

Use unified KV cache for shared context across slots:

```python
from forge import BudgetMode, setup_backend

server, ctx = await setup_backend(
    backend="llamafile",
    gguf_path="/models/phi3.gguf",
    n_slots=4,
    kv_unified=True,
    budget_mode=BudgetMode.FORGE_FAST,
)
print(f"Fast total budget: {ctx.budget_tokens} tokens")

# Output: half of total shared context from /props

```

## Summary

- **Context budget resolution** in `forge` combines backend-specific discovery (`ollama` via VRAM tiers, `llamaserver`/`llamafile` via `/props`) with four allocation strategies defined in `BudgetMode`.
- **Ollama** relies on `_ollama_vram_tier_budget` to map GPU memory to preset token limits (4096, 32768, or 262144), supporting full, fast (half), or manual modes.
- **Llama-server and llamafile** execute a startup-validation cycle where `start_with_budget` starts the process, queries `/props` via `get_server_context()`, and optionally restarts with adjusted `-c` flags for `FORGE_FAST` optimization.
- **Multi-slot inference** requires adjusting calculations based on `kv_unified`: non-unified configurations multiply per-slot context by `n_slots` before halving for fast mode, while unified configurations use the total directly.
- All resolution paths converge in `setup_backend`, which returns a `ContextManager` configured with a verified integer budget regardless of backend complexity.

## Frequently Asked Questions

### How does `FORGE_FAST` mode improve inference performance?

`FORGE_FAST` reduces the KV cache memory footprint by allocating half the available context window (calculated as `full_context // 2`). According to the implementation in [`src/forge/server.py`](https://github.com/antoinezambelli/forge/blob/main/src/forge/server.py) (lines 91-124), this reduction decreases memory bandwidth pressure and cache management overhead, trading maximum sequence length for faster token generation speeds, particularly beneficial in multi-slot scenarios.

### Can I override the automatic context detection on Ollama backends?

Yes, by specifying `budget_mode=BudgetMode.MANUAL` and providing the `manual_tokens` parameter to `setup_backend`. While Ollama normally relies on `_ollama_vram_tier_budget` to automatically select 4096, 32768, or 262144 tokens based on detected GPU VRAM, manual mode bypasses this hardware detection and sets `num_ctx` directly on the client via `client.set_num_ctx()`.

### Why does context calculation differ between unified and non-unified KV caches?

In non-unified KV cache configurations (`kv_unified=False`), each parallel slot maintains separate KV memory, so `/props` reports context per-slot and `forge` multiplies this by `n_slots` to calculate total capacity before halving for `FORGE_FAST`. In unified configurations (`kv_unified=True`), all slots share a single KV cache, meaning `/props` already reports the total shared context, requiring no multiplication adjustment during budget resolution (lines 61-64).

### What happens if the server rejects my manual context size?

When using `BudgetMode.MANUAL` with `llamaserver` or `llamafile` backends, `start_with_budget` starts the server with `-c <manual_tokens>` and subsequently queries `/props` via `get_server_context()` to verify the actual allocated context. If the server binary caps the value below your request due to model or hardware limitations, the resolved budget reflects the server's reported maximum rather than the requested value, ensuring `ContextManager` never assumes more tokens than physically available.