How to Configure Budget Modes in Forge: Backend, Manual, Forge-Full, and Forge-Fast Explained

Forge provides four distinct budget modes that determine how the maximum token context is allocated, from automatic backend detection to manually specified limits or performance-optimized halved contexts.

Forge determines how much context (in tokens) a model may use through configurable budget modes defined in the BudgetMode enum at src/forge/server.py (lines 26-33). The mode is selected once when the backend starts or when a proxy connects to an external server, and it directly impacts memory usage and inference latency. Understanding these modes allows you to balance between maximum context length and computational performance.

Understanding Forge Budget Modes

Budget modes control the context window resolution strategy used by Forge's ServerManager. When you initialize a proxy or start a managed backend, Forge must decide how many tokens to allocate for the model's context. The BudgetMode enum defines four options—BACKEND, MANUAL, FORGE_FULL, and FORGE_FAST—each implementing a different approach to resolving this budget.

The resolution logic lives in ServerManager.resolve_budget and ServerManager.start_with_budget (lines 52-89 in src/forge/server.py). These methods handle the specific startup sequences required for each mode, including the two-phase restart process used by FORGE_FAST.

The Four Budget Mode Options

Each budget mode serves a distinct use case, from development flexibility to production performance optimization.

Backend Mode (BACKEND)

Backend mode lets the underlying inference engine determine the context size automatically. When using this mode, Forge does not pass an explicit -c flag to the server. Instead, it queries the backend's /props endpoint after startup to discover the default context size.

This is the default behavior for managed backends like llama-server or llamafile, making it ideal when you want the backend to self-configure based on available hardware.

Manual Mode (MANUAL)

Manual mode enforces an exact, user-specified token count. When you select this mode, you must provide the budget_tokens parameter. Forge starts the server with -c <budget_tokens> and uses that exact value as the budget.

Use this mode when you need deterministic, reproducible context lengths for experiments or when working with specific hardware constraints that require precise memory allocation.

Forge-Full Mode (FORGE_FULL)

Forge-full mode mirrors the behavior of backend mode but makes the intent explicit: use the maximum safe context for the current hardware tier. It resolves the budget exactly like BACKEND by reading from /props, but the naming clarifies that you intentionally want the largest possible context window without sacrificing safety guarantees.

This mode is preferred when you want to ensure you're utilizing the full capability of your GPU or RAM allocation.

Forge-Fast Mode (FORGE_FAST)

Forge-fast mode optimizes for inference speed by cutting the context window in half. According to the source code in src/forge/server.py (lines 50-73), this mode performs a two-phase startup:

1. First, it launches the server without a -c flag to discover the maximum possible context via /props
2. It computes half of that maximum (max // 2)
3. It restarts the server with the halved -c value
4. Returns the halved budget

This reduces attention computation time roughly by half while maintaining the same safety guarantees, making it ideal for latency-sensitive applications where you can trade context length for speed.

Implementation Details

The budget resolution logic centers on ServerManager methods in src/forge/server.py:

• resolve_budget: Determines the token count based on the selected mode
• start_with_budget: Handles the actual server startup sequence, including the conditional restart logic for FORGE_FAST

For MANUAL mode, the implementation either returns the user-provided token count (when using Ollama) or reads the server's actual -c value after startup (for llama-server/llamafile). The FORGE_FAST implementation specifically checks if budget_mode == BudgetMode.FORGE_FAST: to trigger the two-phase discovery and restart process.

When using the proxy (ProxyServer in src/forge/proxy/proxy.py, lines 53-55), you specify the mode via the budget_mode constructor argument and optionally budget_tokens for manual configuration.

Configuring Budget Modes via ProxyServer

When working with the high-level proxy API, configure your budget mode through the ProxyServer constructor:

from forge.proxy.proxy import ProxyServer, BudgetMode

# Backend mode: Let the backend decide (default behavior)

proxy = ProxyServer(
    backend="llamaserver",
    gguf="model.gguf",
    budget_mode=BudgetMode.BACKEND,
)
proxy.start()

# Manual mode: Fixed token budget of 4096 tokens

proxy = ProxyServer(
    backend="llamaserver",
    gguf="model.gguf",
    budget_mode=BudgetMode.MANUAL,
    budget_tokens=4096,
)
proxy.start()

# Forge-full mode: Explicitly request maximum safe context

proxy = ProxyServer(
    backend="llamaserver",
    gguf="model.gguf",
    budget_mode=BudgetMode.FORGE_FULL,
)
proxy.start()

# Forge-fast mode: Halve the maximum context for speed

proxy = ProxyServer(
    backend="llamaserver",
    gguf="model.gguf",
    budget_mode=BudgetMode.FORGE_FAST,
)
proxy.start()

Direct ServerManager Configuration

If you are not using the proxy layer and working directly with ServerManager from src/forge/server.py:

from forge.server import ServerManager, BudgetMode

mgr = ServerManager(backend="llamaserver", port=8080)

# Forge-fast: Will start, query max context, restart with half that size

budget = await mgr.start_with_budget(
    model="llama-2-7b",
    gguf_path="models/llama-2-7b.gguf",
    budget_mode=BudgetMode.FORGE_FAST,
)
print("Effective budget:", budget)

The resolved budget ultimately flows to ContextManager in src/forge/context/manager.py, where it influences compaction strategies and token allocation during inference.

Summary

• Budget modes control how Forge determines the maximum token context window at startup
• BACKEND uses the inference engine's default context size by querying /props
• MANUAL enforces a fixed token count via the budget_tokens parameter
• FORGE_FULL explicitly selects maximum safe context (equivalent to backend but clearer intent)
• FORGE_FAST halves the maximum context to reduce attention computation time by roughly 50%
• Configuration occurs through ProxyServer constructor arguments or directly via ServerManager.start_with_budget
• Implementation details are located in src/forge/server.py (lines 26-33 for the enum, lines 52-89 for resolution logic)

Frequently Asked Questions

What is the difference between backend and forge-full modes?

Both modes resolve to the same context size by reading the backend's /props endpoint, but FORGE_FULL makes the intent explicit in your code that you want the maximum safe context. According to the source code in src/forge/server.py, they follow identical resolution paths, but FORGE_FULL signals clearly to other developers that you intentionally chose full context utilization rather than simply accepting defaults.

How does forge-fast mode improve inference speed?

FORGE_FAST cuts the context budget to half of the hardware maximum (max // 2), which reduces the attention computation time roughly by half. As implemented in src/forge/server.py (lines 50-73), it starts the server once to discover the maximum, computes the halved value, then restarts with the reduced context size. This trade-off sacrifices maximum context length for significantly faster token generation.

Can I change the budget mode after starting the server?

No, the budget mode is selected once at initialization. Once ProxyServer or ServerManager starts the backend process, the context size is fixed for that session. To change modes, you must stop the current server instance and create a new one with the desired budget_mode parameter. This design ensures consistent memory allocation throughout the inference session.

Where does the resolved budget get consumed in the codebase?

After ServerManager.resolve_budget determines the token limit, the value flows to ContextManager in src/forge/context/manager.py. This component uses the budget to manage context compaction strategies, determining how many tokens to retain during sliding window operations and when to trigger context window management routines.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}