How to Troubleshoot Validation Failures in Forge Proxy Mode

When the Forge proxy rejects an LLM response, the failure occurs within the guardrails pipeline at checkpoints including response validation, retry budget exhaustion, or tool-spec mismatches.

The antoinezambelli/forge repository implements a proxy server that intercepts chat-completion requests to enforce guardrails on tool-calling LLMs. When you encounter validation failures in proxy mode, understanding the specific checkpoint where the request fails is essential for resolution. This guide breaks down the validation pipeline, identifies common failure points in the source code, and provides debugging strategies based on the actual implementation.

Understanding the Guardrails Validation Pipeline

The Forge proxy processes every chat-completion request through a structured validation pipeline before returning results to the caller. In proxy/handler.py, the handle_chat_completions method orchestrates this flow by initializing a ResponseValidator and ErrorTracker to monitor the LLM output.

The validation sequence follows this path:

1. Request preprocessing – The handler extracts tool specifications from the OpenAI-style request via _extract_tool_specs (lines 50-66) and builds an internal tool_names list (lines 70-72).
2. Response examination – The ResponseValidator.validate method (lines 64-94 of guardrails/response_validator.py) inspects the raw LLM output.
3. Retry accounting – The Guardrails.check method uses ErrorTracker (lines 19-26 of guardrails/guardrails.py) to count consecutive failures against the max_retries budget.
4. Result return – The handler returns either the validated response or a nudge message instructing the model to correct its output.

When validation fails, the proxy generates specific error types: a retry nudge for unparseable text responses, an unknown-tool nudge for invalid tool calls, or a fatal error when the retry budget is exhausted.

Common Causes of Validation Failures

Response Validation and Rescue Failures

The ResponseValidator distinguishes between plain text responses and structured ToolCall objects. When the LLM returns plain text instead of a tool call, the validator attempts to rescue a tool call using the rescue_tool_call function from prompts/templates.py (lines 64-71).

If rescue parsing fails, the validator generates a retry nudge (lines 72-79). This commonly occurs when:

• The model outputs malformed JSON or natural language instead of the expected tool syntax
• The rescue_enabled parameter is set to False (default is True)
• The text pattern does not match any known tool invocation format

Unknown Tool Errors

When the LLM returns a list of ToolCall objects, the validator verifies that every called tool exists in the allowed tool_names list (lines 81-94 of response_validator.py). An unknown-tool nudge triggers when:

• The model calls a function not present in the request's tools array
• The tool name is misspelled or uses different casing than defined in _extract_tool_names
• The proxy's automatic respond tool injection fails (lines 108-114 of handler.py)

The proxy automatically injects the internal respond tool when any tools are declared, so missing injection indicates a handler configuration issue.

Retry Budget Exhaustion

The ErrorTracker class (lines 19-22 of error_tracker.py) maintains a counter of consecutive validation failures. After max_retries failures (default 3, configurable via the handler), the guardrails return a fatal result rather than attempting another retry. This failure mode appears in logs as "Retries exhausted" (lines 45-50).

Bypassed Guardrails

If the request contains no tools in the tool_specs array (lines 19-20 of handler.py), the proxy bypasses validation entirely. This explains why some requests succeed without guardrail checks while others trigger validation—guardrails only activate when tool_specs is non-empty.

Step-by-Step Debugging Guide

Follow these systematic steps to identify the root cause of validation failures:

1. Enable debug logging – Set the logger to DEBUG level for forge.proxy and forge.guardrails. The code logs critical path decisions such as "No tools in request, passing through to backend" (line 19 of handler.py) and retry exhaustion warnings.

2. Inspect tool name alignment – Verify that the LLM's output references exactly the names extracted by _extract_tool_names. The validator checks against this specific list, so any deviation triggers an unknown-tool error.

3. Test rescue behavior independently – Import rescue_tool_call from prompts/templates.py and test it with your raw text responses to determine if rescue parsing is viable for your use case.

4. Validate tool specification extraction – Check that _extract_tool_specs (lines 50-66) correctly parses your request's OpenAI-style tool definitions into the internal format expected by the validator.

5. Adjust retry thresholds – Modify the max_retries parameter when constructing the ProxyServer if the LLM is noisy (increase to 5) or if you want faster failure (decrease to 1).

6. Run unit tests – Execute tests/unit/test_response_validator.py and tests/unit/test_proxy_handler.py to reproduce failure paths and compare against expected log output.

Code Examples

Reproducing an Unknown Tool Validation Failure

This example demonstrates how to trigger a validation error by mismatching the available tools with the model's intentions:

from forge.proxy import ProxyServer
import requests

# Initialize proxy pointing to a local backend

proxy = ProxyServer(backend_url="http://localhost:11434")
proxy.start()

# Send request with limited tools while expecting a weather call

payload = {
    "model": "llama3",
    "messages": [{"role": "user", "content": "Give me the weather"}],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "search",
                "description": "Search the web",
                "parameters": {}
            }
        }
    ],
}

resp = requests.post(
    f"{proxy.url}/v1/chat/completions", 
    json=payload
).json()

# If the model attempts to call "get_weather", validation fails

print(resp)  # Contains "unknown_tool" nudge

proxy.stop()

Configuring Retry Limits and Rescue Behavior

Increase resilience against noisy models by adjusting the guardrails configuration:

from forge.proxy import ProxyServer

proxy = ProxyServer(
    backend_url="http://localhost:11434",
    max_retries=5,          # Default is 3; increase for unstable models

    rescue_enabled=True,    # Attempt to parse tool calls from text

)

Direct Validator Inspection

Test the validation logic independently without running the full proxy:

from forge.guardrails import ResponseValidator, ValidationResult
from forge.core.workflow import TextResponse

validator = ResponseValidator(
    tool_names=["search", "calculate"], 
    rescue_enabled=True
)

# Simulate a text response that should be rescued as a search call

text_response = TextResponse(content="search(query='London weather')")
result: ValidationResult = validator.validate(text_response)

print(result.tool_calls)   # Recovered ToolCall object if rescue succeeds

print(result.is_valid)     # Boolean indicating validation status

Key Files and Implementation Details

Component	File Path	Key Functionality
Request Handler	proxy/handler.py	Entry point at handle_chat_completions; manages ResponseValidator initialization and respond tool injection (lines 108-114)
Response Validator	guardrails/response_validator.py	Core validation at validate method; handles rescue logic (lines 64-71) and tool name verification (lines 81-94)
Guardrails Orchestration	guardrails/guardrails.py	Guardrails.check coordinates validation with retry budgeting via ErrorTracker (lines 19-26)
Error Tracking	guardrails/error_tracker.py	ErrorTracker class monitors consecutive failures; check retries_exhausted property (lines 19-22)
Rescue Templates	prompts/templates.py	Contains rescue_tool_call function for parsing tool calls from plain text
Unit Tests	tests/unit/test_response_validator.py	Exercises failure paths including unknown tools and rescue scenarios
Handler Tests	tests/unit/test_proxy_handler.py	Validates proxy integration and logging output

Summary

• Validation failures in Forge proxy mode occur at three checkpoints: response parsing (including rescue attempts), tool name verification, and retry budget exhaustion.
• Unknown-tool errors indicate the LLM called a function not present in the tool_names list extracted from the request's tools array.
• Retry nudges appear when the validator cannot rescue a plain-text response as a valid tool call, or when rescue is disabled.
• Fatal errors occur after max_retries consecutive failures, tracked by ErrorTracker in guardrails/error_tracker.py.
• Guardrails bypass requests that contain no tools, explaining why some calls skip validation entirely.
• Debug via logs by enabling DEBUG level for forge.proxy and inspecting the specific checkpoint where validation fails.

Frequently Asked Questions

What causes "unknown_tool" validation errors in Forge proxy mode?

The unknown-tool error occurs in ResponseValidator.validate (lines 81-94 of guardrails/response_validator.py) when the LLM attempts to call a tool not present in the request's allowed tool_names list. This happens if the model hallucinates a function name, misspells a valid tool, or if the tools array in your request is incomplete. Verify that _extract_tool_names (lines 70-72 of handler.py) correctly parsed your OpenAI-style tool definitions and that the model references these exact names.

How does the rescue mechanism work for text responses?

When the LLM returns plain text instead of a structured tool call, the ResponseValidator attempts rescue parsing via rescue_tool_call in prompts/templates.py (lines 64-71). If the text contains a recognizable tool invocation pattern, the validator extracts it into a proper ToolCall object. If rescue fails or is disabled (rescue_enabled=False), the validator returns a retry nudge (lines 72-79) asking the model to format its response correctly.

Where can I adjust the retry limits for validation failures?

The retry budget is controlled by the max_retries parameter passed to the ProxyServer constructor in proxy/handler.py (line 31). The default value is 3, but you can increase it to 5 or higher for noisy models that require multiple attempts, or reduce it to 1 for faster failure. The ErrorTracker class (lines 19-22 of error_tracker.py) monitors these attempts and sets retries_exhausted when the limit is reached.

Why are guardrails bypassed for some requests?

The proxy validates only requests that contain tools. In handler.py (lines 19-20), the code checks if tool_specs is empty; if so, it passes the request directly to the backend without instantiating the ResponseValidator. This optimization prevents unnecessary overhead for simple chat requests. To force validation on all requests, ensure your payload includes at least one tool definition, which automatically triggers the internal respond tool injection (lines 108-114).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}