# How to Troubleshoot Validation Failures in Forge Proxy Mode

> Troubleshoot validation failures in Forge proxy mode by identifying errors in response validation, retry budget, or tool specs. Learn how to fix LLM rejections.

- Repository: [Antoine/forge](https://github.com/antoinezambelli/forge)
- Tags: how-to-guide
- Published: 2026-05-22

---

**When the Forge proxy rejects an LLM response, the failure occurs within the guardrails pipeline at checkpoints including response validation, retry budget exhaustion, or tool-spec mismatches.**

The `antoinezambelli/forge` repository implements a proxy server that intercepts chat-completion requests to enforce guardrails on tool-calling LLMs. When you encounter validation failures in proxy mode, understanding the specific checkpoint where the request fails is essential for resolution. This guide breaks down the validation pipeline, identifies common failure points in the source code, and provides debugging strategies based on the actual implementation.

## Understanding the Guardrails Validation Pipeline

The Forge proxy processes every chat-completion request through a structured validation pipeline before returning results to the caller. In [`proxy/handler.py`](https://github.com/antoinezambelli/forge/blob/main/proxy/handler.py), the `handle_chat_completions` method orchestrates this flow by initializing a `ResponseValidator` and `ErrorTracker` to monitor the LLM output.

The validation sequence follows this path:

1. **Request preprocessing** – The handler extracts tool specifications from the OpenAI-style request via `_extract_tool_specs` (lines 50-66) and builds an internal `tool_names` list (lines 70-72).
2. **Response examination** – The `ResponseValidator.validate` method (lines 64-94 of [`guardrails/response_validator.py`](https://github.com/antoinezambelli/forge/blob/main/guardrails/response_validator.py)) inspects the raw LLM output.
3. **Retry accounting** – The `Guardrails.check` method uses `ErrorTracker` (lines 19-26 of [`guardrails/guardrails.py`](https://github.com/antoinezambelli/forge/blob/main/guardrails/guardrails.py)) to count consecutive failures against the `max_retries` budget.
4. **Result return** – The handler returns either the validated response or a nudge message instructing the model to correct its output.

When validation fails, the proxy generates specific error types: a *retry* nudge for unparseable text responses, an *unknown-tool* nudge for invalid tool calls, or a *fatal* error when the retry budget is exhausted.

## Common Causes of Validation Failures

### Response Validation and Rescue Failures

The `ResponseValidator` distinguishes between plain text responses and structured `ToolCall` objects. When the LLM returns plain text instead of a tool call, the validator attempts to *rescue* a tool call using the `rescue_tool_call` function from [`prompts/templates.py`](https://github.com/antoinezambelli/forge/blob/main/prompts/templates.py) (lines 64-71).

If rescue parsing fails, the validator generates a retry nudge (lines 72-79). This commonly occurs when:
- The model outputs malformed JSON or natural language instead of the expected tool syntax
- The `rescue_enabled` parameter is set to `False` (default is `True`)
- The text pattern does not match any known tool invocation format

### Unknown Tool Errors

When the LLM returns a list of `ToolCall` objects, the validator verifies that every called tool exists in the allowed `tool_names` list (lines 81-94 of [`response_validator.py`](https://github.com/antoinezambelli/forge/blob/main/response_validator.py)). An **unknown-tool** nudge triggers when:
- The model calls a function not present in the request's `tools` array
- The tool name is misspelled or uses different casing than defined in `_extract_tool_names`
- The proxy's automatic `respond` tool injection fails (lines 108-114 of [`handler.py`](https://github.com/antoinezambelli/forge/blob/main/handler.py))

The proxy automatically injects the internal `respond` tool when any tools are declared, so missing injection indicates a handler configuration issue.

### Retry Budget Exhaustion

The `ErrorTracker` class (lines 19-22 of [`error_tracker.py`](https://github.com/antoinezambelli/forge/blob/main/error_tracker.py)) maintains a counter of consecutive validation failures. After `max_retries` failures (default 3, configurable via the handler), the guardrails return a **fatal** result rather than attempting another retry. This failure mode appears in logs as "Retries exhausted" (lines 45-50).

### Bypassed Guardrails

If the request contains no tools in the `tool_specs` array (lines 19-20 of [`handler.py`](https://github.com/antoinezambelli/forge/blob/main/handler.py)), the proxy bypasses validation entirely. This explains why some requests succeed without guardrail checks while others trigger validation—guardrails only activate when `tool_specs` is non-empty.

## Step-by-Step Debugging Guide

Follow these systematic steps to identify the root cause of validation failures:

1. **Enable debug logging** – Set the logger to `DEBUG` level for `forge.proxy` and `forge.guardrails`. The code logs critical path decisions such as "No tools in request, passing through to backend" (line 19 of [`handler.py`](https://github.com/antoinezambelli/forge/blob/main/handler.py)) and retry exhaustion warnings.

2. **Inspect tool name alignment** – Verify that the LLM's output references exactly the names extracted by `_extract_tool_names`. The validator checks against this specific list, so any deviation triggers an unknown-tool error.

3. **Test rescue behavior independently** – Import `rescue_tool_call` from [`prompts/templates.py`](https://github.com/antoinezambelli/forge/blob/main/prompts/templates.py) and test it with your raw text responses to determine if rescue parsing is viable for your use case.

4. **Validate tool specification extraction** – Check that `_extract_tool_specs` (lines 50-66) correctly parses your request's OpenAI-style tool definitions into the internal format expected by the validator.

5. **Adjust retry thresholds** – Modify the `max_retries` parameter when constructing the `ProxyServer` if the LLM is noisy (increase to 5) or if you want faster failure (decrease to 1).

6. **Run unit tests** – Execute [`tests/unit/test_response_validator.py`](https://github.com/antoinezambelli/forge/blob/main/tests/unit/test_response_validator.py) and [`tests/unit/test_proxy_handler.py`](https://github.com/antoinezambelli/forge/blob/main/tests/unit/test_proxy_handler.py) to reproduce failure paths and compare against expected log output.

## Code Examples

### Reproducing an Unknown Tool Validation Failure

This example demonstrates how to trigger a validation error by mismatching the available tools with the model's intentions:

```python
from forge.proxy import ProxyServer
import requests

# Initialize proxy pointing to a local backend

proxy = ProxyServer(backend_url="http://localhost:11434")
proxy.start()

# Send request with limited tools while expecting a weather call

payload = {
    "model": "llama3",
    "messages": [{"role": "user", "content": "Give me the weather"}],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "search",
                "description": "Search the web",
                "parameters": {}
            }
        }
    ],
}

resp = requests.post(
    f"{proxy.url}/v1/chat/completions", 
    json=payload
).json()

# If the model attempts to call "get_weather", validation fails

print(resp)  # Contains "unknown_tool" nudge

proxy.stop()

```

### Configuring Retry Limits and Rescue Behavior

Increase resilience against noisy models by adjusting the guardrails configuration:

```python
from forge.proxy import ProxyServer

proxy = ProxyServer(
    backend_url="http://localhost:11434",
    max_retries=5,          # Default is 3; increase for unstable models

    rescue_enabled=True,    # Attempt to parse tool calls from text

)

```

### Direct Validator Inspection

Test the validation logic independently without running the full proxy:

```python
from forge.guardrails import ResponseValidator, ValidationResult
from forge.core.workflow import TextResponse

validator = ResponseValidator(
    tool_names=["search", "calculate"], 
    rescue_enabled=True
)

# Simulate a text response that should be rescued as a search call

text_response = TextResponse(content="search(query='London weather')")
result: ValidationResult = validator.validate(text_response)

print(result.tool_calls)   # Recovered ToolCall object if rescue succeeds

print(result.is_valid)     # Boolean indicating validation status

```

## Key Files and Implementation Details

| Component | File Path | Key Functionality |
|-----------|-----------|-------------------|
| Request Handler | [`proxy/handler.py`](https://github.com/antoinezambelli/forge/blob/main/proxy/handler.py) | Entry point at `handle_chat_completions`; manages `ResponseValidator` initialization and `respond` tool injection (lines 108-114) |
| Response Validator | [`guardrails/response_validator.py`](https://github.com/antoinezambelli/forge/blob/main/guardrails/response_validator.py) | Core validation at `validate` method; handles rescue logic (lines 64-71) and tool name verification (lines 81-94) |
| Guardrails Orchestration | [`guardrails/guardrails.py`](https://github.com/antoinezambelli/forge/blob/main/guardrails/guardrails.py) | `Guardrails.check` coordinates validation with retry budgeting via `ErrorTracker` (lines 19-26) |
| Error Tracking | [`guardrails/error_tracker.py`](https://github.com/antoinezambelli/forge/blob/main/guardrails/error_tracker.py) | `ErrorTracker` class monitors consecutive failures; check `retries_exhausted` property (lines 19-22) |
| Rescue Templates | [`prompts/templates.py`](https://github.com/antoinezambelli/forge/blob/main/prompts/templates.py) | Contains `rescue_tool_call` function for parsing tool calls from plain text |
| Unit Tests | [`tests/unit/test_response_validator.py`](https://github.com/antoinezambelli/forge/blob/main/tests/unit/test_response_validator.py) | Exercises failure paths including unknown tools and rescue scenarios |
| Handler Tests | [`tests/unit/test_proxy_handler.py`](https://github.com/antoinezambelli/forge/blob/main/tests/unit/test_proxy_handler.py) | Validates proxy integration and logging output |

## Summary

- **Validation failures** in Forge proxy mode occur at three checkpoints: response parsing (including rescue attempts), tool name verification, and retry budget exhaustion.
- **Unknown-tool errors** indicate the LLM called a function not present in the `tool_names` list extracted from the request's `tools` array.
- **Retry nudges** appear when the validator cannot rescue a plain-text response as a valid tool call, or when rescue is disabled.
- **Fatal errors** occur after `max_retries` consecutive failures, tracked by `ErrorTracker` in [`guardrails/error_tracker.py`](https://github.com/antoinezambelli/forge/blob/main/guardrails/error_tracker.py).
- **Guardrails bypass** requests that contain no tools, explaining why some calls skip validation entirely.
- **Debug via logs** by enabling DEBUG level for `forge.proxy` and inspecting the specific checkpoint where validation fails.

## Frequently Asked Questions

### What causes "unknown_tool" validation errors in Forge proxy mode?

The **unknown-tool** error occurs in `ResponseValidator.validate` (lines 81-94 of [`guardrails/response_validator.py`](https://github.com/antoinezambelli/forge/blob/main/guardrails/response_validator.py)) when the LLM attempts to call a tool not present in the request's allowed `tool_names` list. This happens if the model hallucinates a function name, misspells a valid tool, or if the `tools` array in your request is incomplete. Verify that `_extract_tool_names` (lines 70-72 of [`handler.py`](https://github.com/antoinezambelli/forge/blob/main/handler.py)) correctly parsed your OpenAI-style tool definitions and that the model references these exact names.

### How does the rescue mechanism work for text responses?

When the LLM returns plain text instead of a structured tool call, the `ResponseValidator` attempts **rescue** parsing via `rescue_tool_call` in [`prompts/templates.py`](https://github.com/antoinezambelli/forge/blob/main/prompts/templates.py) (lines 64-71). If the text contains a recognizable tool invocation pattern, the validator extracts it into a proper `ToolCall` object. If rescue fails or is disabled (`rescue_enabled=False`), the validator returns a retry nudge (lines 72-79) asking the model to format its response correctly.

### Where can I adjust the retry limits for validation failures?

The retry budget is controlled by the `max_retries` parameter passed to the `ProxyServer` constructor in [`proxy/handler.py`](https://github.com/antoinezambelli/forge/blob/main/proxy/handler.py) (line 31). The default value is 3, but you can increase it to 5 or higher for noisy models that require multiple attempts, or reduce it to 1 for faster failure. The `ErrorTracker` class (lines 19-22 of [`error_tracker.py`](https://github.com/antoinezambelli/forge/blob/main/error_tracker.py)) monitors these attempts and sets `retries_exhausted` when the limit is reached.

### Why are guardrails bypassed for some requests?

The proxy validates only requests that contain tools. In [`handler.py`](https://github.com/antoinezambelli/forge/blob/main/handler.py) (lines 19-20), the code checks if `tool_specs` is empty; if so, it passes the request directly to the backend without instantiating the `ResponseValidator`. This optimization prevents unnecessary overhead for simple chat requests. To force validation on all requests, ensure your payload includes at least one tool definition, which automatically triggers the internal `respond` tool injection (lines 108-114).