# How Switchyard Handles Context Window Exceeded Errors and Maps Them to SwitchyardError

> Learn how Switchyard gracefully handles 'context window exceeded' LLM errors, transforming them into consistent SwitchyardError responses for seamless integration.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**Switchyard converts upstream LLM "context window exceeded" errors into a unified `LlmClientError::ContextWindowExceeded` variant, then maps these through its routing layer, HTTP server, and Python bindings to deliver consistent `SwitchyardError` responses with the canonical code `context_length_exceeded`.**

The NVIDIA NeMo Switchyard repository provides a unified gateway for routing requests to multiple LLM providers. When a prompt exceeds a model's token limit, Switchyard detects this condition early and propagates it through several architectural layers—from the Rust LLM client through to HTTP responses and Python exceptions—ensuring callers receive predictable, actionable error information regardless of their integration method.

## Detection in the Rust LLM Client

The error handling pipeline begins in `libsy-llm-client`, Switchyard's low-level provider interface.

When an upstream LLM returns a context window overflow response—for example, OpenAI's `error.code == "context_length_exceeded"`—the client creates a specific error variant. This definition appears in [`crates/libsy-llm-client/src/client.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/src/client.rs):

```rust
LlmClientError::ContextWindowExceeded { model, message, .. }

```

This early detection allows the system to distinguish token limit violations from transient failures or authentication errors, enabling appropriate fallback strategies and clear user messaging.

## Propagation Through the Routing Layer

Rather than immediately surfacing the error, Switchyard's routing algorithm in [`crates/libsy/src/algorithms/llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/llm_class.rs) records the condition as a **routing fallback reason**:

```rust
RoutingFallbackReason::ContextWindow

```

This distinction matters for multi-model deployments. When one model rejects a prompt for length reasons, Switchyard can attempt the request against alternative models with larger context windows. The error bubbles up to the server layer unchanged, preserving the full diagnostic information for the final response.

## Mapping to HTTP Responses

The central error translation occurs in [`crates/switchyard-server/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/lib.rs) within the `client_error` function. When processing `LlmClientError::ContextWindowExceeded`, the server constructs an **ApiError** with these exact fields:

| Field | Value |
|-------|-------|
| `status` | `400 Bad Request` |
| `error_type` | `"invalid_request_error"` |
| `code` | `"context_length_exceeded"` |
| `message` | Upstream error message preserved |

The implementation at lines 909-914:

```rust
LlmClientError::ContextWindowExceeded { message, .. } => error_response(
    StatusCode::BAD_REQUEST,
    message,
    "invalid_request_error",
    "context_length_exceeded",
),

```

The `ApiError` is then serialized into the provider-specific JSON envelope—matching OpenAI or Anthropic response formats—and returned to the caller.

## Python Exception Mapping

For Python integrations, `switchyard-py` bridges Rust errors to Python exceptions. The conversion happens in two locations:

1. **Exception class definition** in [`crates/switchyard-py/src/errors.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-py/src/errors.rs) (lines 11-14) creates `ContextWindowExceededError`
2. **Binding conversion** in [`crates/switchyard-py/src/libsy_bindings.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-py/src/libsy_bindings.rs) (lines 278-283) inspects the Rust error and wraps it appropriately

## Practical Code Examples

### Rust: Direct LLM Client Usage

```rust
use switchyard_libsy::client::LlmClient;
use switchyard_libsy::errors::LlmClientError;

async fn request_model() -> Result<String, LlmClientError> {
    let client = LlmClient::new(...);
    match client.chat(...).await {
        Ok(resp) => Ok(resp),
        Err(LlmClientError::ContextWindowExceeded { message, .. }) => {
            // Custom handling: truncate prompt or inform user
            log::warn!("Context window exceeded: {}", message);
            Err(LlmClientError::ContextWindowExceeded { message, .. })
        }
        Err(e) => Err(e),
    }
}

```

### Python: Catching Mapped Exceptions

```python
from switchyard import Switchyard
from switchyard.errors import ContextWindowExceededError

sw = Switchyard()
try:
    sw.chat(model="gpt-4", messages=[...])
except ContextWindowExceededError as e:
    print("Prompt too long:", e)
    # Application logic: trim prompt or request smaller model

```

### HTTP: Error Response Format

```bash
curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4","messages":[{"role":"user","content":"<very long prompt>"}]}'

```

**Response:**

```json
{
  "error": {
    "message": "This model's maximum context length is 8192 tokens...",
    "type": "invalid_request_error",
    "code": "context_length_exceeded"
  }
}

```

## Key Implementation Files

| Component | File Path | Purpose |
|-----------|-----------|---------|
| Server error mapping | `crates/switchyard-server/src/lib.rs#L909-L914` | Translates `ContextWindowExceeded` to HTTP 400 with standardized error code |
| LLM client error definition | `crates/libsy-llm-client/src/client.rs#L356-L363` | Defines the `ContextWindowExceeded` variant |
| Routing fallback handling | `crates/libsy/src/algorithms/llm_class.rs#L530-L537` | Marks error as `RoutingFallbackReason::ContextWindow` |
| Python exception class | `crates/switchyard-py/src/errors.rs#L11-L14` | Creates `ContextWindowExceededError` |
| Python binding conversion | `crates/switchyard-py/src/libsy_bindings.rs#L278-L283` | Maps Rust error to Python exception |

## Summary

- **Early detection**: `libsy-llm-client` captures context window violations as `LlmClientError::ContextWindowExceeded`
- **Routing intelligence**: The error triggers `RoutingFallbackReason::ContextWindow` for potential model failover
- **Consistent HTTP responses**: Server returns `400 Bad Request` with `code: "context_length_exceeded"`
- **Python parity**: Rust errors convert to `ContextWindowExceededError` for SDK users
- **Provider compatibility**: Error format matches OpenAI/Anthropic conventions for drop-in replacement

## Frequently Asked Questions

### What HTTP status code does Switchyard return for context window exceeded errors?

Switchyard returns **400 Bad Request** with `error_type: "invalid_request_error"` and `code: "context_length_exceeded"`. This matches OpenAI's API behavior, making Switchyard compatible with existing client implementations expecting that specific error format.

### Can Switchyard automatically retry with a different model when the context window is exceeded?

Yes. The routing layer in [`crates/libsy/src/algorithms/llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/llm_class.rs) classifies this error as `RoutingFallbackReason::ContextWindow`, allowing the scheduler to attempt the request against alternative models with larger context windows if configured in your routing policy.

### How do I catch context window errors in Python code using Switchyard?

Import `ContextWindowExceededError` from `switchyard.errors` and wrap your `chat()` calls in a try-except block. The exception preserves the upstream message for debugging while providing a typed catch condition for programatic response—such as truncating your prompt or selecting a different model.

### Where is the error code "context_length_exceeded" defined in the Switchyard source?

The canonical string `"context_length_exceeded"` is hardcoded in [`crates/switchyard-server/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/lib.rs) at line 912 within the `client_error` function's match arm for `LlmClientError::ContextWindowExceeded`. This ensures consistency across all response paths regardless of which upstream provider originally signaled the error.