How to Debug Routing Decisions in Switchyard Using Logging and Tracing Features
Enable SWITCHYARD_DEBUG=1 and query the /metrics endpoint to capture detailed routing decisions through Python logs and Prometheus counters.
Debugging routing decisions in NVIDIA Switchyard requires visibility into both the Python launcher layer and the underlying Rust router. The switchyard-server crate implements the core routing logic, while Python launchers (Claude, Codex, OpenClaw) provide user-facing instrumentation. Together, they expose routing internals through structured logging and OpenTelemetry metrics.
Enable Debug Logging for Routing Visibility
Switchyard uses a Python logger named switchyard to surface routing events from Rust. To capture these details, activate debug file logging via environment variable.
Set the Environment Variable
export SWITCHYARD_DEBUG=1
When this variable is truthy, the launcher calls configure_debug_file_logging() from [switchyard/cli/launchers/launcher_runtime.py](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/launchers/launcher_runtime.py). This function:
- Creates a log file at
./switchyard_debug.log(relative to the working directory) - Attaches a
logging.FileHandlerwithlevel=logging.DEBUG - Sets the global
switchyardlogger to DEBUG while keeping CLI output at INFO
Launch with Debug Logging Enabled
# Example: launch Claude launcher with routing debug logs
switchyard launch claude --model my-route
This produces timestamped entries showing every routing step.
Read the Debug Log File
The log reveals the exact routing pipeline execution. Typical entries include model name resolution, route selection algorithms, and upstream model assignment.
Example Log Output
2026-08-15 12:34:56,789 DEBUG switchyard.router: Routing request for model 'my-route'
2026-08-15 12:34:56,790 DEBUG switchyard.router: Selected route 'switchyard/random' → upstream model 'openai/gpt-4'
2026-08-15 12:34:57,015 INFO switchyard.router: Upstream call succeeded (status=200)
These entries originate from the Rust tracing crate and are funneled into Python's logging infrastructure via the libsy wrapper. The log references [native_server.py](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/launchers/native_server.py) and [openclaw_launcher.py](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/launchers/openclaw_launcher.py) as handler entry points.
Programmatically Extract Routing Decisions
import pathlib
log_path = pathlib.Path("switchyard_debug.log")
for line in log_path.read_text().splitlines():
if "Selected route" in line:
print(line) # e.g., "Selected route 'switchyard/random' → upstream model 'openai/gpt-4'"
Query Prometheus Metrics for Aggregate Routing Data
Switchyard exposes routing statistics through a /metrics endpoint (default port 4000). These counters provide aggregate visibility into route selection patterns across all requests.
Key Router Metrics
| Metric Name | Type | Description |
|---|---|---|
switchyard_router_retry_recovered_total |
Counter | Retries successfully resolved by the router |
switchyard_router_selected_model_total |
Counter | Frequency of upstream model selection by model name |
Sample Metrics Response
# HELP switchyard_router_retry_recovered_total Number of retries recovered by the router
# TYPE switchyard_router_retry_recovered_total counter
switchyard_router_retry_recovered_total{otel_scope_name="switchyard"} 0
# HELP switchyard_router_selected_model_total Counts of selected upstream models
# TYPE switchyard_router_selected_model_total counter
switchyard_router_selected_model_total{model="openai/gpt-4"} 42
switchyard_router_selected_model_total{model="anthropic/claude"} 13
The model label contains the final upstream model chosen after routing algorithm execution. Metrics are defined in [crates/switchyard-server/src/metrics.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/metrics.rs) and emitted from [router.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/router.rs).
Fetch Metrics via HTTP
import requests
resp = requests.get("http://localhost:4000/metrics")
for line in resp.text.splitlines():
if line.startswith("switchyard_router_selected_model_total"):
print(line)
Correlate Logs and Metrics for Complete Tracing
To debug why a specific request received a particular routing decision:
- Timestamp alignment — match log entry timestamps with metric scrape intervals
- Label verification — confirm the
modellabel in metrics matches the upstream model logged - Algorithm inspection — check log entries for intermediate stages (e.g., classifier verdicts, capacity-based fallbacks, stage-router overrides)
The Rust router implementation in [crates/switchyard-server/src/lib.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/lib.rs) emits both tracing events and Prometheus updates, ensuring consistency between log files and metrics.
Enable Debug Logging Programmatically
For embedded or scripted usage, set the environment variable before importing Switchyard modules:
import os
import logging
os.environ["SWITCHYARD_DEBUG"] = "1" # activates file logging
from switchyard.cli.launch_command import launch
# Launch with routing diagnostics captured to file
launch(["claude", "--model", "my-route"])
This pattern ensures the switchyard logger is configured before any routing occurs.
Source Files Reference
Summary
- Set
SWITCHYARD_DEBUG=1to enable detailed file logging of routing decisions - Read
switchyard_debug.logto see per-request route selection and upstream model assignment - Query
/metricson port 4000 for aggregate routing statistics via Prometheus counters - Cross-reference timestamps between logs and metrics to reconstruct complete routing behavior
- Source files in
crates/switchyard-server/contain the Rust implementation that emits both traces
Frequently Asked Questions
How do I know which routing algorithm was used for my request?
Check the debug log for entries containing Selected route. The logged string includes the route identifier (e.g., switchyard/random), which indicates whether random selection, a stage-router override, or classifier-driven routing was applied. The algorithm implementation in [router.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/router.rs) emits these specifics before finalizing the upstream model choice.
Can I debug routing without restart?
Yes. The /metrics endpoint is always active and does not require SWITCHYARD_DEBUG. For log-based debugging, you must restart the launcher after setting the environment variable, as configure_debug_file_logging() runs once during launcher initialization.
Why do my metrics show different counts than my log lines?
Metric counters aggregate across all requests since server start, while logs record individual transactions. The switchyard_router_selected_model_total counter increments atomically in the Rust layer, so it may include requests from other clients or internal health checks not visible in your specific launcher log file.
Where are retry decisions logged?
Retry events appear as DEBUG entries with the switchyard.router logger name, and the metric switchyard_router_retry_recovered_total tracks successful retries. Correlating log timestamps with retry status codes (e.g., 503 followed by 200) reconstructs the full retry sequence handled by the router.
Have a question about this repo?
These articles cover the highlights, but your codebase questions are specific. Give your agent direct access to the source. Share this with your agent to get started:
curl -s "https://instagit.com/install.md" 
Maintain an open-source project? Get it listed too →