How to Debug Switchyard Applications: End-to-End Logging for NVIDIA NeMo’s Python-Rust Stack

Enable logging.DEBUG in Python and set RUST_LOG=debug to trace requests across Switchyard’s bilingual architecture, using respx for mocked testing and cargo test for Rust validation.

Switchyard is a sophisticated request-routing framework that mediates between Python client code and LLM back-ends through a native Rust server. Debugging applications built with the NVIDIA-NeMo/Switchyard repository requires tracing execution across the Python façade and the PyO3 bindings that interface with the Rust core. This guide covers how to debug Switchyard applications by leveraging the logging module for Python components and the tracing crate for Rust diagnostics, with specific techniques for common failure modes.

Understanding the Switchyard Architecture

Before inserting breakpoints, you must understand where code executes. The system spans four distinct layers:

Layer	Language	Role	Key Source File
Python façade	Python	Exposes switchyard.* APIs and loads Rust bindings	switchyard/__init__.py
Rust server	Rust (PyO3)	Hosts the HTTP server and parses TOML deployment configs	switchyard_rust/server.py
Libsy algorithms	Python	Implements routing strategies that select endpoints	switchyard/libsy/algorithms.py
LLM client bindings	Rust	Translates provider-neutral protocol to HTTP calls	crates/libsy-llm-client

All traffic flows Client → Python façade → Rust server → Provider-neutral protocol → Upstream LLM. Because the stack spans two languages, you must monitor two distinct debugging surfaces simultaneously.

Enabling Verbose Logging Across the Stack

Python Side Configuration

Most Switchyard modules use the standard logging library. To capture request routing, configuration loading, and error handling, configure the root logger or the specific "switchyard" logger:

import logging

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("switchyard")
logger.debug("Switchyard debug enabled")

Setting level=logging.DEBUG ensures that high-level routing decisions in switchyard/libsy/algorithms.py emit detailed traces before invoking the native layer.

Rust Server Diagnostics

The Rust components use the tracing crate, which respects the RUST_LOG environment variable. To see granular event logs from the native server:

export RUST_LOG=debug
uv run python -m switchyard_rust.server

When launching via the CLI entry point (switchyard-server), the same variable applies. The switchyard_rust/server.py file reads this environment variable during initialization to configure the subscriber.

Unified Debug Environment

Combine both logging systems when running a full application:

export RUST_LOG=debug
python -c "import logging; logging.basicConfig(level=logging.DEBUG); import switchyard"

This prints every Python log line alongside Rust tracing events, allowing you to correlate the high-level Python flow in switchyard/__init__.py with low-level Rust request processing.

Debugging the Rust Server Process

Foreground Execution and TOML Validation

Run the Rust server in the foreground to catch startup failures immediately. The binary prints its own startup banner and configuration errors. If the server aborts, inspect the generated TOML files—misconfigured routes surface as parsing errors with exact file paths and line numbers logged to stdout.

Unit Testing with Cargo

For rapid iteration without launching an HTTP server, use the Rust test harness:

cargo test --workspace

Rust unit tests in crates/*/tests compile with the env_logger test harness and respect RUST_LOG=debug. This is the fastest way to verify that a routing algorithm behaves correctly before integration testing.

Testing and Debugging Python Bindings

Pytest with Verbose Output

The Python test suite exercises PyO3 entry points. Run with maximum verbosity to see each test name and any print or logging output:

uv run pytest -vv

This executes tests in tests/test_libsy_minimal_bindings.py, which demonstrates how Python bindings interact with the Rust layer. Use these tests as a sandbox for adding temporary logger.debug statements.

Isolating Requests with respx

When debugging a specific routing decision, use the respx library to mock upstream HTTP calls. This isolates the logic in switchyard/libsy/algorithms.py from external network variables.

import logging
import respx
from switchyard.libsy import algorithms

logging.basicConfig(level=logging.DEBUG)

@respx.mock
def test_routing_debug():
    respx.get("https://api.openai.com/v1/chat/completions").mock(
        return_value=respx.Response(200, json={"choices": []})
    )
    result = algorithms.route_request(...)
    assert result is not None

Enable logging.DEBUG before the mock to see exactly which endpoint the routing algorithm selected in the selected_route field.

Common Debugging Scenarios

• “Provider-neutral request type not found”: This indicates a mismatch between the Python wrapper and Rust struct definitions. Verify that the switchyard-protocol crate version matches the Python package using pip show switchyard.

• Server crashes on startup: Invalid TOML configuration (e.g., missing endpoints) causes immediate termination. Run with RUST_LOG=debug to see the exact parsing error and line number before the abort.

• Unexpected latency: If the routing algorithm falls back to a slower endpoint, enable logging.DEBUG on the Python side and inspect the selected_route field in each request log emitted by switchyard/libsy/algorithms.py.

• Missing environment variables: API keys like OPENAI_API_KEY must be exported before starting the server. The Rust side reads these via std::env during initialization in switchyard_rust/server.py; they are not hot-reloaded.

Quick Debugging Checklist

• Set logging.basicConfig(level=logging.DEBUG) in your Python entry script to capture façade-level events.
• Export RUST_LOG=debug (or trace) before launching the server to enable Rust tracing output.
• Run uv run pytest -vv to verify Python bindings and catch PyO3 interface errors.
• Execute cargo test --workspace to validate Rust implementations in isolation.
• Inspect server logs for lines prefixed with [DEBUG switchyard::...] to trace native execution.
• Use respx to mock external endpoints and isolate routing logic from network variability.
• Verify that all required environment variables are present in the process environment before starting switchyard-server.

Summary

• Combine logging systems: Use Python’s logging.DEBUG alongside RUST_LOG=debug to trace requests from switchyard/__init__.py through the Rust server to the upstream LLM.
• Validate configurations early: Run the server in foreground mode to catch TOML parsing errors with exact line numbers before they cause runtime failures.
• Test layers independently: Use cargo test --workspace for Rust algorithms and respx mocks for Python routing logic to isolate defects without external dependencies.

Frequently Asked Questions

How do I enable debug logging for the Switchyard Rust server?

Set the RUST_LOG environment variable to debug or trace before launching the server. The Rust components use the tracing crate, which emits granular event logs including HTTP request parsing and TOML configuration loading. This applies whether you run switchyard-server directly or launch via uv run python -m switchyard_rust.server.

Why is my Switchyard application failing to start with a TOML error?

The Rust server strictly validates deployment configuration files at startup. Invalid TOML syntax or missing endpoint definitions cause immediate termination with parsing errors. Run the server with RUST_LOG=debug to see the exact file path and line number where the parser failed, then cross-reference with the schema requirements in switchyard_rust/server.py.

How can I test Switchyard routing algorithms without hitting real LLM endpoints?

Use the respx library to mock HTTP routes in your pytest suite. Import algorithms from switchyard.libsy, enable logging.DEBUG to capture the selected_route field, and decorate tests with @respx.mock to intercept calls to providers like OpenAI. This isolates the routing decision logic from network latency and API quota limits.

Where does Switchyard log the selected route for each request?

The routing decision is logged by the Python layer in switchyard/libsy/algorithms.py at the DEBUG level. Look for log entries containing the selected_route field after calling algorithms.route_request(). If using the Rust server directly, the tracing events in crates/libsy-llm-client log the final HTTP target before dispatch.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}