Best Practices for Developing with Switchyard: A Comprehensive Guide

TLDR: Develop with Switchyard by using uv for dependency management, running both Python (pytest) and Rust (cargo test) test suites, adhering to strict typing with mypy --strict, avoiding panic! in Rust code, and following Conventional Commits with documentation updates for every public API change.

Switchyard, developed under the NVIDIA-NeMo organization, is a Python-centric orchestration layer that bridges client applications and large-language-model (LLM) back-ends. The core routing and serving logic is implemented in Rust via PyO3 bindings, while the Python package supplies thin wrappers, utilities, and a CLI. This guide distills the best practices for developing with Switchyard directly from the repository's structure, source files, and development documentation.

Overview of the Switchyard Repository Structure

Understanding the project layout is the first step toward writing effective contributions. The repository is organized to keep the Rust implementation stable and performant while the Python side remains easy to script and extend.

Area	Description	Key Files
Python entry point	switchyard package — version and top-level imports	[switchyard/__init__.py](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/__init__.py)
Rust bindings	PyO3 facade exposing native server and algorithm crates	[switchyard_rust/__init__.py](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/__init__.py), [switchyard_rust/_native.py](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/_native.py)
Routing algorithms	Typed Python wrappers around the libsy Rust crate	[switchyard/libsy/algorithms.py](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py)
Server implementation	Native Axum server, TOML-driven config, exposed via CLI	[crates/switchyard-server/README.md](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/README.md)
Protocol definitions	Provider-neutral request/response types	[crates/protocol/README.md](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/protocol/README.md)
Documentation	MkDocs source for architecture, concepts, and CLI reference	[docs/architecture.md](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/architecture.md), [docs/cli_reference.md](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/cli_reference.md)
Tests	Pytest for Python API, cargo test for Rust	tests/
Examples	Minimal usage snippets and Litellm integration	[examples/libsy.py](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py)

---

Setting Up Your Development Environment

The first best practice for developing with Switchyard is to establish a clean, reproducible environment using uv, the modern Python package manager.

Install Dependencies with uv

uv sync            # install core + dev dependencies

source .venv/bin/activate

The uv sync command reads the project's pyproject.toml and uv.lock files to install pinned dependencies, including development tools like ruff, mypy, and pytest. Full setup instructions are documented in [DEVELOPMENT.md](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/DEVELOPMENT.md).

---

Running the Full Test Suite

Testing is the gatekeeper for any change submit — a failing test is a "do not merge" signal. Switchyard maintains parallel test suites for its Python API and Rust core.

uv run pytest tests/ -v                          # Python unit tests

cargo test --workspace                           # Rust unit tests

uv run ruff check .                              # Lint

uv run mypy switchyard                           # Strict type-check

• pytest covers the Python wrappers, CLI behavior, and integration flows in the tests/ directory.
• cargo test validates the Rust crates under crates/, including the routing engine (libsy) and the Axum server.
• ruff enforces import ordering and style rules.
• mypy runs in strict mode, requiring explicit type annotations across every public Python function.

You should run all four commands locally before pushing any commit. In CI, these same commands run on every pull request.

---

Following the Strict Coding Style

The repository implements a strict, consistent style across both languages. Adhere to the following rules when developing with Switchyard:

• Line length: Keep lines at or below 100 characters (ruff's E501 is ignored because formatting is left to black).
• Type hints: Required everywhere on the Python side. The py.typed marker is present, and mypy runs in --strict mode, so untyped functions fail the build.
• Import ordering: All imports must be sorted according to ruff's I rules. Run uv run ruff check . --fix to auto-correct typical issues.
• Rust error handling: Avoid panic!, unwrap(), and .except() in production code. Propagate errors with the ? operator and typed Result types. The Rust crate README files detail these expectations.

Example of proper Rust error handling in Switchyard's style:

// Good: propagate errors with ?
fn load_config(path: &str) -> Result<Config, ConfigError> {
    let contents = fs::read_to_string(path)?;
    let config: Config = toml::from_str(&contents)?;
    Ok(config)
}

// Bad: panics in production code
fn load_config_bad(path: &str) -> Config {
    let contents = fs::read_to_string(path).unwrap();
    toml::from_str(&contents).expect("invalid config")
}

---

Commit Discipline and Documentation

Conventional Commits

Use Conventional Commits with the format type(scope): summary. For example:

feat(routing): add latency-aware algorithm
fix(server): resolve TOML config hot-reload race
docs(protocol): clarify ChatRequest message field semantics

Keep each commit to one logical change. Never commit secrets — the repo maintains a .gitignore entry for secrets/ and expects contributors to use .env.example templates instead.

Documentation Updates

One of the cornerstones of developing with Switch reliably is keeping docs in sync with code:

• Whenever you change a public Python function or class, add a docstring and update the corresponding Markdown page under docs/.
• For Rust changes, add /// comments on public structs, enums, and functions.
• When CLI flags or TOML config keys change, update the generated CLI reference and the TOML schema documentation at [docs/reference/toml_schema.md](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/reference/toml_schema.md).

---

Core Best-Practice Patterns

This section covers the recurring development patterns you'll apply when extending Switchyard.

Extending Routing Logic

Implement a new algorithm following these steps:

1. Add the Rust implementation under crates/libsy/src/, mirroring the existing module pattern.
2. Expose a thin Python wrapper in switchyard/libsy/algorithms.py.
3. Write unit tests in both Rust (cargo test) and Python (pytest).
4. Update the TOML schema documentation to discuss any new config keys.

Interacting with the Protocol-Neutral API

Ensure you use the structs defined in crates/protocol rather than hand-crafting JSON payloads. This consistency is what keeps Switchyard able to route across different LLM back-ends.

from switchyard.protocol import ChatRequest

req = ChatRequest(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello"}],
    temperature=0.7,
)

Always import ChatRequest and related types from the switchyard.protocol module — never construct raw dictionaries for the wire format.

Running the Native Server

switchyard-server --config routes.toml --port 4000

• Keep the TOML config version-controlled in your application's repository.
• Use the switchyard CLI (switchyard --help) for health checks and A/B testing utilities.
• Reference the full configuration guide at crates/switchyard-server/CONFIGURATION.md.

Testing with Mocked Back-Ends

To avoid flaky tests and external network calls:

• Python tests: Use respx for HTTP mocking (already a dev dependency).
• Rust tests: Use wiremock-rs or a similar crate to simulate upstream LLM services.

Both approaches let you test routing, retries, and error handling deterministically.

---

Common Pitfalls and How to Avoid Them

Teams new to Switchyard often hit a few recurring issues. Here's a quick reference:

Symptom	Likely Cause	Remedy
cargo test fails with "unreachable code"	A panic! left in production code	Remove panic! and switch to proper Result handling
mypy reports missing type hints	New function added without annotations	Add explicit type hints and run uv run mypy locally
CLI command crashes on missing env var	Forgetting to set a provider key (OPENAI_API_KEY, etc.)	Add a .env.example entry and document required variables in README.md
Tests are flaky in CI	Tests depend on external network or shared mutable state	Mock external calls (respx/wiremock) and set deterministic seeds
Lint errors from import order	New imports not sorted per ruff rules	Run uv run ruff check . --fix before staging

---

Example: Simple End-to-End Run

The following minimal example demonstrates the core flow (source: [examples/libsy.py](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py)):

from switchyard.libsy import RoutingEngine
from switchyard.protocol import ChatRequest

engine = RoutingEngine.from_toml("examples/routing_config.toml")

request = ChatRequest(
    model="gpt-4",
    messages=[{"role": "user", "content": "What is Switchyard?"}],
)

response = engine.route_chat(request)
print(response.choices[0].message.content)

This snippet demonstrates three best-practice behaviors:

1. TOML-driven configuration — the routing config is loaded from a file, not hard-coded.
2. Provider-neutral requests — ChatRequest comes from the switchyard.protocol package.
3. Python facade over Rust core — RoutingEngine internally invokes the Rust routing engine via PyO3.

---

Quick Reference Checklist

• Environment — Install uv and activate the virtual environment.
• Code style — Run ruff and mypy locally before each commit.
• Testing — Ensure both pytest and cargo test pass.
• Documentation — Update MkDocs pages for any public API change.
• Error handling — No panic! in Rust production code; always use typed errors.
• Secrets — Never commit real API keys; use .env.example as a template.

---

Summary

• Use uv for deterministic dependency management and virtual environment setup.
• Maintain dual test coverage — pytest for Python and cargo test for Rust — for every change.
• Adhere to strict typing — mypy --strict and full Python type hints are mandatory.
• Avoid panics in Rust — propagate errors with ? and Result types throughout production code.
• Update docs with every API change — MkDocs pages, CLI reference, and TOML schemas must stay in sync.
• Mock external LLM back-ends in tests to prevent flakiness and speed up CI runs.

---

Frequently Asked Questions

What is Switchyard primarily built with?

Switchyard has a Rust core for routing and serving logic, exposed to Python through PyO3 bindings. The Python package provides thin wrappers, algorithms, and a CLI, but the performance-critical server (an Axum-based native server) runs on Rust.

How do I run the Switchyard test suite?

Run uv run pytest tests/ -v for Python unit tests and cargo test --workspace for Rust tests. Additionally, run uv run ruff check . and uv run mypy switchyard to verify linting and type-checking.

What are the Python coding requirements for a contribution?

All public functions and classes must have explicit type hints, docstrings, and imports sorted by ruff's rules. The project runs mypy in --strict mode, so any missing annotation is a build failure.

Can I use Switchyard to connect to OpenAI or other LLM providers?

Yes. Switchyard is provider-neutral; you configure providers via TOML files (see the switchyard-server configuration docs) and interact through the protocol structs in crates/protocol. The examples/ directory also includes a Litellm integration example for experimental provider compatibility.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}