# Best Practices for Developing with Switchyard: A Comprehensive Guide

> Master Switchyard development with our comprehensive guide. Learn best practices for dependency management testing strict typing Rust safety and commit conventions to build robust applications.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: best-practices
- Published: 2026-08-23

---

**TLDR:** Develop with Switchyard by using `uv` for dependency management, running both Python (`pytest`) and Rust (`cargo test`) test suites, adhering to strict typing with `mypy --strict`, avoiding `panic!` in Rust code, and following Conventional Commits with documentation updates for every public API change.

Switchyard, developed under the **NVIDIA-NeMo** organization, is a Python-centric orchestration layer that bridges client applications and large-language-model (LLM) back-ends. The core routing and serving logic is implemented in **Rust** via PyO3 bindings, while the Python package supplies thin wrappers, utilities, and a CLI. This guide distills the best practices for developing with Switchyard directly from the repository's structure, source files, and development documentation.

## Overview of the Switchyard Repository Structure

Understanding the project layout is the first step toward writing effective contributions. The repository is organized to keep the Rust implementation stable and performant while the Python side remains easy to script and extend.

| Area | Description | Key Files |
|------|-------------|-----------|
| **Python entry point** | `switchyard` package — version and top-level imports | [[`switchyard/__init__.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/__init__.py)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/__init__.py) |
| **Rust bindings** | PyO3 facade exposing native server and algorithm crates | [[`switchyard_rust/__init__.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/__init__.py)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/__init__.py), [[`switchyard_rust/_native.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/_native.py)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/_native.py) |
| **Routing algorithms** | Typed Python wrappers around the `libsy` Rust crate | [[`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py) |
| **Server implementation** | Native Axum server, TOML-driven config, exposed via CLI | [[`crates/switchyard-server/README.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/README.md)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/README.md) |
| **Protocol definitions** | Provider-neutral request/response types | [[`crates/protocol/README.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/protocol/README.md)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/protocol/README.md) |
| **Documentation** | MkDocs source for architecture, concepts, and CLI reference | [[`docs/architecture.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/architecture.md)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/architecture.md), [[`docs/cli_reference.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/cli_reference.md)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/cli_reference.md) |
| **Tests** | Pytest for Python API, `cargo test` for Rust | [`tests/`](https://github.com/NVIDIA-NeMo/Switchyard/tree/main/tests) |
| **Examples** | Minimal usage snippets and Litellm integration | [[`examples/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py) |

---

## Setting Up Your Development Environment

The first best practice for developing with Switchyard is to establish a clean, reproducible environment using **`uv`**, the modern Python package manager.

### Install Dependencies with `uv`

```bash
uv sync            # install core + dev dependencies

source .venv/bin/activate

```

The `uv sync` command reads the project's [`pyproject.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/pyproject.toml) and `uv.lock` files to install pinned dependencies, including development tools like `ruff`, `mypy`, and `pytest`. Full setup instructions are documented in [[`DEVELOPMENT.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/DEVELOPMENT.md)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/DEVELOPMENT.md).

---

## Running the Full Test Suite

**Testing is the gatekeeper for any change submit** — a failing test is a "do not merge" signal. Switchyard maintains parallel test suites for its Python API and Rust core.

```bash
uv run pytest tests/ -v                          # Python unit tests

cargo test --workspace                           # Rust unit tests

uv run ruff check .                              # Lint

uv run mypy switchyard                           # Strict type-check

```

- **`pytest`** covers the Python wrappers, CLI behavior, and integration flows in the `tests/` directory.
- **`cargo test`** validates the Rust crates under `crates/`, including the routing engine (`libsy`) and the Axum server.
- **`ruff`** enforces import ordering and style rules.
- **`mypy`** runs in **strict mode**, requiring explicit type annotations across every public Python function.

You should run all four commands locally before pushing any commit. In CI, these same commands run on every pull request.

---

## Following the Strict Coding Style

The repository implements a strict, consistent style across both languages. Adhere to the following rules when developing with Switchyard:

- **Line length**: Keep lines at or below 100 characters (ruff's `E501` is ignored because formatting is left to `black`).
- **Type hints**: Required everywhere on the Python side. The `py.typed` marker is present, and mypy runs in `--strict` mode, so untyped functions fail the build.
- **Import ordering**: All imports must be sorted according to ruff's `I` rules. Run `uv run ruff check . --fix` to auto-correct typical issues.
- **Rust error handling**: Avoid `panic!`, `unwrap()`, and `.except()` in production code. Propagate errors with the `?` operator and typed `Result` types. The Rust crate README files detail these expectations.

Example of proper Rust error handling in Switchyard's style:

```rust
// Good: propagate errors with ?
fn load_config(path: &str) -> Result<Config, ConfigError> {
    let contents = fs::read_to_string(path)?;
    let config: Config = toml::from_str(&contents)?;
    Ok(config)
}

// Bad: panics in production code
fn load_config_bad(path: &str) -> Config {
    let contents = fs::read_to_string(path).unwrap();
    toml::from_str(&contents).expect("invalid config")
}

```

---

## Commit Discipline and Documentation

### Conventional Commits

**Use Conventional Commits** with the format `type(scope): summary`. For example:

```git
feat(routing): add latency-aware algorithm
fix(server): resolve TOML config hot-reload race
docs(protocol): clarify ChatRequest message field semantics

```

Keep each commit to one logical change. Never commit secrets — the repo maintains a `.gitignore` entry for `secrets/` and expects contributors to use `.env.example` templates instead.

### Documentation Updates

One of the cornerstones of developing with Switch reliably is keeping docs in sync with code:

- Whenever you change a public Python function or class, add a docstring and update the corresponding Markdown page under `docs/`.
- For Rust changes, add `///` comments on public structs, enums, and functions.
- When CLI flags or TOML config keys change, update the generated CLI reference and the TOML schema documentation at [[`docs/reference/toml_schema.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/reference/toml_schema.md)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/reference/toml_schema.md).

---

## Core Best-Practice Patterns

This section covers the recurring development patterns you'll apply when extending Switchyard.

### Extending Routing Logic

**Implement a new algorithm** following these steps:

1. Add the Rust implementation under `crates/libsy/src/`, mirroring the existing module pattern.
2. Expose a thin Python wrapper in [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py).
3. Write unit tests in both Rust (`cargo test`) and Python (`pytest`).
4. Update the TOML schema documentation to discuss any new config keys.

### Interacting with the Protocol-Neutral API

Ensure you use the structs defined in `crates/protocol` rather than hand-crafting JSON payloads. This consistency is what keeps Switchyard able to route across different LLM back-ends.

```python
from switchyard.protocol import ChatRequest

req = ChatRequest(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello"}],
    temperature=0.7,
)

```

Always import `ChatRequest` and related types from the `switchyard.protocol` module — never construct raw dictionaries for the wire format.

### Running the Native Server

```bash
switchyard-server --config routes.toml --port 4000

```

- Keep the TOML config version-controlled in your application's repository.
- Use the `switchyard` CLI (`switchyard --help`) for health checks and A/B testing utilities.
- Reference the full configuration guide at [`crates/switchyard-server/CONFIGURATION.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/CONFIGURATION.md).

### Testing with Mocked Back-Ends

To avoid flaky tests and external network calls:

- **Python tests**: Use `respx` for HTTP mocking (already a dev dependency).
- **Rust tests**: Use `wiremock-rs` or a similar crate to simulate upstream LLM services.

Both approaches let you test routing, retries, and error handling deterministically.

---

## Common Pitfalls and How to Avoid Them

Teams new to Switchyard often hit a few recurring issues. Here's a quick reference:

| Symptom | Likely Cause | Remedy |
|---------|--------------|--------|
| `cargo test` fails with "unreachable code" | A `panic!` left in production code | Remove `panic!` and switch to proper `Result` handling |
| `mypy` reports missing type hints | New function added without annotations | Add explicit type hints and run `uv run mypy` locally |
| CLI command crashes on missing env var | Forgetting to set a provider key (`OPENAI_API_KEY`, etc.) | Add a `.env.example` entry and document required variables in [`README.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/README.md) |
| Tests are flaky in CI | Tests depend on external network or shared mutable state | Mock external calls (`respx`/`wiremock`) and set deterministic seeds |
| Lint errors from import order | New imports not sorted per ruff rules | Run `uv run ruff check . --fix` before staging |

---

## Example: Simple End-to-End Run

The following minimal example demonstrates the core flow (source: [[`examples/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py)):

```python
from switchyard.libsy import RoutingEngine
from switchyard.protocol import ChatRequest

engine = RoutingEngine.from_toml("examples/routing_config.toml")

request = ChatRequest(
    model="gpt-4",
    messages=[{"role": "user", "content": "What is Switchyard?"}],
)

response = engine.route_chat(request)
print(response.choices[0].message.content)

```

This snippet demonstrates three best-practice behaviors:

1. **TOML-driven configuration** — the routing config is loaded from a file, not hard-coded.
2. **Provider-neutral requests** — `ChatRequest` comes from the `switchyard.protocol` package.
3. **Python facade over Rust core** — `RoutingEngine` internally invokes the Rust routing engine via PyO3.

---

## Quick Reference Checklist

- [ ] **Environment** — Install `uv` and activate the virtual environment.
- [ ] **Code style** — Run `ruff` and `mypy` locally before each commit.
- [ ] **Testing** — Ensure both `pytest` and `cargo test` pass.
- [ ] **Documentation** — Update MkDocs pages for any public API change.
- [ ] **Error handling** — No `panic!` in Rust production code; always use typed errors.
- [ ] **Secrets** — Never commit real API keys; use `.env.example` as a template.

---

## Summary

- **Use `uv`** for deterministic dependency management and virtual environment setup.
- **Maintain dual test coverage** — `pytest` for Python and `cargo test` for Rust — for every change.
- **Adhere to strict typing** — `mypy --strict` and full Python type hints are mandatory.
- **Avoid panics in Rust** — propagate errors with `?` and `Result` types throughout production code.
- **Update docs with every API change** — MkDocs pages, CLI reference, and TOML schemas must stay in sync.
- **Mock external LLM back-ends** in tests to prevent flakiness and speed up CI runs.

---

## Frequently Asked Questions

### What is Switchyard primarily built with?

Switchyard has a **Rust core** for routing and serving logic, exposed to Python through **PyO3 bindings**. The Python package provides thin wrappers, algorithms, and a CLI, but the performance-critical server (an Axum-based native server) runs on Rust.

### How do I run the Switchyard test suite?

Run `uv run pytest tests/ -v` for Python unit tests and `cargo test --workspace` for Rust tests. Additionally, run `uv run ruff check .` and `uv run mypy switchyard` to verify linting and type-checking.

### What are the Python coding requirements for a contribution?

All public functions and classes must have explicit type hints, docstrings, and imports sorted by ruff's rules. The project runs `mypy` in `--strict` mode, so any missing annotation is a build failure.

### Can I use Switchyard to connect to OpenAI or other LLM providers?

Yes. Switchyard is provider-neutral; you configure providers via TOML files (see the `switchyard-server` configuration docs) and interact through the protocol structs in `crates/protocol`. The `examples/` directory also includes a Litellm integration example for experimental provider compatibility.