# How to Install NVIDIA Switchyard: Python Package, Rust Server, and Library Integration

> Install NVIDIA Switchyard easily via pip for Python, cargo for the Rust server, or integrate individual crates into your Rust projects. Get started with Switchyard today.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: getting-started
- Published: 2026-08-23

---

**Install NVIDIA Switchyard via `pip install nemo-switchyard` for Python bindings, `cargo install --locked switchyard-server` for the standalone HTTP server, or add individual crates to your [`Cargo.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/Cargo.toml) for embedded Rust applications.**

NVIDIA Switchyard is a hybrid Python‑Rust orchestration layer for large‑language‑model (LLM) traffic. According to the NVIDIA‑NeMo/Switchyard source code, the project separates concerns into three installable components: Python bindings that expose routing algorithms via PyO3, a production‑grade Rust server binary, and modular Rust crates for custom integrations. Depending on your use case—whether you need a Python library, a dedicated proxy server, or embedded routing logic—you will follow different installation paths.

## Prerequisites

Before installing any component, verify your environment meets the minimum requirements. The Python package requires **Python ≥ 3.10**, while the Rust server and libraries require **Rust ≥ 1.96.1**. Hardware constraints apply to pre‑built wheels: Linux x86_64 wheels require a CPU with AVX2‑class instructions, and aarch64 wheels require a Neoverse N1‑class CPU.

## Install the Python Package

The simplest way to use Switchyard in Python applications is via the `nemo-switchyard` package published on PyPI. This package bundles the native Rust extension and has no runtime dependencies beyond the compiled binary.

Run the following command:

```bash
pip install nemo-switchyard

```

After installation, import the top‑level `switchyard` module to access the `libsy` routing algorithms. Because the native extension is bundled, you do not need a separate Rust toolchain for basic Python usage. The entry point is defined in [`pyproject.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/pyproject.toml), with core algorithm implementations located in [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py).

## Install the Standalone Rust Server

For production deployments requiring a dedicated LLM routing service, install the `switchyard-server` binary from crates.io. This server loads TOML deployment configurations and proxies requests to upstream LLM providers.

Execute:

```bash
cargo install --locked switchyard-server

```

Validate the installation with a dry‑run:

```bash
switchyard-server --config routes.toml --dry-run

```

Start the server on your desired port (for example, 4000):

```bash
switchyard-server --config routes.toml --port 4000

```

Configuration details and deployment schemas are documented in [`crates/switchyard-server/README.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/README.md) and the Getting Started guide at [`docs/getting_started.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/getting_started.md).

## Use the Rust Libraries in Custom Applications

To embed Switchyard’s routing logic directly into your own Rust binaries, add the required crates to your [`Cargo.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/Cargo.toml):

```toml
[dependencies]
switchyard-libsy = "0.2.0"
switchyard-protocol = "0.2.0"
switchyard-llm-client = "0.2.0"
switchyard-translation = "0.2.0"

```

These crates provide distinct functionality: `switchyard-libsy` contains the core routing algorithms (such as `RoundRobin`), `switchyard-protocol` defines provider‑neutral request and response types, `switchyard-llm-client` handles HTTP translation logic, and `switchyard-translation` manages wire‑format codecs. Source files for these components reside in the respective `crates/` directory of the repository.

## Development Setup (Optional)

When contributing to Switchyard or building from source, clone the repository and run the following commands. The `dev` dependency group contains linting, formatting, and test utilities that are excluded from the published wheel.

```bash
uv sync                     # Install Python dev dependencies

uv run maturin develop      # Build the native PyO3 extension

cargo test --workspace      # Run Rust unit tests

uv run pytest tests/ -v     # Run Python test suite

```

## Verification and Usage Examples

After installation, verify functionality using the patterns below.

### Python API Example

The following snippet demonstrates routing a request through the Python bindings. The `switchyard.libsy.Algorithms` class exposes native Rust routing logic compiled via PyO3.

```python
import switchyard

# Initialize a round-robin router

router = switchyard.libsy.Algorithms.round_robin()

# Construct a provider-neutral request

request = {
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello, Switchyard!"}]
}

# Route the request; the library handles HTTP translation internally

response = router.route(request)
print(response["choices"][0]["message"]["content"])

```

### Server and cURL Example

Start the standalone server and query it with an OpenAI‑compatible payload. This workflow is documented in [`docs/getting_started.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/getting_started.md) under the Server Path section.

```bash

# Start the server in the background

switchyard-server --config routes.toml --port 4000 &

```

```bash

# Send a chat completion request

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
        "model": "gpt-4o-mini",
        "messages": [{"role":"user","content":"What is Switchyard?"}]
      }'

```

### Rust Library Example

Embed the routing algorithm directly in a Rust application. The source for `RoundRobin` is located in [`crates/switchyard-libsy/src/algorithms.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-libsy/src/algorithms.rs).

```rust
use switchyard_libsy::algorithms::RoundRobin;
use switchyard_protocol::ChatRequest;

fn main() {
    let router = RoundRobin::new();
    let request = ChatRequest {
        model: "gpt-4o-mini".into(),
        messages: vec![("user".into(), "Hello from Rust!".into())],
        ..Default::default()
    };
    let response = router.route(request);
    println!("{:?}", response);
}

```

## Summary

- **Python users** should run `pip install nemo-switchyard` to receive the bundled PyO3 extension with no additional Rust toolchain required.
- **Server operators** should run `cargo install --locked switchyard-server` to deploy the standalone HTTP proxy defined in `crates/switchyard-server/`.
- **Rust developers** should add specific crates—`switchyard-libsy`, `switchyard-protocol`, `switchyard-llm-client`, and `switchyard-translation`—to [`Cargo.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/Cargo.toml) for embedded use.
- **Contributors** must install Python ≥ 3.10 and Rust ≥ 1.96.1, then use `uv` and `cargo` to build and test the hybrid codebase.

## Frequently Asked Questions

### Do I need to install Rust to use the Python package?

No. The `nemo-switchyard` wheel distributed on PyPI contains the pre‑compiled Rust extension bundled via PyO3. You only need Python ≥ 3.10 and a compatible CPU (AVX2 for x86_64, Neoverse N1 for aarch64). A Rust toolchain is only required if you intend to build the package from source.

### What is the difference between the Python bindings and the standalone server?

The **Python bindings** expose Switchyard’s routing algorithms as an importable library for embedding in Python applications, as implemented in [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py). The **standalone server** (`switchyard-server`) is a production‑grade Rust binary that runs as a separate process, loads TOML configurations, and exposes an HTTP interface for proxying LLM requests. Choose the former for library integration and the latter for infrastructure‑level traffic management.

### Can I install the server from PyPI instead of crates.io?

No. The `nemo-switchyard` PyPI package contains the Python bindings and native extension, but it does not install the `switchyard-server` binary. To run the standalone server, you must use `cargo install --locked switchyard-server` or build the binary from the `crates/switchyard-server/` source directory.

### Which crate provides the core routing algorithms?

The **`switchyard-libsy`** crate (version 0.2.0) contains the core routing logic, including implementations like `RoundRobin`. This crate is used internally by the Python bindings via PyO3 and can be linked directly into custom Rust applications by adding it to [`Cargo.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/Cargo.toml).