How to Install Switchyard from NVIDIA-NeMo: Python, Server, and Rust Crate Setup

You can install Switchyard as a Python package via pip install nemo-switchyard, as a standalone binary via cargo install --locked switchyard-server, or embed the routing core directly in Rust by adding crates like switchyard-libsy to your Cargo.toml.

Switchyard is a modular LLM routing library maintained by NVIDIA that enables intelligent request distribution across multiple language model backends. According to the NVIDIA-NeMo/Switchyard source code, the project supports three distinct installation pathways depending on whether you need Python bindings, a standalone proxy server, or native Rust libraries.

Prerequisites

Before installing Switchyard, ensure your environment meets the following requirements:

• Python >= 3.10 (required for the nemo-switchyard package)
• Rust >= 1.96.1 (required for building the server or using the crates directly)
• Platform: Linux x86-64 or aarch64 wheels are provided for Python; other platforms require building from source

Installation Methods

Switchyard offers three primary installation paths, each suited to different deployment scenarios.

Python Bindings via pip

Install the nemo-switchyard package to embed the routing engine in Python applications or manage the native server through PyO3 bindings:

pip install nemo-switchyard

The wheel contains the compiled Rust libraries (switchyard-libsy, switchyard-protocol, and others), so no additional system-level dependencies are required. After installation, import the package to create a client that communicates with the Rust server:

import nemo_switchyard as sw

# Create a client that talks to the native Rust server

client = sw.client("http://localhost:4000")
response = client.complete(prompt="Hello, world!")
print(response.text)

The client class is defined in crates/switchyard-py/src/lib.rs within the NVIDIA-NeMo/Switchyard repository.

Standalone Server via Cargo

Install the switchyard-server binary to run a dedicated proxy that accepts OpenAI/Anthropic-compatible requests and forwards them to configured backends:

cargo install --locked switchyard-server

After installation, start the server with a TOML configuration file:

# Validate your configuration

switchyard-server --config routes.toml --dry-run

# Start listening on port 4000

switchyard-server --config routes.toml --port 4000

The server reads routing policies from routes.toml, which defines client connections, backend endpoints, and routing logic. For complete configuration options, refer to crates/switchyard-server/README.md in the source repository.

Rust Libraries via Cargo.toml

To embed the routing core directly in a Rust application, add the following dependencies to your Cargo.toml:

[dependencies]
switchyard-libsy = "0.2.0"
switchyard-protocol = "0.2.0"
switchyard-llm-client = "0.2.0"
switchyard-translation = "0.2.0"

Use the crates to build a custom router:

use switchyard_libsy::Algorithm;
use switchyard_protocol::client::Client;
use switchyard_translation::lib::translate_request;

fn main() {
    // Build a client pointing at a backend endpoint
    let client = Client::new("http://localhost:8000/v1/chat/completions");

    // Create a provider-neutral request
    let request = translate_request::from_openai_chat("What is Switchyard?");

    // Execute the routing algorithm
    let response = Algorithm::run_stream(request, &client);

    println!("LLM response: {}", response.unwrap());
}

The core routing logic resides in crates/libsy/src/algorithms.rs, while protocol-agnostic request types are defined in crates/protocol/src/lib.rs.

Development Installation

To install Switchyard from source for development or testing:

# Install Python dependencies

uv sync

# Build native extensions in editable mode

uv run maturin develop

# Run Rust workspace tests

cargo test --workspace

# Run Python test suite

uv run pytest tests/ -v

These commands are documented in INSTALLATION.md within the NVIDIA-NeMo/Switchyard repository.

Summary

• Python users should run pip install nemo-switchyard to get the compiled bindings including the native Rust libraries.
• Server operators should use cargo install --locked switchyard-server to deploy the standalone proxy with OpenAI-compatible API support.
• Rust developers should add switchyard-libsy, switchyard-protocol, and related crates to Cargo.toml to embed routing logic directly.
• All methods require Python >= 3.10 or Rust >= 1.96.1, with Linux x86-64 wheels provided for convenience.

Frequently Asked Questions

What are the system requirements for installing Switchyard?

Switchyard requires Python >= 3.10 for the PyPI package or Rust >= 1.96.1 for building the server or using the crates. Pre-built Linux x86-64 and aarch64 wheels are available for Python installations; other platforms must compile from source using Rust.

How do I verify my Switchyard installation is working?

For Python installations, verify by importing the module: import nemo_switchyard as sw. For the standalone server, run switchyard-server --help to confirm the binary is in your PATH. You can also test the server with switchyard-server --config routes.toml --dry-run to validate your configuration file without starting the service.

Can I use Switchyard on macOS or Windows?

While the repository provides Linux x86-64 and aarch64 wheels for Python, macOS and Windows users can install Switchyard by building from source. Install Rust >= 1.96.1, then run cargo install --locked switchyard-server or build the Python bindings locally using maturin develop.

How do I update Switchyard to the latest version?

Update Python installations with pip install --upgrade nemo-switchyard. For the Rust server or crates, run cargo install --locked switchyard-server again or update the version numbers in your Cargo.toml and run cargo update.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}