What Programming Languages Does Switchyard Support? Native Rust and Python Bindings Explained

Switchyard provides a native Rust implementation with official Python bindings, allowing developers to embed the routing engine directly or use the standalone server via HTTP from any programming language.

Switchyard, an open-source LLM routing framework developed by NVIDIA-NeMo, offers multiple integration paths depending on your programming language requirements. While the core routing engine is implemented in Rust for performance and safety, the project maintains first-class Python bindings that expose identical capabilities to Python developers without requiring Rust code.

Core Implementation in Rust

The foundation of Switchyard is written entirely in Rust, providing memory safety, zero-cost abstractions, and high-performance routing logic for LLM request handling.

The Rust Library (crates/libsy)

The primary Rust implementation lives in the crates/libsy directory, specifically within the switchyard-libsy crate. According to the source code in crates/libsy/README.md, this crate contains all routing algorithms, protocol definitions, and the translation layer required for request processing.

The StageRouter algorithm and supporting routing logic expose a stream-based asynchronous API. You can embed this library directly into Rust applications by adding the dependencies to your Cargo.toml and importing switchyard_libsy::Algorithm along with switchyard_protocol::Request.

The Rust Server (switchyard-server)

For standalone deployments, the switchyard-server crate provides a binary that exposes OpenAI-compatible HTTP endpoints. The server implementation resides in crates/switchyard-server/src/lib.rs, allowing the routing engine to run as a separate service that any HTTP-capable client can access.

First-Class Python Support

Switchyard provides official Python bindings that wrap the Rust core, enabling Python developers to utilize the same routing algorithms through idiomatic Python code.

Python Bindings Architecture

The Python integration is implemented in crates/switchyard-py/src/lib.rs, which creates the _switchyard_rust native extension module. This thin wrapper translates between Python objects and Rust structs, exposing the switchyard.libsy package to Python users. The bindings maintain API parity with the Rust implementation, including the stage_router algorithm and the step-based streaming interface documented in the repository's README.md.

Embedding in Python Applications

Python developers can install Switchyard directly from source using pip with the Rust toolchain available. The Python API mirrors the Rust interface, allowing you to construct StageRouter instances and process requests through an async iterator pattern. Example usage is documented in the "Path 2 — Embed the Library" section of the main README.md.

Universal Access via HTTP

While Rust and Python offer native client libraries for embedded use cases, Switchyard's server speaks standard OpenAI-compatible HTTP endpoints. This architectural choice means any programming language capable of making HTTP requests—such as JavaScript, Go, Java, or C#—can interact with Switchyard as a proxy. However, deep integration that embeds the routing engine directly into an application process is limited to Rust and Python.

Implementation Examples

Embedding the Rust Library

To embed Switchyard in a Rust application, add the dependencies to your Cargo.toml:

// Cargo.toml
[dependencies]
switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git", branch = "main" }
switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git", branch = "main" }
tokio = { version = "1", features = ["macros", "rt"] }

Then implement the routing logic in your application:

use switchyard_libsy::Algorithm;
use switchyard_libsy::StageRouter;
use switchyard_protocol::Request;

#[tokio::main]
async fn main() {
    // Build a stage-router algorithm
    let algorithm = StageRouter::new(
        "capable".into(),
        "efficient".into(),
        "efficient_first",
        0.5,
    );

    // Normalized request dict
    let request = Request {
        model: "switchyard".into(),
        messages: vec![],
        ..Default::default()
    };

    // Run the algorithm - it yields steps that the host must execute
    let mut stream = algorithm.run_stream(request);
    while let Some(step) = stream.next().await {
        match step {
            Step::CallModel(call) => { /* Host makes the actual LLM call here */ }
            Step::Done(outcome) => { /* Process final result */ }
        }
    }
}

Using the Python Bindings

Install from source and import the library:

# pip install git+https://github.com/NVIDIA-NeMo/Switchyard.git

from switchyard.libsy import LlmResponse, Step
from switchyard.libsy.algorithms import stage_router

# Build the algorithm

algorithm = stage_router(
    "capable",
    "efficient",
    picker="efficient_first",
    confidence_threshold=0.5,
)

async def route(request: dict, clients: dict) -> LlmResponse.Agg | LlmResponse.Stream:
    async for step in algorithm.run_stream(request):
        match step:
            case Step.CallModel(call):
                # Host makes the real model call and returns a normalized response

                call.respond(await clients[call.models[0]].call(call.request))
            case Step.Done(outcome):
                return outcome.response or await clients[outcome.selected_model_ids[0]].call(outcome.request)

Summary

• Switchyard's core routing engine is implemented in Rust within crates/libsy and crates/switchyard-server, as detailed in crates/libsy/README.md.
• Python bindings are provided through the switchyard-py crate, exposing the Rust API via crates/switchyard-py/src/lib.rs as the _switchyard_rust module.
• While any programming language can interact with Switchyard through its OpenAI-compatible HTTP server, native embedded support requiring direct library integration is limited to Rust and Python.
• The StageRouter algorithm and streaming request processing are available in both languages with minimal performance overhead in the Python bindings.

Frequently Asked Questions

Can I use Switchyard with JavaScript or TypeScript?

While there are no native JavaScript bindings, you can interact with Switchyard through its HTTP server, which exposes OpenAI-compatible endpoints at crates/switchyard-server/src/lib.rs. This allows Node.js applications to use Switchyard as a proxy without native integration, though you cannot embed the routing engine directly in a JavaScript process.

Is Rust required to install the Python package?

Yes, installing the Python bindings from source requires a Rust toolchain because the switchyard-py crate compiles Rust code into a native Python extension module. The Rust compiler builds the core library found in crates/libsy and exposes it through the FFI layer defined in crates/switchyard-py/src/lib.rs.

Which programming language provides the best performance for Switchyard?

Rust offers the native performance characteristics of the core library with zero overhead. The Python bindings introduce minimal overhead through FFI calls to the Rust core, making both suitable for high-throughput scenarios, though Rust eliminates the Python GIL constraints for maximum concurrency when processing multiple routing streams simultaneously.

Where can I find the API documentation for the Rust implementation?

The Rust library documentation is located in crates/libsy/README.md, which details the StageRouter algorithm, the Algorithm trait, and embedding patterns. The server implementation details and HTTP interface code are available in crates/switchyard-server/src/lib.rs for developers examining the proxy capabilities.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}