# How to Embed Switchyard Routing Algorithms in a Custom Rust Application

> Embed Switchyard routing algorithms in your Rust app using switchyard-libsy. Integrate routing decisions directly into your service and manage HTTP traffic with ease.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-17

---

**You can embed Switchyard routing algorithms in any Rust service by consuming the `switchyard-libsy` crate, which returns a `StepStream` that yields routing decisions (`Step::CallModel`) your application executes via its own HTTP stack, then feeds responses back through `Algorithm::drive`.**

The NVIDIA-NeMo/Switchyard repository provides provider-neutral LLM traffic orchestration primitives through the `switchyard-libsy` crate. Because the core routing logic is completely decoupled from networking, you can embed sophisticated algorithms—such as `Passthrough`, `Random`, or `StageRouter`—into custom proxies, API gateways, or LLM orchestrators without importing heavy HTTP client dependencies.

## Architecture of the Routing Library

The `switchyard-libsy` crate exposes a **driver pattern** that separates routing intelligence from transport execution. The central types defined in [`crates/libsy/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/lib.rs) are:

- **`Algorithm`** – The routing engine trait (e.g., `Passthrough`, `StageRouter`).
- **`StepStream`** – An async stream produced by `Algorithm::run_stream` that yields routing steps.
- **`Step`** – Individual routing instructions, primarily `Step::CallModel` containing candidate target IDs.
- **`Decision`** – Final routing outcome exposed via `switchyard_protocol::Decision`.

Your host application acts as the driver: it consumes the `StepStream`, performs the actual HTTP calls to LLM endpoints, and pushes `Response` objects back into the algorithm via `stream.drive`. This design lets you integrate Switchyard into existing Rust async runtimes (Tokio, async-std) or custom proxy layers without architectural lock-in.

## Adding switchyard-libsy to Your Project

Add the core crates to your [`Cargo.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/Cargo.toml) along with async utilities:

```toml
[dependencies]
async-trait = "0.1"
futures = "0.3"
tokio = { version = "1", features = ["macros", "rt"] }
switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" }
switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" }

```

The `switchyard-protocol` crate provides provider-neutral request/response contracts (`Request`, `Response`) used by all algorithms.

## Driving Routing Algorithms with Custom Transport

Embedding Switchyard requires implementing a driver loop that bridges the `StepStream` to your HTTP stack. The pattern is consistent across all algorithms:

1. **Instantiate** an `Algorithm` (e.g., `Passthrough::new`).
2. **Invoke** `algorithm.run_stream(request)` to obtain a `StepStream`.
3. **Poll** the stream for `Step::CallModel`, execute the HTTP call to the candidate target, and capture the `Response`.
4. **Drive** the response back into the stream via `stream.drive(response)` to advance the state machine.
5. **Collect** the final `Step::Done` containing the `Decision`.

### Basic Passthrough Routing

The `Passthrough` algorithm (implemented in [`crates/libsy/src/algorithms/passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/passthrough.rs)) demonstrates the minimal integration. It always routes to a single pre-configured target:

```rust
use switchyard_libsy::{Algorithm, Passthrough};
use switchyard_protocol::{Request, Response};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Initialize the algorithm with a static target
    let algo = Passthrough::new("my-model".into());
    
    // Build a provider-neutral request
    let request = Request::new_chat(
        "You are a helpful assistant.".into(),
        vec!["Explain quantum entanglement.".into()],
    );
    
    // Start the routing stream
    let mut stream = algo.run_stream(request);
    
    // Drive the stream
    while let Some(step) = stream.next().await {
        match step {
            switchyard_libsy::Step::CallModel { candidates, .. } => {
                // Host executes HTTP call to the first candidate
                let target = &candidates[0];
                let response: Response = your_http_client_call(target, &step.request).await?;
                
                // Feed response back to advance the algorithm
                stream.drive(response).await?;
            }
            switchyard_libsy::Step::Done { decision, .. } => {
                println!("Routed to: {}", decision.target);
                break;
            }
            _ => {}
        }
    }
    Ok(())
}

```

### Signal-Driven Routing with StageRouter

For dynamic workloads, the `StageRouter` (defined in [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs)) accepts external signals—such as tool usage or progress events—to inform routing decisions. You supply these signals via `ToolSignals` from [`crates/libsy/src/algorithms/util/tool_signals.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/tool_signals.rs):

```rust
use switchyard_libsy::{Algorithm, StageRouter, StageRouterConfig, ToolSignals};
use switchyard_protocol::Request;

// Configure primary and fallback targets
let config = StageRouterConfig {
    primary_target: "coding-agent".into(),
    fallback_target: Some("judge-model".into()),
    ..Default::default()
};

let router = StageRouter::new(config);

// Prepare signal stream (populated by your application logic)
let signals = ToolSignals::default();

// Build request and run stream
let request = Request::new_chat("Refactor this function".into(), vec![]);
let mut stream = router.run_stream(request);

// Drive loop identical to Passthrough, but routing decisions may change
// based on signals injected via router.update_signals()
while let Some(step) = stream.next().await {
    // Handle Step::CallModel and Step::Done as shown above
}

```

## Using the Built-in LLM Client Driver

If you prefer not to implement the driver loop manually, the `switchyard-llm-client` crate provides a drop-in consumer. The `run` function in [`crates/libsy-llm-client/src/run.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/src/run.rs) handles `StepStream` iteration and performs HTTP calls using your configured client:

```rust
use switchyard_libsy_llm_client::run;
use switchyard_libsy::{Algorithm, Passthrough};
use switchyard_protocol::Request;

let algo = Passthrough::new("my-model".into());
let request = Request::new_chat("Tell me a joke".into(), vec![]);

// `run` drives the stream and returns the final Decision
let decision = run(algo, request, /* your http_client */).await?;
println!("Selected model: {}", decision.target);

```

This approach trades flexibility for convenience, abstracting the `Step::CallModel` handling while preserving the algorithmic logic.

## Summary

- **`switchyard-libsy`** provides decoupled routing algorithms that yield `StepStream` objects rather than making HTTP calls directly.
- **Driver implementation** requires polling `Step::CallModel`, executing the network request yourself, and feeding the `Response` back via `stream.drive`.
- **Key source files** include [`crates/libsy/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/lib.rs) (public API), [`crates/libsy/src/algorithms/passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/passthrough.rs) (static routing), and [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs) (signal-driven routing).
- **`switchyard-protocol`** defines provider-neutral `Request` and `Response` structures in [`crates/protocol/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/protocol/src/lib.rs).
- **Optional convenience** is available through `switchyard-llm-client` in [`crates/libsy-llm-client/src/run.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/src/run.rs) for ready-made stream consumption.

## Frequently Asked Questions

### Does switchyard-libsy perform HTTP requests internally?

No. The crate is intentionally transport-agnostic. It yields `Step::CallModel` instructions that your host application must execute. This architecture allows embedding Switchyard into existing proxies or custom networking stacks without pulling in unwanted HTTP dependencies.

### Which routing algorithms are available for embedding?

The library ships with several implementations in `crates/libsy/src/algorithms/`: `Passthrough` for static targets, `Random` for load distribution, and `StageRouter` for signal-driven orchestration. All implement the `Algorithm` trait defined in the crate root.

### How do I feed custom telemetry into the StageRouter?

The `StageRouter` accepts `ToolSignals` (importable from [`crates/libsy/src/algorithms/util/tool_signals.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/tool_signals.rs)). Your application updates this signal container with runtime telemetry—such as token counts, tool invocations, or latency metrics—and the router adjusts its `Step` generation accordingly when you call `router.update_signals()`.

### Can I use Switchyard routing without Tokio?

Yes. While the examples use Tokio, the `Algorithm` trait and `StepStream` rely only on standard `futures::Stream` and `async-trait`. You can drive the stream using any async runtime or even manually poll it in synchronous wrappers, provided you maintain the async `drive` contract for feeding responses back.