# How to Embed Switchyard's Routing Algorithms in a Rust Application Using switchyard-libsy

> Embed Switchyard's routing algorithms in your Rust application using switchyard-libsy. Orchestrate LLM traffic efficiently without HTTP dependencies and integrate seamlessly with any transport layer.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**`switchyard-libsy` is the core Rust crate that exposes provider-neutral routing algorithms as a stream of `Step` values, letting you embed LLM traffic orchestration into any existing transport or proxy layer without pulling in HTTP dependencies.**

Embedding Switchyard's routing algorithms in your Rust application gives you fine-grained control over how LLM requests are distributed across model targets. The `switchyard-libsy` crate, located in the NVIDIA-NeMo/Switchyard repository, implements this logic as a pure algorithmic layer that yields routing decisions rather than performing network calls itself.

## Architecture of switchyard-libsy

The crate follows a strict separation between **routing logic** and **transport execution**. This design lets you plug Switchyard into proxies, gateways, or custom orchestrators without modifying your existing HTTP stack.

| Component | Responsibility | Key Types |
|-----------|---------------|-----------|
| `switchyard-libsy` | Core routing algorithms (`Passthrough`, `Random`, `StageRouter`) | `Algorithm`, `Step`, `StepStream` |
| `switchyard-protocol` | Provider-neutral request/response contracts | `Request`, `Response`, `Decision` |
| `switchyard-llm-client` | Ready-made consumer that drives `StepStream` and performs HTTP calls | `run` function |
| Host Application | Supplies model targets, executes HTTP calls, processes results | — |

The central abstraction is the **`Algorithm`** trait. When you invoke `Algorithm::run_stream`, you receive a **`StepStream`** that yields **`Step`** values. Your application interprets each step—typically calling a model when asked—and feeds the resulting `Response` back via `Algorithm::drive`. The algorithm eventually produces a **`Decision`** recording which target was chosen and why.

## Setting Up switchyard-libsy

Add the crates to your [`Cargo.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/Cargo.toml):

```toml
[dependencies]
async-trait = "0.1"
futures = "0.3"
switchyard-libsy = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" }
switchyard-protocol = { git = "https://github.com/NVIDIA-NeMo/Switchyard.git" }
tokio = { version = "1", features = ["macros", "rt"] }

```

See [`crates/libsy/README.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/README.md) in the repository for the complete dependency list and version compatibility notes.

## Embedding the Passthrough Algorithm

The **`Passthrough`** algorithm in [`crates/libsy/src/algorithms/passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/passthrough.rs) is the simplest routing strategy—it unconditionally forwards all requests to a single model target. This makes it ideal for learning the embedding pattern or for deployments with dedicated model instances.

```rust
use switchyard_libsy::{Algorithm, Passthrough};
use switchyard_protocol::{Request, Response};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // 1. Instantiate the algorithm with a fixed target.
    let algo = Passthrough::new("my-model".into());

    // 2. Build a provider-neutral request.
    let request = Request::new_chat(
        "You are a helpful assistant.".into(),
        vec!["Explain quantum entanglement.".into()],
    );

    // 3. Obtain the step stream from the algorithm.
    let mut stream = algo.run_stream(request);

    // 4. Drive the stream: respond to each step the algorithm emits.
    while let Some(step) = stream.next().await {
        match step {
            switchyard_libsy::Step::CallModel { candidates, .. } => {
                // Passthrough guarantees exactly one candidate.
                let target = &candidates[0];
                
                // Your HTTP call goes here—switchyard-libsy has no opinion on transport.
                let response: Response = call_llm(target, step.request).await?;
                
                // Feed the response back to advance the algorithm.
                stream.drive(response).await?;
            }
            switchyard_libsy::Step::Done { decision, .. } => {
                println!("Routed to: {}", decision.target);
                break;
            }
            _ => {}
        }
    }
    Ok(())
}

async fn call_llm(target: &str, request: Request) -> anyhow::Result<Response> {
    // Your existing HTTP client logic.
    todo!()
}

```

Key types are re-exported from [`crates/libsy/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/lib.rs): `Algorithm`, `Step`, and `StepStream` form the public API surface.

## Using StageRouter for Signal-Driven Routing

The **`StageRouter`** algorithm in [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs) implements more sophisticated routing based on external signals—tool usage, progress indicators, or custom application events. This powers use cases like coding-agent launchers that hand off between specialized models.

```rust
use switchyard_libsy::{
    Algorithm, StageRouter, StageRouterConfig, ToolSignals,
};
use switchyard_protocol::Request;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Configure primary and fallback targets.
    let config = StageRouterConfig {
        primary_target: "coding-agent".into(),
        fallback_target: Some("judge-model".into()),
        ..Default::default()
    };
    let router = StageRouter::new(config);

    // Signals drive routing decisions (e.g., tool execution progress).
    let signals = ToolSignals::default();

    let request = Request::new_chat(
        "Refactor this function to use async/await".into(),
        vec![],
    );

    let mut stream = router.run_stream(request);
    
    while let Some(step) = stream.next().await {
        match step {
            switchyard_libsy::Step::CallModel { candidates, context } => {
                // StageRouter may rank candidates by signal state.
                let target = select_best(candidates, &signals);
                let response = call_llm(target, context.request).await?;
                stream.drive(response).await?;
            }
            switchyard_libsy::Step::Done { decision, .. } => {
                println!("Final routing decision: {:?}", decision);
                break;
            }
            _ => {}
        }
    }
    Ok(())
}

```

Signal utilities live in [`crates/libsy/src/algorithms/util/tool_signals.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/tool_signals.rs). The `ToolSignals` type lets you feed runtime information into the router without coupling your signal sources to the algorithm implementation.

## Driving Routing with the Built-In Client

If you prefer not to implement the `StepStream` driver yourself, the **`switchyard-llm-client`** crate provides a drop-in solution. The `run` function in [`crates/libsy-llm-client/src/run.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/src/run.rs) consumes a `StepStream` and performs HTTP calls using your configured client.

```rust
use switchyard_libsy_llm_client::run;
use switchyard_libsy::{Algorithm, Passthrough};
use switchyard_protocol::Request;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let algo = Passthrough::new("my-model".into());
    let request = Request::new_chat("Tell me a joke".into(), vec![]);

    // `run` handles the stream loop and returns the final Decision.
    let decision = run(algo, request, /* your http_client */).await?;
    println!("Model {} responded successfully", decision.target);
    
    Ok(())
}

```

This approach sacrifices some control for convenience—you lose the ability to interpose custom logic between steps, but gain a working integration in minimal code.

## Key Integration Points

When embedding `switchyard-libsy` in production applications, focus on these implementation locations:

- **[`crates/libsy/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/lib.rs)** — Public API re-exports; start here to understand available algorithms and types.
- **[`crates/libsy/src/algorithms/passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/passthrough.rs)** — Reference implementation for single-target routing.
- **[`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs)** — Complex signal-driven routing with fallback support.
- **[`crates/protocol/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/protocol/src/lib.rs)** — `Request`, `Response`, and `Decision` structures for provider-neutral communication.
- **[`crates/libsy-llm-client/src/run.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/src/run.rs)** — Production-ready driver if you don't need custom step handling.

The algorithm-stream pattern makes `switchyard-libsy` embeddable in any async Rust runtime. Because networking is entirely your responsibility, you retain full control over connection pooling, retry policies, observability hooks, and authentication.

## Summary

- **`switchyard-libsy`** exposes routing algorithms as `StepStream` producers, not HTTP clients.
- **Embed algorithms** by instantiating `Passthrough`, `StageRouter`, or custom `Algorithm` implementations, then driving their `StepStream` with your transport code.
- **Three integration depths**: manual stream driving (full control), `ToolSignals` for dynamic routing, or `switchyard-llm-client::run` for convenience.
- **Zero HTTP dependencies** in the core crate—bring your own `reqwest`, `hyper`, or internal client.
- **Provider-neutral protocol** via `switchyard-protocol` keeps your application decoupled from specific LLM API shapes.

## Frequently Asked Questions

### Does switchyard-libsy make HTTP requests automatically?

No. The crate is intentionally transport-agnostic. It yields `Step::CallModel` values that your application interprets and executes. This design lets you integrate with existing proxy infrastructure, custom connection pools, or specialized authentication layers without modification. The `switchyard-llm-client` crate provides an optional HTTP implementation if you need one.

### How do I choose between Passthrough and StageRouter?

**`Passthrough`** in [`crates/libsy/src/algorithms/passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/passthrough.rs) routes all traffic to a single fixed target—use it for dedicated model deployments or when learning the embedding pattern. **`StageRouter`** in [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs) responds to runtime signals and supports fallback targets—use it for dynamic scenarios like coding agents that may need to hand off between specialized models based on tool usage or progress indicators.

### Can I implement custom routing algorithms?

Yes. Implement the **`Algorithm`** trait from [`crates/libsy/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/lib.rs). Your implementation must provide `run_stream`, which returns a `StepStream` yielding your custom `Step` sequence. The trait design ensures your algorithm remains decoupled from transport concerns while interoperating with existing Switchyard tooling.

### What information does a Decision contain?

The **`Decision`** type from `switchyard-protocol` records the final target model ID, the routing path taken (which algorithms or stages processed the request), and metadata about why that choice was made. Inspect `Decision` after your `StepStream` yields `Step::Done` to log routing telemetry or enforce post-hoc validation policies.