# How to Implement Custom Routing Algorithms in the libsy Crate: A Complete Guide

> Implement custom routing algorithms in the libsy crate by defining the Algorithm trait and using the Driver to manage decisions and model calls. Unlock advanced routing capabilities today.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**Implement a custom routing algorithm in the libsy crate by implementing the `Algorithm` trait in [`crates/libsy/src/core/algorithm.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/core/algorithm.rs), using the `Driver` to emit decisions and offload model calls.**

The **libsy crate** provides the core routing engine for NVIDIA's Switchyard framework. Whether you're building affinity-based routing, model hedging, or custom heuristics, understanding how to implement custom routing algorithms requires deep familiarity with the `Algorithm` trait and its asynchronous `Driver` interface.

## Understanding the Algorithm Trait

The foundation of custom routing is the **`Algorithm`** trait defined in [`crates/libsy/src/core/algorithm.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/core/algorithm.rs). This trait represents an asynchronous, `Arc`-compatible object that transforms a `Request` into a `Response` through interaction with a host-provided `Driver`.

### Required Methods

Your implementation must provide two core methods:

- **`fn name(&self) -> &str`** — Returns a low-cardinality identifier used for tracing and observability.
- **`async fn route(self: Arc<Self>, driver: Driver, request: Request) -> Result<Response>`** — Contains your complete routing logic.

The trait's default **`run_stream`** implementation (located at line 76 in [`algorithm.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/algorithm.rs)) wraps your `route` method with panic-catching and task spawning, guaranteeing a terminal `Step::Done` or `Step::Error` even if your algorithm panics.

### Thread Safety Requirements

Your `Algorithm` implementation must be **thread-safe** (`Send + Sync`). A single `Arc`-wrapped instance can be shared across many concurrent requests, making proper synchronization essential for any shared mutable state.

## The Driver Interface: Your Path to Host Interaction

The **`Driver`** struct (defined at line 91 in [`algorithm.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/algorithm.rs)) serves as your algorithm's only channel to the outside world. Each request receives a fresh `Driver` via `Driver::new` (line 100), ensuring isolation of step channels.

### Key Driver Methods

| Method | Signature | Purpose |
|--------|-----------|---------|
| `call_model` | `async fn call_model(&self, request: Request, models: Vec<ModelId>, is_answer_call: bool) -> Result<Response>` | Publish a model call and await the host's fulfillment |
| `decide` | `async fn decide(&self, decision: Decision) -> Result<()>` | Emit a routing decision recorded in the trace |

Use **`driver.decide`** to publish intermediate routing choices for observability. Use **`driver.call_model`** to offload actual model inference, setting `is_answer_call` to `true` for the final answer-generating call.

## Building Your First Custom Algorithm

Follow these concrete steps to implement custom routing algorithms in libsy.

### Step 1: Create Your Algorithm Struct

Define a struct holding any static configuration or shared state:

```rust
// File: crates/libsy/src/algorithms/first_target.rs
use async_trait::async_trait;
use std::sync::Arc;
use switchyard_protocol::{Decision, ModelId, Request, Response};
use crate::{driver::Driver, Algorithm, Result};

pub struct FirstTarget {
    targets: Vec<ModelId>,
}

```

### Step 2: Implement the Algorithm Trait

Provide the required trait implementation:

```rust
impl FirstTarget {
    pub fn new(targets: Vec<ModelId>) -> Self {
        Self { targets }
    }
}

#[async_trait]
impl Algorithm for FirstTarget {
    fn name(&self) -> &str {
        "first_target"
    }

    async fn route(
        self: Arc<Self>,
        driver: Driver,
        request: Request,
    ) -> Result<Response> {
        let first = self
            .targets
            .first()
            .ok_or(crate::LibsyError::NoTargets)?
            .clone();

        let decision = Decision::new(first.clone(), true);
        driver.decide(decision).await?;

        driver.call_model(request, vec![first], true).await
    }
}

```

### Step 3: Register and Expose (Optional)

To expose your algorithm via the public API, add a factory function in [`crates/libsy/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/lib.rs):

```rust
// In crates/libsy/src/lib.rs
pub mod algorithms;
pub use algorithms::first_target::FirstTarget;

```

## Advanced Patterns: Concurrency and Hedging

The libsy crate supports sophisticated routing patterns through its concurrency model. The driver tracks parallel calls via a **`FuturesUnordered`** pool (see the `drive` function at line 31 in [`algorithm.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/algorithm.rs)).

### Implementing Model Hedging

Spawn multiple model calls concurrently and await the first successful result:

```rust
use futures::stream::{FuturesUnordered, StreamExt};

async fn route(
    self: Arc<Self>,
    driver: Driver,
    request: Request,
) -> Result<Response> {
    let mut futures = FuturesUnordered::new();
    
    for model in &self.targets {
        let driver_clone = driver.clone();
        let req = request.clone();
        let model = model.clone();
        
        futures.push(async move {
            driver_clone.call_model(req, vec![model], true).await
        });
    }
    
    // Return first successful response
    while let Some(result) = futures.next().await {
        if let Ok(response) = result {
            return Ok(response);
        }
    }
    
    Err(crate::LibsyError::AllTargetsFailed)
}

```

### Protecting Shared State

Use `Arc<Mutex<_>>` or `parking_lot::RwLock` for mutable state shared across requests:

```rust
use parking_lot::RwLock;
use std::collections::HashMap;

pub struct CachedAlgorithm {
    cache: Arc<RwLock<HashMap<String, ModelId>>>,
}

```

## Testing Your Custom Algorithm

The libsy crate provides robust testing utilities in [`crates/libsy/src/core/testing.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/core/testing.rs).

### Using the test_drive Helper

```rust
// File: crates/libsy/src/algorithms/first_target_tests.rs
#[cfg(test)]
mod tests {
    use super::*;
    use crate::core::testing::{test_drive, echo};
    use switchyard_protocol::{text_request, ModelId};

    fn make_algo() -> Arc<dyn Algorithm> {
        Arc::new(FirstTarget::new(vec![ModelId::from("demo-model")]))
    }

    #[tokio::test]
    async fn selects_first_target() -> Result<()> {
        let algo = make_algo();
        let request = Request {
            llm_request: text_request(
                Some("auto".into()), 
                "hello".into()
            ),
            raw_request: None,
            metadata: None,
        };
        
        let (trace, response) = test_drive(algo, request, echo()).await?;
        
        assert_eq!(trace[0].selected_model_id(), "demo-model");
        assert_eq!(
            response.llm_response.as_agg().unwrap().completion_text(),
            "demo-model"
        );
        Ok(())
    }
}

```

The **`echo`** helper returns the model name as the response text, making verification straightforward.

## Key Source Files Reference

| File | Purpose |
|------|---------|
| [`crates/libsy/src/core/algorithm.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/core/algorithm.rs) | `Algorithm` trait, `Driver`, `CallModel`, `Step`, `drive` loop |
| [`crates/libsy/src/algorithms/passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/passthrough.rs) | Minimal forwarding reference implementation |
| [`crates/libsy/src/algorithms/rand.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/rand.rs) | Random target selection with state handling |
| [`crates/libsy/src/algorithms/util/target_selector.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/target_selector.rs) | Reusable target selection utilities |
| [`crates/libsy/src/core/testing.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/core/testing.rs) | `test_drive`, `echo`, and test harness helpers |
| [`crates/libsy/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/lib.rs) | Public API re-exports |

## Summary

- Implement the **`Algorithm`** trait in [`crates/libsy/src/core/algorithm.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/core/algorithm.rs) to create custom routing logic.
- Use the **`Driver`** interface to emit decisions via `decide()` and offload model calls via `call_model()`.
- Ensure **thread safety** (`Send + Sync`) since algorithms run under `Arc` across concurrent requests.
- Leverage **built-in testing** with `test_drive` and `echo` from [`crates/libsy/src/core/testing.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/core/testing.rs).
- Study reference implementations in **[`passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/passthrough.rs)** and **[`rand.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/rand.rs)** for patterns on state management and randomization.
- Add custom algorithms without Python changes—`libsy` is deliberately minimal with all routing logic in Rust.

## Frequently Asked Questions

### What is the minimum code needed to implement a custom routing algorithm?

Define a struct implementing the `Algorithm` trait with `name()` and `async fn route()`. The `route` method receives a `Driver` and `Request`, uses `driver.decide()` to record routing choices, and returns the result of `driver.call_model()`. See [`crates/libsy/src/algorithms/passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/passthrough.rs) for the minimal working example.

### How does the Driver handle concurrent model calls?

The `Driver` tracks pending calls in a `FuturesUnordered` pool via the internal `drive` function at line 31 of [`algorithm.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/algorithm.rs). You can spawn multiple `call_model` futures concurrently; the driver manages their lifecycle and emits corresponding `Step` items to the host.

### Can I share state between different requests to the same algorithm?

Yes, but you must protect shared mutable state with synchronization primitives like `Arc<Mutex<_>>` or `parking_lot::RwLock`. The `Algorithm` trait requires `Send + Sync`, and each request holds an `Arc` clone of your algorithm instance.

### How do I test my custom algorithm without a full Switchyard deployment?

Use the `test_drive` helper from [`crates/libsy/src/core/testing.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/core/testing.rs). This function runs your algorithm against a mock host, returning the complete trace and final response. The `echo` helper simulates a model that returns its own ID, enabling verification of routing decisions.

### Where should I place my custom algorithm files?

Create new files under `crates/libsy/src/algorithms/` following the existing naming convention ([`my_algo.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/my_algo.rs) for implementation, [`my_algo_tests.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/my_algo_tests.rs) for tests). Reference implementations in [`passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/passthrough.rs) and [`rand.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/rand.rs) demonstrate the expected structure and patterns.