How to Implement Custom Routing Algorithms in Switchyard's libsy Library

You implement custom routing algorithms in the libsy library by creating a Rust module that implements the Algorithm trait and optionally the Classifier trait, then composing them with the FallThrough driver to handle model selection and telemetry.

Switchyard's libsy crate provides the core routing infrastructure for LLM request orchestration. To implement custom routing algorithms in the libsy library, you work with the Algorithm trait defined in crates/libsy/src/core/algorithm.rs, which uses a Driver to publish decisions and call models through an async, observable workflow. The library separates classification logic from execution flow, allowing you to mix custom selection strategies with built-in telemetry and error handling.

Understanding the Core Abstractions

The routing system centers on three key components defined in the source code.

Algorithm trait – Located in crates/libsy/src/core/algorithm.rs, this trait requires an async route method that receives a Driver and Request, returning a Response. The Driver provides methods like decide for publishing routing decisions and call_model for offloading work to specific targets.

Classifier trait – Also in the core module, this trait defines a score method that returns a (Classification, Option<Response>) tuple. Classifiers determine which model(s) should handle a request without executing the calls themselves.

FallThrough composition – Found in crates/libsy/src/algorithms/fall_through.rs, this generic orchestrator implements the common "select-target → call-model → decide" workflow. Most built-in algorithms, including the reference Random implementation in crates/libsy/src/algorithms/rand.rs, use FallThrough as their execution engine.

Step-by-Step Implementation

Follow these steps to add a custom routing algorithm to the libsy codebase.

Create a New Module

Add a new file under crates/libsy/src/algorithms/ (for example, my_algo.rs). This keeps your implementation alongside existing algorithms like rand.rs and passthrough.rs.

Implement the Classifier

Create a classifier that determines target selection. The Classifier trait uses async_trait and requires a score method that analyzes the request and returns confidence scores.

use async_trait::async_trait;
use switchyard_protocol::{Decision, ModelId, Request, Response};
use crate::core::classifier::{Classification, Classifier, Score};
use crate::{LibsyError, Result};

/// Classifier that always selects the first available target.
pub struct FirstClassifier;

#[async_trait]
impl<S> Classifier<S> for FirstClassifier
where
    S: Send + 'static,
{
    async fn score(
        &self,
        _state: &mut S,
        _request: &mut Request,
        _driver: Option<&crate::core::algorithm::Driver>,
    ) -> Result<(Classification, Option<Response>)> {
        Ok((Classification::Scores(vec![Score {
            confidence: 1.0,
            target: ModelId::from("first/target"),
        }]), None))
    }
}

Implement the Algorithm Trait

Wrap your classifier in a struct that implements Algorithm. The route method receives the Driver and delegates execution to your logic.

use crate::core::algorithm::{Algorithm, Driver};
use std::sync::Arc;

pub struct FirstAlgo {
    inner: crate::algorithms::fall_through::FallThrough<()>,
}

impl FirstAlgo {
    pub fn new(targets: Vec<ModelId>) -> Result<Self> {
        let classifier = Arc::new(FirstClassifier);
        let inner = crate::algorithms::fall_through::FallThrough::<()>::new(targets)
            .with_name("first_algo")
            .with_classifier(classifier);
        Ok(Self { inner })
    }
}

#[async_trait]
impl Algorithm for FirstAlgo {
    fn name(&self) -> &str {
        "first_algo"
    }

    async fn route(self: Arc<Self>, driver: Driver, request: Request) -> Result<Response> {
        self.inner.execute(driver, request).await
    }
}

Compose with FallThrough

Use FallThrough to avoid reimplementing boilerplate like retries, decision logging, and telemetry. According to the source in crates/libsy/src/algorithms/fall_through.rs, this driver handles the execution loop while your classifier focuses purely on selection logic.

Expose and Test

Register your module in crates/libsy/src/lib.rs:

mod algorithms {
    pub mod my_algo;
    // existing modules...
}
pub use algorithms::my_algo::FirstAlgo;

Run the workspace test suite to verify contract compliance:

cargo test --workspace

Python Integration

To make your algorithm available from Python, add a factory function in switchyard/libsy/algorithms.py:

from switchyard_rust.libsy import my_algo as my_algo_rs

def first_algo(targets):
    """Factory returning the Rust-implemented FirstAlgo."""
    return my_algo_rs.FirstAlgo.new(targets)

Client code can then instantiate your custom router:

from switchyard.libsy import first_algo
from switchyard_protocol import ModelId

algo = first_algo([ModelId("first/target"), ModelId("second/target")])

Testing Your Custom Algorithm

The libsy crate provides a test harness in crate::core::testing that lets you validate algorithms without real LLM endpoints. Use test_drive with the echo mock model:

#[tokio::test]
async fn custom_first_algo_selects_first_target() -> Result<()> {
    use crate::core::testing::{echo, test_drive};
    use switchyard_protocol::{text_request, ModelId};

    let algo = FirstAlgo::new(vec![
        ModelId::from("first/target"), 
        ModelId::from("second/target")
    ])?;
    
    let request = Request {
        llm_request: text_request(Some("auto".to_string()), "hello".to_string()),
        raw_request: None,
        metadata: None,
    };
    
    let (trace, _response) = test_drive(Arc::new(algo), request, echo()).await?;
    assert_eq!(trace[0].selected_model_id(), "first/target");
    Ok(())
}

Because libsy performs no I/O directly, these tests run quickly and deterministically.

Summary

• Implement the Algorithm trait in crates/libsy/src/core/algorithm.rs to define your router's entry point and name.
• Create a Classifier using async_trait to encapsulate target selection logic separate from execution.
• Compose with FallThrough from crates/libsy/src/algorithms/fall_through.rs to reuse telemetry, retries, and decision publishing.
• Expose in lib.rs to make your algorithm importable across the workspace and from Python.
• Test with test_drive using the built-in harness to verify behavior without external LLM dependencies.

Frequently Asked Questions

What is the difference between the Algorithm and Classifier traits in libsy?

The Algorithm trait defines the top-level routing entry point responsible for executing the full request lifecycle, while the Classifier trait focuses solely on scoring and selecting targets. Algorithm implementations use a Driver to call models and publish decisions, whereas Classifier implementations return a Classification struct containing confidence scores. You can mix custom classifiers with existing Algorithm implementations like FallThrough, or implement Algorithm directly for full control over the routing flow.

How does the Driver struct enable custom routing algorithms?

The Driver struct, defined in crates/libsy/src/core/algorithm.rs, provides the bridge between your algorithm and the Switchyard runtime. It offers decide to log routing decisions for telemetry, call_model to emit Step::CallModel events that the host fulfills, and handles the async execution context. Because Driver manages the side effects, your custom algorithm remains pure logic that is easy to test and compose.

Can I implement routing algorithms that call multiple models in sequence?

Yes, the Algorithm trait supports arbitrary execution patterns. While FallThrough implements the common single-target case, you can call driver.call_model multiple times within your route implementation to implement ensemble methods, cascading fallbacks, or parallel routing. Each call returns a Step that you can inspect before proceeding, allowing you to implement complex multi-model workflows while retaining libsy's telemetry and error handling.

Where should I place custom algorithm files in the Switchyard repository?

Place new algorithm implementations in crates/libsy/src/algorithms/ alongside built-in examples like rand.rs and passthrough.rs. Create a new file for your module (e.g., custom_router.rs), then expose it in crates/libsy/src/lib.rs with a pub mod declaration. If you need Python bindings, also update switchyard/libsy/algorithms.py to export a factory function that wraps your Rust struct.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}