How to Implement a Custom Algorithm in Switchyard by Extending the Algorithm and Classifier Traits

To implement a custom routing algorithm in NVIDIA-NeMo/Switchyard, create a struct that implements the Algorithm trait by defining the name() and route() methods, optionally implement Classifier for request preprocessing, and register the implementation as an Arc<dyn Algorithm> with the Runner.

Switchyard provides a Rust-based framework for intelligent LLM request routing through two extensible contracts defined in the NVIDIA-NeMo/Switchyard repository. The Algorithm trait orchestrates the routing lifecycle while the Classifier trait enables request inspection and transformation before model selection occurs.

Understanding the Core Traits

The Algorithm Trait

The Algorithm trait in crates/libsy/src/core/algorithm.rs defines the contract for routing logic. You must implement two required methods:

• fn name(&self) -> &str – Returns a static identifier used for telemetry and tracing.
• async fn route(self: Arc<Self>, driver: Driver, request: Request) -> Result<RoutingOutcome> – Contains the core decision logic that selects models and orchestrates calls.

The Algorithm::run_stream method is provided by the trait and handles streaming orchestration; you never need to override it.

The Classifier Trait

The Classifier trait in crates/libsy/src/core/classifier.rs provides a mechanism to score candidate models and optionally rewrite requests before routing. The required signature is:

async fn score<S>(
    &self,
    state: &mut S,
    request: &mut Request,
    driver: &Driver,
) -> Result<(Classification, Option<Response>)>

If the classifier determines the request is already satisfied, it returns a Response in the second tuple element. Otherwise, it returns Classification::Scores(Vec<Score>) for the algorithm to evaluate.

Implementing a Basic Custom Algorithm

Here is a minimal implementation that routes every request to the first available model in the default category:

use std::sync::Arc;
use async_trait::async_trait;
use switchyard_protocol::{Category, Request, Response};
use switchyard_libsy::{Algorithm, Driver, RoutingOutcome, Result};

/// Routes to the first available model in the default category.
pub struct FirstModelAlgo;

#[async_trait]
impl Algorithm for FirstModelAlgo {
    fn name(&self) -> &str {
        "first-model"
    }

    async fn route(
        self: Arc<Self>,
        driver: Driver,
        request: Request,
    ) -> Result<RoutingOutcome> {
        // Retrieve the first candidate for the default category (usually "chat").
        let category = Category::default();
        let model = driver.first_model_for(&category)?.clone();

        // Offload the model call through the driver.
        let response = driver.call_model(request.clone(), vec![model.clone()]).await?;

        // Record observability evidence.
        driver.set_evidence(serde_json::json!({ "selected": model.to_string() }));

        // Return the final outcome.
        Ok(RoutingOutcome::answered(model, request, response))
    }
}

Key implementation details from the source:

• Driver::first_model_for is defined at algorithm.rs:1004.
• RoutingOutcome::answered constructs the final result at algorithm.rs:71.

Adding a Classifier for Request Rewriting

Classifiers enable prompt engineering and pre-processing. The following example injects a system prompt and assigns a confidence score:

use async_trait::async_trait;
use switchyard_protocol::{Request, Response};
use switchyard_libsy::{Classifier, Driver, Classification, Score, Result};

/// Injects a system prompt and scores the "auto" target.
pub struct PromptInjector;

#[async_trait]
impl Classifier<()> for PromptInjector {
    async fn score(
        &self,
        _state: &mut (),
        request: &mut Request,
        _driver: &Driver,
    ) -> Result<(Classification, Option<Response>)> {
        // Mutate the request in-place.
        request.llm_request.messages.insert(
            0,
            switchyard_protocol::Message::system("You are a helpful assistant.".to_string())
        );

        // Assign full confidence to automatic selection.
        let score = Score {
            target: request.model_id().unwrap_or_else(|| "auto".into()),
            confidence: 1.0,
            category: None,
        };
        Ok((Classification::Scores(vec![score]), None))
    }
}

Reference implementations show that Classifier::score accepts mutable state and request parameters, allowing side effects like token counting or content filtering before the algorithm makes final routing decisions.

Composing Classifiers in Your Algorithm

Integrate the classifier by invoking it within your route implementation and handling the Classification result:

use async_trait::async_trait;
use switchyard_libsy::{Algorithm, Driver, RoutingOutcome, Result};

pub struct PromptedAlgo {
    classifier: PromptInjector,
}

#[async_trait]
impl Algorithm for PromptedAlgo {
    fn name(&self) -> &str {
        "prompted"
    }

    async fn route(
        self: Arc<Self>,
        driver: Driver,
        mut request: Request,
    ) -> Result<RoutingOutcome> {
        // Execute classification which may rewrite the request.
        let (classification, maybe_resp) = 
            self.classifier.score(&mut (), &mut request, &driver).await?;

        // Handle early exit if the classifier already answered.
        if let Some(resp) = maybe_resp {
            let target = request.model_id().unwrap_or_else(|| "auto".into());
            return Ok(RoutingOutcome::answered(target, request, resp));
        }

        // Select the highest-scoring target.
        let target = classification
            .argmax(false)?
            .expect("classifier must produce a score")
            .target;

        let response = driver.call_model(request.clone(), vec![target.clone()]).await?;
        driver.set_evidence(serde_json::json!({ "chosen": target.to_string() }));
        Ok(RoutingOutcome::answered(target, request, response))
    }
}

The Classification::argmax utility at classifier.rs:38 simplifies selecting the highest-confidence candidate.

Registering Your Algorithm with the Runner

Expose your implementation to the Switchyard runtime by wrapping it in an Arc and passing it to the Runner:

use switchyard_runner::Runner;
use std::sync::Arc;

fn main() -> anyhow::Result<()> {
    // Instantiate your algorithm.
    let algo = Arc::new(FirstModelAlgo);

    // Load configuration defining model categories.
    let config = std::fs::read_to_string("switchyard-config.toml")?;

    // Initialize the runner with your algorithm.
    let runner = Runner::new(algo, &config)?;

    // Execute the runtime.
    runner.execute()
}

The Runner entry point in crates/switchyard-runner/src/lib.rs handles the HTTP server lifecycle and invokes run_stream for each incoming request.

Summary

• Implement the Algorithm trait by defining name() and route() methods in crates/libsy/src/core/algorithm.rs.
• Use the Driver parameter to query available models via first_model_for() or models_for() and offload calls via call_model().
• Optionally implement Classifier in crates/libsy/src/core/classifier.rs to score targets and rewrite requests before routing.
• Return RoutingOutcome::answered() or RoutingOutcome::route_to() from your route method to complete the request lifecycle.
• Register the algorithm as Arc<dyn Algorithm> with switchyard_runner::Runner to activate it in the serving stack.

Frequently Asked Questions

How do I access the list of available models in my custom algorithm?

Call driver.first_model_for(&category) to retrieve the highest-priority model for a specific category, or use driver.models_for(&category) to obtain the full ordered list of candidates. These methods are defined in crates/libsy/src/core/algorithm.rs and return model identifiers that you can pass to driver.call_model().

Can a classifier modify the request before the algorithm routes it?

Yes. The Classifier::score method receives &mut Request as a parameter, allowing you to insert system prompts, rewrite user messages, or add metadata before the algorithm evaluates routing options. Mutations occur before the algorithm calls driver.call_model().

What is the difference between RoutingOutcome::answered and RoutingOutcome::route_to?

Use RoutingOutcome::answered(target, request, response) when your algorithm or classifier has already obtained a final response from a model. Use RoutingOutcome::route_to(target, request) when you want to delegate the actual model invocation to downstream infrastructure without handling the response in your algorithm logic.

How do I compose multiple classifiers in a single algorithm?

Chain classifiers by invoking them sequentially in your route method or use the cascade pattern demonstrated in crates/prefill-router/src/algorithm.rs. Each classifier can transform the request and return scores; aggregate the Classification results or use Classification::argmax to select the final target after all classifiers have executed.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}