# Composite Routing in NVIDIA Switchyard: Combining LLM Classifier and Stage Router Cascades

> Discover how Switchyard integrates LLM classifiers with Stage Router cascades for advanced composite routing. Learn about the TierSetter processor and model pool determination.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: deep-dive
- Published: 2026-09-12

---

**Switchyard implements composite routing by stacking an LLM‑based judge (`LlmTaskClassifier`) over a stage router (`StageRouter`), using a `TierSetter` processor to translate the classifier’s verdict into a routing tier that determines the final model pool.**

NVIDIA NeMo Switchyard is a Rust-based inference routing framework that enables intelligent request distribution through **composite routing** patterns. This architecture chains multiple algorithms—specifically an LLM classifier and a tiered stage router—to dynamically route requests to the most appropriate model based on content analysis. By examining the implementation in [`crates/libsy/src/algorithms/composite.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/composite.rs), we can trace how these components form a cohesive cascade.

## Architecture of the Composite Router

The **Composite Router** defined in [`crates/libsy/src/algorithms/composite.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/composite.rs) orchestrates a three-part pipeline: an LLM classifier acting as a judge, a `TierSetter` processor that bridges decisions to routing state, and a stage router that executes the final selection.

### The Judge (LLM Classifier)

The judge is an instance of `LlmTaskClassifier` implemented in [`crates/libsy/src/algorithms/llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/llm_class.rs). It evaluates incoming requests and returns a **category**—either `Capable` or `Efficient`—indicating which tier of service is required. The classifier is configured via `TaskClassifierConfig` and executes only on specific `ClassifyTrigger` events defined in [`crates/libsy/src/algorithms/util/affinity.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/affinity.rs): `UserTurn` (first user message) or `NewSession` (new routing identity). This prevents the costly LLM evaluation from running on every single request.

### The TierSetter Bridge

`TierSetter` serves as the critical bridge between the judge and the stage router. After the judge returns a category, `TierSetter` extracts the winning verdict, translates it to a corresponding `Tier` enum variant (`Category::Capable` maps to `Tier::Capable`, `Category::Efficient` maps to `Tier::Efficient`), and invokes `set_fall_open` to configure the stage router’s default tier. The implementation caches the selected tier in a `Mutex<HashMap>` keyed by routing identity (session ID), ensuring that subsequent requests in the same session reuse the last verdict without re-invoking the classifier.

### The Stage Router

The final component is the **Stage Router**, constructed from `StageRouterConfig` in [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs). Under normal operation, this router selects models from tier-specific pools. In the composite pattern, the stage router receives its tier assignment from `TierSetter` rather than static configuration, effectively using the classifier’s output as its routing imperative.

## Execution Flow and Processor Stack

The `CompositeRouter` struct wires these components together using a **fall-through** pattern. It implements the `Algorithm` trait and delegates execution to a `FallThrough` processor (defined in [`crates/libsy/src/algorithms/fall_through.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/fall_through.rs)) that chains the judge, `TierSetter`, and stage route into a single execution pipeline.

As shown in the source, the construction looks like this:

```rust
let setter = TierSetter {
    judge,
    trigger,
    message_hash_fallback,
    tiers: Mutex::new(HashMap::new())
};
let route = build_stage_route(config.stage)?
    .with_name("composite")
    .with_processor(Arc::new(setter));
CompositeRouter { route }

```

The `FallThrough` processor ensures sequential execution: the judge classifies, `TierSetter` updates the tier state, and the stage router selects the final model from the appropriate pool.

## Configuring Composite Routing

Switchyard supports both programmatic and declarative configuration for composite routes.

### Programmatic Configuration

To build a composite router in Rust, assemble the `CompositeRouterConfig` with both judge and stage configurations:

```rust
use switchyard_libsy::algorithms::composite::{
    CompositeRouter, CompositeRouterConfig,
};
use switchyard_libsy::algorithms::util::affinity::ClassifyTrigger;
use switchyard_libsy::algorithms::stage::StageRouterConfig;
use switchyard_libsy::algorithms::llm_class::{TaskClassifierConfig, LlmClassifierConfig};

let composite_cfg = CompositeRouterConfig {
    // Judge configuration – runs on the first user turn of a session
    judge: TaskClassifierConfig {
        classify_trigger: ClassifyTrigger::UserTurn,
        message_hash_fallback: true,
        ..Default::default()
    },
    // Stage router configuration – maps tiers to model pools
    stage: StageRouterConfig {
        tier_models: vec![
            (Category::Capable, vec!["strong".into()]),
            (Category::Efficient, vec!["weak".into()]),
        ]
        .into_iter()
        .collect(),
        ..Default::default()
    },
};

let router = CompositeRouter::new(composite_cfg).expect("valid composite router");

```

### TOML Configuration

For deployments using `switchyard-runner`, define the cascade in TOML:

```toml
[targets.classifier]
id = "model/classifier"
type = "llm_classifier"
classifier_target = "judge"

[targets.strong]
id = "model/strong"
type = "llm"

[targets.weak]
id = "model/weak"
type = "llm"

[routes.composite]
id = "switchyard/composite"
type = "composite"

[routes.composite.classifier]
target = "classifier"          # the judge

[routes.composite.stage]
target = "stage"               # the stage router

[routes.stage]
type = "stage"
[routes.stage.tier_capable]
models = ["model/strong"]
[routes.stage.tier_efficient]
models = ["model/weak"]

```

This configuration mirrors the programmatic structure: the composite route references a classifier target (the judge) and a stage target, with the stage router mapping each tier to specific model endpoints.

## Summary

- **Composite routing** in Switchyard chains an `LlmTaskClassifier` (judge) with a `StageRouter` via the `TierSetter` processor, as implemented in [`crates/libsy/src/algorithms/composite.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/composite.rs).
- The judge returns content categories (`Capable` or `Efficient`) which `TierSetter` translates into routing tiers and caches per session.
- **ClassifyTrigger** logic in [`util/affinity.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/util/affinity.rs) limits LLM invocations to `UserTurn` or `NewSession` events, preventing performance overhead on every request.
- The **Stage Router** consumes the tier set by `set_fall_open` to select from tier-specific model pools defined in `StageRouterConfig`.
- Both Rust APIs and TOML configurations support building these cascades, parsed by [`crates/switchyard-runner/src/config.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-runner/src/config.rs).

## Frequently Asked Questions

### What is the role of the TierSetter in Switchyard's composite routing?

The **TierSetter** acts as a stateful bridge between the LLM classifier and the stage router. It receives the judge’s category output, translates it into a `Tier` enum variant, calls `set_fall_open` to configure the stage router’s default tier, and caches this decision per routing identity to avoid redundant classification.

### How does Switchyard prevent the LLM classifier from running on every request?

The implementation uses `ClassifyTrigger` configurations (`UserTurn` or `NewSession`) defined in [`crates/libsy/src/algorithms/util/affinity.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/affinity.rs). The composite router explicitly disallows the `EveryRequest` trigger, ensuring the expensive `LlmTaskClassifier` only evaluates requests at conversation start or session creation.

### Can composite routes be configured without writing Rust code?

Yes. The `switchyard-runner` binary parses composite routing definitions from TOML configuration files, as shown in [`crates/switchyard-runner/src/config.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-runner/src/config.rs). You can define the judge, stage router, and tier mappings declaratively in the configuration file.

### Which Rust trait must the Composite Router implement to integrate with Switchyard?

The `CompositeRouter` implements the **`Algorithm`** trait, allowing it to function as a first-class routing strategy within the Switchyard framework. It delegates actual execution to a `FallThrough` processor that manages the internal cascade.