# How the LLM Classifier in NVIDIA Switchyard Differentiates Between Strong and Weak Tiers

> Learn how the NVIDIA Switchyard LLM classifier routes requests between strong and weak tiers using a configurable solve probability threshold. Optimize your model's efficiency today.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: deep-dive
- Published: 2026-08-15

---

**The LLM classifier in Switchyard differentiates between "strong" (capable) and "weak" (efficient) tiers by comparing a judge model's solve probability (`p_solve`) against a configurable threshold, routing to the weak tier when the probability meets or exceeds that threshold and to the strong tier otherwise.**

Switchyard is NVIDIA's open-source routing framework for LLM inference. At its core sits the `LlmTaskClassifier` algorithm, which makes intelligent routing decisions to optimize the cost-performance trade-off. This article examines how the classifier determines which tier—weak (efficient) or strong (capable)—should handle each incoming request based on the actual implementation in the Switchyard source code.

## Classifier Operating Modes

The `LlmTaskClassifier` supports two distinct modes for tier selection, each implementing the `routing_tier` method to label requests.

### Capability Mode

In **capability mode**, a dedicated judge model evaluates the task upfront before any inference occurs. The judge returns a `TaskClassifierVerdict` containing:

- `p_solve`: The estimated probability that the efficient model can successfully complete the task
- A capability boundary (`supported`, `uncertain`, `unsupported`, or `unmatched`)
- Rule identifiers for traceability

### Escalation Mode

In **escalation mode**, the system attempts inference with the efficient model first. A judge then evaluates whether that response is sufficient or if the request must "escalate" to the capable model. This mode incurs the latency of potentially two model calls but avoids judge overhead for simple requests that the efficient model handles well.

## How the Routing Tier Is Determined

Both classifier variants implement `routing_tier` to return static string labels based on which model ID was selected. The implementation in [`crates/libsy/src/algorithms/llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/llm_class.rs) reveals the mapping logic.

For **capability mode** (`TaskClassifier`):

```rust
fn routing_tier(&self, selected_model_id: &ModelId) -> Option<&'static str> {
    if self.efficient_target == self.capable_target {
        None
    } else if *selected_model_id == self.efficient_target {
        Some("weak")
    } else if *selected_model_id == self.capable_target {
        Some("strong")
    } else {
        None
    }
}

```

For **escalation mode** (`EscalationClassifier`):

```rust
fn routing_tier(&self, selected_model_id: &ModelId) -> Option<&'static str> {
    if self.capable == self.efficient {
        None
    } else if *selected_model_id == self.capable {
        Some("strong")
    } else if *selected_model_id == self.efficient {
        Some("weak")
    } else {
        None
    }
}

```

Note that when both targets reference the same model, no tier distinction exists and `None` is returned.

## The Threshold-Based Decision Logic

The core differentiation mechanism lives in `TaskClassifierPolicy::to_classification`. The policy computes a dynamic threshold from the verdict's capability boundary, then selects the target by comparing `p_solve` against that threshold.

### Threshold Computation

```rust
fn threshold(&self, verdict: &TaskClassifierVerdict) -> Option<f64> {
    Some(self.base_threshold + f64::from(verdict.boundary_steps()?) * self.threshold_step)
}

```

The threshold increases for less certain boundaries:
- **Supported** tasks use `base_threshold` directly
- **Uncertain** tasks add one `threshold_step`
- **Unsupported** tasks add two `threshold_step` values

### Target Selection

```rust
let target = if verdict.p_solve >= threshold
    || (threshold - verdict.p_solve).abs() <= f64::EPSILON {
    &self.efficient_target
} else {
    &self.capable_target
};

```

This logic appears in [`crates/libsy/src/algorithms/llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/llm_class.rs) at lines 1010-1016. The epsilon comparison ensures floating-point stability at equality boundaries.

## Configuration Parameters

The classifier's behavior is controlled through `TaskClassifierConfig`, defined at lines 26-34 of [`llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/llm_class.rs):

```rust
pub struct TaskClassifierConfig {
    pub base_threshold: f64,
    pub threshold_step: f64,
    ...
}

```

| Parameter | Purpose | Typical Effect |
|-----------|---------|--------------|
| `base_threshold` | Minimum solve probability for weak-tier routing of supported tasks | Lower values increase weak-tier usage |
| `threshold_step` | Penalty added per uncertainty level | Higher values more aggressively promote uncertain tasks to strong tier |

Lowering `base_threshold` from 0.5 to 0.3, for example, increases traffic to the efficient model but risks more failed completions.

## Practical Implementation Examples

### Python: Configuring a Capability-Based Classifier

```python
from switchyard.libsy import algorithms as alg
from switchyard.libsy import LlmClassifierConfig, TaskClassifierConfig

cfg = TaskClassifierConfig(
    base_threshold=0.5,      # 50% solve prob needed for weak tier

    threshold_step=0.1,      # extra slack for uncertain/unsupported tasks

    session_affinity=False,
)

classifier = alg.llm_task_classifier(
    LlmClassifierConfig.Capability(
        judge_target="judge",
        efficient_target="efficient",   # weak tier

        capable_target="capable",       # strong tier

        config=cfg,
    )
)

```

The factory function resides in [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py).

### Rust: Direct Classifier Instantiation

```rust
let classifier = LlmTaskClassifier::new(
    LlmClassifierConfig::Capability {
        judge_target: ModelId::from("judge"),
        efficient_target: ModelId::from("efficient"),
        capable_target: ModelId::from("capable"),
        config: TaskClassifierConfig {
            base_threshold: 0.6,
            threshold_step: 0.0,
            ..Default::default()
        },
    },
)?;

```

Construction logic appears in [`llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/llm_class.rs) at lines 650-660.

### CLI Integration

```python
from switchyard import launch_command

launch_command.run([
    "claude",
    "--model", "my-route",
    "--config", "routes.toml",
])

```

The CLI entry point is [`switchyard/cli/switchyard_cli.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/switchyard_cli.py).

## Key Implementation Files

| File | Role |
|------|------|
| [`crates/libsy/src/algorithms/llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/llm_class.rs) | Core `LlmTaskClassifier`, `TaskClassifierPolicy`, and `routing_tier` implementations |
| [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py) | Python façade for classifier construction |
| [`switchyard/cli/switchyard_cli.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/switchyard_cli.py) | CLI entry point for pipeline execution |

## Summary

- **Two modes**: Capability mode uses upfront judge evaluation; escalation mode tries efficient inference first with fallback
- **Threshold mechanism**: `base_threshold` plus `threshold_step` per uncertainty level determines the cutoff for weak-tier routing
- **Decision rule**: `p_solve >= threshold` selects weak (efficient); otherwise strong (capable)
- **Explicit mapping**: `routing_tier` methods convert selected `ModelId` to `"weak"` or `"strong"` labels for observability
- **Configuration**: `TaskClassifierConfig` exposes tunable parameters without code changes

## Frequently Asked Questions

### How does the classifier handle cases where the efficient and capable models are identical?

When `efficient_target == capable_target`, both `routing_tier` implementations return `None` (lines 888 and 887 in [`llm_class.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/llm_class.rs)). No tier distinction exists since both routing options reference the same underlying model.

### What happens when `p_solve` exactly equals the computed threshold?

The implementation uses `(threshold - verdict.p_solve).abs() <= f64::EPSILON` to handle floating-point equality, routing to the efficient target in this case. This ensures deterministic behavior at boundary conditions.

### Can the same classifier instance switch between capability and escalation modes?

No. The mode is determined at construction time through the `LlmClassifierConfig` enum variant (`Capability` or `Escalation`). Each variant carries distinct fields and instantiates different internal structs (`TaskClassifier` vs. `EscalationClassifier`).

### Where does the judge model itself run?

The judge is treated as another model target referenced by `ModelId`. In the examples, `"judge"` resolves through Switchyard's standard model routing configuration, which may point to a lightweight classifier model or even a rules-based evaluator depending on deployment setup.