How the Escalation Router Mode Functions in Switchyard's LLM Classifier Strategy

The escalation router mode is a specialized configuration of Switchyard's LLM-task-classifier algorithm that triggers tier escalation based on tool-signal failures or judge verdicts, with configurable confirmation thresholds and contextual hand-off notes.

Switchyard's routing system optimizes cost and latency by directing requests to an efficient tier (cheaper, faster models) by default, escalating to a capable tier (more powerful models) only when necessary. The escalation router mode provides deterministic, signal-driven escalation powered by the LLM-classifier strategy. This article explains the complete implementation based on the NVIDIA-NeMo/Switchyard source code.

Core Components of Escalation Router Mode

Four primary structures work together to enable escalation routing:

Component	Purpose	Source Location
EscalationJudgeConfig	Defines escalation thresholds: confirmations, recent_turn_window, window_message_chars	crates/libsy/src/algorithms/util/escalation.rs
EscalationJudge	Structured judge that evaluates condensed transcripts and returns boolean escalate verdicts	Same file
StageRouterConfig	Route configuration holding handoff_notes and optional llm_fallback capability judge	crates/libsy/src/algorithms/stage.rs
stage_router (Python façade)	Python interface exposing escalation_note, deescalation_note, and only_on_wrong_signal_escalation parameters	switchyard_rust/libsy.py

Server-side TOML configurations activate this mode through mode = "escalation" declarations, automatically wiring the judge into the routing cascade in crates/switchyard-server/src/config.rs.

The Escalation Decision Flow

The escalation router mode operates through a five-stage pipeline:

1. Tool-Signal Processing

The ToolSignalProcessor examines recent tool results—such as failed bash commands—and emits a binary signal (true for failure, false otherwise). This provides immediate, deterministic escalation triggers without LLM latency.

2. Signal-Driven Escalation

When the signal exceeds the confidence_threshold configured in StageClassifier, the turn immediately escalates to the capable tier. This bypasses probabilistic classification for clear failure cases.

3. Escalation Note Attachment

Signal-driven escalations include the user-provided escalation_note appended to the request sent to the capable model. This note is omitted for normal turns, with behavior controlled by only_on_wrong_signal_escalation (default true).

4. Optional Fallback Judge

If signals don't force a decision, the router invokes the Escalation Judge—an LlmTaskClassifier operating in Capability mode. The judge receives a condensed transcript built by summarize_for_judge in escalation.rs and returns a verdict.

5. De-escalation Handling

When the judge decides to de-escalate (keeping the turn on the efficient tier after prior escalation), an optional deescalation_note returns to the efficient model to maintain context continuity.

Configuring Escalation Router Mode

TOML Server Configuration

# routes.toml

[routes.my_escalation_route]
id = "my_escalation_route"
mode = "escalation"
escalation = { confirmations = 2, recent_turn_window = 5, window_message_chars = 2000 }

The confirmations parameter requires two consecutive escalation verdicts before permanent tier switching, preventing flapping. The server automatically creates an EscalationJudgeConfig from this declaration and injects it into the routing cascade.

Python API Configuration

from switchyard_rust import stage_router

router = stage_router(
    capable_target="gpt-4-capable",
    efficient_target="gpt-4-efficient",
    picker="efficient_first",
    confidence_threshold=0.5,
    escalation_note="the previous model stalled – please diagnose",
    deescalation_note="thanks for the fix, continue as efficient",
    only_on_wrong_signal_escalation=True,  # default: attach note only on signal-driven escalation

)

This configuration wires the EscalationJudge when escalation_note is supplied, alongside the optional LlmFallback capability classifier.

Inside the Rust Implementation

EscalationJudgeConfig Validation

The configuration enforces sensible bounds in escalation.rs (lines 60-77):

fn validate(&self) -> Result<()> {
    // confirmations must be ≥ 1
    // recent_turn_window must be non-zero
    // window_message_chars must be ≥ 50
    ...
}

Verdict-to-Classification Mapping

The EscalationPolicy::to_classification method translates boolean verdicts into routing decisions:

match verdict {
    Some(v) if v.escalate => Classification::Scores(vec![
        Score { target: self.capable, score: 1.0 }
    ]),
    Some(_) => Classification::Scores(vec![
        Score { target: self.efficient, score: 1.0 }
    ]),
    None => Classification::Ambiguous(Vec::new()),
}

StageRouter Assembly

In build_route within stage.rs, the cascade conditionally adds the escalation judge:

if let Some(fallback) = config.llm_fallback {
    router = router.with_classifier(Arc::new(SourceStamp {
        inner: Arc::new(LlmTaskClassifier::new(
            LlmClassifierConfig::Capability { ... }
        )?),
        source: DecisionSource::LlmClassifier,
    }));
}

Integration Test: Signal-Driven Escalation

The repository's test suite verifies note attachment behavior in crates/libsy/src/algorithms/stage.rs (lines 66-89):

#[tokio::test]
async fn a_signal_driven_escalation_hands_the_note_to_the_model() -> Result<()> {
    let recorder = Arc::new(Recorder::default());
    let router = recording_router(config_with_notes())?;

    // First turn: no signal, stays on efficient tier
    test_drive(router.clone(), turn_request(false), recorder.serve()).await?;
    // Second turn: signal triggers escalation
    test_drive(router.clone(), turn_request(true), recorder.serve()).await?;

    let calls = recorder.routed();
    assert_eq!(calls[0].target, "weak");   // efficient, no note
    assert_eq!(calls[1].target, "strong"); // capable, note attached
    assert!(calls[1].messages.last().unwrap().ends_with(ESCALATION));
    Ok(())
}

This test confirms that escalation notes appear only on escalated turns, not on standard efficient-tier routing.

Why Use Escalation Router Mode in LLM Classifier Strategy?

Three primary advantages drive adoption:

• Safety Net: Tool-signal failures trigger immediate capable-tier escalation without waiting for probabilistic classification—critical for coding-agent workloads where compilation errors demand immediate powerful model intervention.

• Signal-Driven Context: The escalation_note parameter provides actionable context ("previous model stalled") directly to the capable model, improving recovery success rates.

• Configurable Stability: The confirmations parameter prevents tier flapping by requiring sustained failure patterns before permanent escalation, balancing responsiveness with stability.

Summary

Switchyard's escalation router mode extends the LLM-classifier strategy with deterministic, signal-driven tier management:

• Triggered by ToolSignalProcessor failures or EscalationJudge verdicts
• Configured via EscalationJudgeConfig with confirmations, recent_turn_window, and window_message_chars
• Activated in TOML through mode = "escalation" or Python through stage_router parameters
• Supports contextual hand-off through escalation_note and deescalation_note
• Validated in escalation.rs and wired into routing cascades in stage.rs

Frequently Asked Questions

What is the difference between escalation router mode and standard LLM classification?

Standard LLM classification routes based on task capability estimation alone. Escalation router mode adds a dedicated EscalationJudge that can be triggered by tool-signal failures, providing deterministic escalation paths independent of capability scoring. According to the Switchyard source, this mode specifically handles "wrong signal" scenarios where efficient-tier models fail operational constraints.

How does the confirmations parameter prevent flapping?

The confirmations parameter in EscalationJudgeConfig requires consecutive escalation verdicts before permanently switching to the capable tier. For example, confirmations = 2 means two separate escalation triggers must occur in the recent_turn_window before the router commits to the capable tier. This filters transient failures from persistent capability mismatches.

When should I use only_on_wrong_signal_escalation = false?

Set only_on_wrong_signal_escalation = false when you want the escalation_note attached to all capable-tier turns, not just signal-driven escalations. This is useful when the capable model needs consistent context about its role, regardless of how it was selected. The default true value optimizes token usage by attaching notes only when explicitly needed for recovery.

Can escalation router mode de-escalate from capable to efficient tier?

Yes. When the EscalationJudge returns a negative verdict on a turn that previously escalated, the router can trigger de-escalation—returning to the efficient tier while optionally transmitting the deescalation_note to maintain conversational continuity. This enables dynamic tier management rather than one-way escalation ladders.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}