How PickerMode Options Interact with Confidence Thresholds in Switchyard

Switchyard's PickerMode determines the fallback tier when the scorer's confidence falls inside the ambiguous band defined by confidence_threshold, with higher threshold values expanding the band and granting the picker greater control over routing decisions.

NVIDIA-NeMo/Switchyard implements a dual-tier routing system that balances cost-efficient and high-capability compute tiers. The interaction between PickerMode options and confidence_threshold values determines whether a turn routes to the Efficient or Capable tier when the scorer's probability signal is ambiguous. Understanding this relationship is critical for tuning stage router behavior in production workloads.

Understanding PickerMode Variants

Switchyard defines picker modes in crates/libsy/src/algorithms/util/stage.rs to establish default routing behavior when the scorer abstains from making a confident decision.

The PickerMode Enum Definition

The PickerMode enum at lines 100–108 supports two operational strategies. EfficientFirst defaults routing to the cheaper Efficient tier, while CapableFirst defaults to the more powerful Capable tier. Notably, the repository does not implement a weighted variant; only efficient_first and capable_first are supported natively.

Default Tier Assignment

The default_tier method (lines 111–117) maps each picker mode to its respective tier:

• EfficientFirst → returns the Efficient tier as the fallback
• CapableFirst → returns the Capable tier as the fallback

This default only applies when the scorer's confidence falls within the ambiguous band calculated from the threshold.

Confidence Threshold Mechanics

The confidence_threshold controls how much corroboration the scorer needs before it can override the picker’s default tier. This logic resides in the pick_tier function at lines 383–420 of stage.rs.

The Ambiguous Band Calculation

The system defines an ambiguous band centered at probability 0.5 with a width determined by the threshold:

Ambiguous range: 0.5 ± (confidence_threshold / 2)

For example, with the default threshold of 0.5, the ambiguous band spans 0.25 to 0.75. With a stricter threshold of 0.8, the band expands to 0.1 to 0.9, making it harder for the scorer to override the default.

Scorer Resolution Logic

The scorer produces a probability between 0 and 1 representing tool-signal alignment. The resolution follows these rules:

• Probability > 0.5 + (threshold/2) → Routes to Capable tier (scorer overrides)
• Probability < 0.5 - (threshold/2) → Routes to Efficient tier (scorer overrides)
• Probability within the ambiguous band → Uses the PickerMode default tier

Interaction Between PickerMode and Confidence Thresholds

The confidence_threshold directly modulates the picker mode's influence through the ambiguous band width. Higher thresholds expand the ambiguous band, meaning the scorer must achieve extreme certainty (very high or very low probability) to override the picker’s default. Conversely, lower thresholds shrink the band, allowing the scorer to decide the tier with modest confidence and reducing the picker’s impact.

For instance, when using PickerMode::CapableFirst with a threshold of 0.8, the system defaults to the expensive Capable tier unless the scorer reports a probability below 0.1 (forcing Efficient) or above 0.9 (confirming Capable). With a threshold of 0.2, the scorer need only fall outside the 0.4–0.6 range to take control.

Configuring StageRouterConfig

Users wire picker modes to confidence thresholds through StageRouterConfig::new in crates/switchyard-runner/src/config.rs (lines 862–870). This constructor accepts a PickerMode and a threshold value (default 0.5).

// Efficient-first with default threshold (0.5)
// Ambiguous band: 0.25-0.75, defaults to Efficient when uncertain
let cfg = StageRouterConfig::new(PickerMode::EfficientFirst, 0.5);

// Capable-first with high threshold (0.8)
// Ambiguous band: 0.1-0.9, strongly favors Capable unless scorer is certain
let cfg = StageRouterConfig::new(PickerMode::CapableFirst, 0.8);

// Direct pick_tier call with low threshold (0.3)
// Narrow band (0.35-0.65) lets scorer decide with modest confidence
let outcome = pick_tier(&signal, PickerMode::EfficientFirst, 0.3);

Note on the Weighted Picker Mode

While the question references a weighted option, the Switchyard source code does not implement this variant. The PickerMode enum only defines EfficientFirst and CapableFirst. Implementing a weighted strategy would require creating a custom picker that adjusts the scorer’s signal weights prior to the confidence check in pick_tier, or modifying the ambiguous band calculation to apply asymmetric thresholds.

Summary

• PickerMode establishes the fallback tier (Efficient or Capable) when the scorer is uncertain.
• confidence_threshold defines the ambiguous band as 0.5 ± threshold/2; values outside this band trigger scorer-controlled routing.
• Higher thresholds widen the ambiguous band, increasing reliance on the picker’s default tier.
• Configuration occurs via StageRouterConfig::new in crates/switchyard-runner/src/config.rs.
• The weighted picker mode is not implemented in the current codebase.

Frequently Asked Questions

What happens if the scorer returns exactly 0.5?

When the probability is exactly 0.5, the value falls within the ambiguous band for any valid confidence threshold (since the band always spans 0.5 ± threshold/2). Therefore, the system routes to the tier defined by the PickerMode default—Efficient for EfficientFirst and Capable for CapableFirst.

How do I force all turns to use the Efficient tier?

Set PickerMode::EfficientFirst and use a high confidence threshold (e.g., 0.9 or 1.0). This creates a very wide ambiguous band (approximately 0.05 to 0.95 with threshold 0.9), ensuring the scorer almost never overrides the default, keeping virtually all turns on the Efficient tier.

Why does increasing the confidence_threshold give more control to the picker?

Increasing the confidence_threshold mathematically expands the ambiguous band range (0.5 ± threshold/2). A wider band means the scorer’s probability must reach more extreme values (closer to 0 or 1) to escape the ambiguous zone and override the default. Until the scorer achieves that extreme confidence, the PickerMode default maintains control.

Where is the weighted picker mode implemented in Switchyard?

The weighted picker mode is not implemented in the NVIDIA-NeMo/Switchyard repository. The PickerMode enum in crates/libsy/src/algorithms/util/stage.rs only contains EfficientFirst and CapableFirst. To implement weighted behavior, you would need to modify the routing logic to apply custom weights to the scorer’s signals before the confidence check in the pick_tier function.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}