Understanding the target_selector Policy in NVIDIA Switchyard for Custom Multi‑Target Routing

The target_selector policy in NVIDIA Switchyard enables deterministic routing by extracting a category name from a JSON pointer in your classifier verdict and mapping it to registered runtime models, allowing you to implement custom multi-target routing by chaining multiple selectors for primary and fallback model selection.

The target_selector policy in NVIDIA's Switchyard framework provides a mechanism for custom multi-target routing that transforms classifier verdicts into concrete model selections. This policy bridges the gap between high-level classification decisions and low-level model execution by interpreting JSON pointers within verdict documents to route requests to specific model categories.

How the target_selector Policy Works

The target_selector policy operates as a Custom Classifier Policy that performs deterministic model selection based on structured classifier output. Located in crates/libsy/src/algorithms/util/target_selector.rs, the policy executes a three-step resolution process:

1. JSON Pointer Extraction: The policy evaluates a configurable JSON pointer against the classifier verdict to extract a target category string.
2. Category Resolution: The extracted string converts to a Category enum variant (e.g., Category::Capable or Category::Efficient).
3. Model Mapping: The policy queries the Driver for runtime models registered under that category, returning the first available model with confidence 1.0.

If the pointer resolves to an unknown category, Category::Judge, or fails to resolve, the policy returns Classification::Ambiguous, triggering fallback evaluation or default model selection.

Implementing Custom Multi‑Target Routing

To leverage the target_selector policy for routing across multiple models, configure the policy within your runner setup using the CustomClassifierPolicy::TargetSelector variant defined in crates/libsy/src/algorithms/llm_class.rs.

Configuring the JSON Pointer

Initialize TargetSelectorPolicy with a JSON pointer string that targets the category field in your classifier verdict:

use switchyard_libsy::algorithms::util::target_selector::TargetSelectorPolicy;

let selector = TargetSelectorPolicy::new("/decision/target")?;

The pointer /decision/target directs the policy to extract the value from the target field nested within a decision object. You can configure multiple selectors with different pointers to create layered routing strategies.

Registering Models with Categories

Before routing, register your models with the Driver using RuntimeModels to establish category-to-model mappings:

use switchyard_libsy::core::algorithm::Driver;
use switchyard_protocol::Category;

let driver = Driver::new(
    "my_app",
    Arc::new(RuntimeModels::new(
        [
            (Category::Capable, vec![ModelId::from("model/gpt-4")]),
            (Category::Efficient, vec![ModelId::from("model/llama-7b")]),
        ]
        .into(),
    )),
).0;

Chaining Selectors for Fallback Logic

Implement custom multi-target routing by evaluating multiple target_selector policies sequentially. If the primary selector returns Ambiguous, proceed to secondary selectors:

let primary = TargetSelectorPolicy::new("/primary/target")?;
let fallback = TargetSelectorPolicy::new("/fallback/target")?;

let primary_cls = primary.to_classification(Some(&verdict), &driver)?;
if primary_cls.argmax(false)?.is_some() {
    // Route to primary model
} else {
    let fallback_cls = fallback.to_classification(Some(&verdict), &driver)?;
    // Route to fallback model
}

Source Code Architecture

The target_selector policy integrates into Switchyard's routing pipeline through several key components:

• target_selector.rs: Contains the core TargetSelectorPolicy implementation, including the to_classification method that performs the pointer evaluation and category mapping.
• llm_class.rs: Defines the CustomClassifierPolicy enum, which includes the TargetSelector variant used to integrate the policy into runner configurations.
• algorithm.rs (switchyard-runner): Translates runner configuration files into instantiated CustomClassifierPolicy objects, wiring selectors into the routing pipeline.
• runner.rs: Serves as the entry point that constructs the Driver and executes the routing algorithm, applying policies at runtime.

Complete Usage Example

The following example demonstrates end-to-end usage of the target_selector policy for model selection:

use switchyard_libsy::algorithms::util::target_selector::TargetSelectorPolicy;
use switchyard_libsy::core::algorithm::Driver;
use switchyard_protocol::Category;
use serde_json::json;

// Initialize driver with category mappings
let driver = Driver::new(
    "routing_app",
    Arc::new(RuntimeModels::new(
        [
            (Category::Capable, vec![ModelId::from("model/gpt-4")]),
            (Category::Efficient, vec![ModelId::from("model/llama-7b")]),
        ]
        .into(),
    )),
).0;

// Create selector targeting the verdict field
let selector = TargetSelectorPolicy::new("/decision/target")?;

// Simulate classifier verdict
let verdict = json!({ "decision": { "target": "capable" } });

// Resolve to classification
let classification = selector.to_classification(Some(&verdict), &driver)?;

// Verify routing target
assert_eq!(
    classification.argmax(false)?.map(|s| s.target),
    Some(ModelId::from("model/gpt-4"))
);

Summary

• The target_selector policy extracts category names using JSON pointers from classifier verdicts to determine routing targets.
• It queries the Driver for models registered under the resolved Category, returning the first match with confidence 1.0.
• Configure multiple selectors with different JSON pointers to implement primary/fallback routing strategies.
• The policy returns Classification::Ambiguous for invalid pointers or Category::Judge, enabling graceful degradation.
• Integration occurs through CustomClassifierPolicy::TargetSelector in crates/libsy/src/algorithms/llm_class.rs.

Frequently Asked Questions

How does the target_selector policy handle missing or invalid JSON pointers?

When the configured JSON pointer fails to resolve, points to an unknown category string, or resolves to Category::Judge, the policy returns Classification::Ambiguous. This signals the router to evaluate subsequent policies in the chain or fall back to a default model configuration.

Can I use multiple target_selector policies for the same request?

Yes. Configure multiple TargetSelectorPolicy instances with different JSON pointers and evaluate them sequentially. The router proceeds to the next selector if the current one returns Ambiguous, enabling sophisticated multi-tier routing logic across different model categories.

What is the difference between Category::Judge and other categories in target selection?

Category::Judge represents a special variant that the target_selector policy treats as ambiguous. When a JSON pointer resolves to this category, the policy returns Classification::Ambiguous rather than attempting model selection, effectively passing control to the next routing policy or default handler.

Where is the target_selector policy configured in a production Switchyard deployment?

The policy configuration resides in crates/switchyard-runner/src/algorithm.rs, where runner configuration files translate into CustomClassifierPolicy::TargetSelector instances. The runner.rs module then instantiates the Driver with these policies to execute the routing algorithm at runtime.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}