Essential Fields Required in a Switchyard TOML Route Configuration

The two universal fields required for every Switchyard route are id (a unique identifier) and type (the routing algorithm); additional fields depend on the route type.

Switchyard, NVIDIA's LLM request routing engine, uses TOML configuration files to map incoming requests to backend models. Understanding the Switchyard TOML route configuration schema is essential for deploying intelligent load balancing, cost optimization, or capability-based routing in production environments.

Core Fields: Required for All Route Types

Every route definition in the NVIDIA-NeMo/Switchyard repository begins with two mandatory keys under a [routes.<name>] table:

• id — A globally unique route identifier used by the server for lookups (e.g., "switchyard/random", "myapp/passthrough")
• type — The routing algorithm selector that determines request dispatch behavior

These fields appear in every valid configuration. The dev-server/config.toml file demonstrates this pattern across multiple route definitions:

[routes.random]
id = "switchyard/random"
type = "random"

[routes.passthrough]
id = "switchyard/passthrough"
type = "passthrough"

Similarly, switchyard/cli/defaults/openrouter.toml configures a production llm_classifier route using only these core fields plus type-specific additions.

Route Type-Specific Required Fields

After declaring id and type, you must provide additional fields based on the selected routing algorithm. The following sections detail the essential fields required in a Switchyard TOML route configuration for each supported type.

Random Routes

Required field: targets

The random type performs uniform selection from a list of backend models. The targets array must reference target names defined elsewhere in your TOML.

[routes.cost_optimizer]
id = "prod/random-selection"
type = "random"
targets = ["oss20b", "dsflash", "mixtral-8x7b"]

Example source: dev-server/config.toml lines 21-24.

Passthrough Routes

Required field: target

The passthrough type forwards every request to a single designated model with no selection logic.

[routes.primary_model]
id = "prod/direct-gpt4"
type = "passthrough"
target = "gpt-4-turbo"

Example source: dev-server/config.toml lines 26-28.

Noop Routes

Required fields: None (core fields only)

The noop type serves as a placeholder or static response generator. It requires only id and type:

[routes.health_check]
id = "internal/noop"
type = "noop"

This route type is useful for testing configuration loading without invoking actual model backends.

Stage Router Routes

Required fields: capable_target, efficient_target, picker, confidence_threshold, recent_turn_window

The stage_router implements a two-stage classifier that attempts an efficient (cheaper/faster) model first, falling back to a capable (stronger) model when confidence is insufficient.

Field	Purpose
capable_target	The stronger fallback model
efficient_target	The cheaper initial model
picker	Selection strategy: "efficient_first" or "capable_first"
confidence_threshold	Numeric cutoff for switching (0.0–1.0)
recent_turn_window	Number of recent conversation turns to evaluate

[routes.smart_stage]
id = "prod/two-stage"
type = "stage_router"
capable_target = "gpt-4-turbo"
efficient_target = "mixtral-8x7b"
picker = "efficient_first"
confidence_threshold = 0.45
recent_turn_window = 5

Full example: dev-server/config.toml lines 55-62.

LLM Classifier Routes

Required fields: classifier_target, strong_target, weak_target, base_threshold, session_affinity, message_hash_fallback

The llm_classifier type uses a trained classifier model to predict query complexity and route accordingly—ideal for cost-performance optimization at scale.

Field	Purpose
classifier_target	The model performing capability classification
strong_target	Destination for high-complexity predictions
weak_target	Destination for low-complexity predictions
base_threshold	Confidence threshold for selecting strong_target
session_affinity	Boolean to pin sessions to chosen targets
message_hash_fallback	Boolean to enable hash-based fallback on classifier failure

[routes.intelligent]
id = "prod/llm-classifier"
type = "llm_classifier"
classifier_target = "classifier-mistral-7b"
strong_target = "gpt-4-turbo"
weak_target = "mixtral-8x7b"
base_threshold = 0.6
session_affinity = true
message_hash_fallback = true

Production reference: switchyard/cli/defaults/openrouter.toml lines 23-26 and onward; complete example at dev-server/config.toml lines 69-78.

Configuration Validation Checklist

Before deploying your Switchyard TOML route configuration, verify:

1. Every [routes.*] table contains id and type
2. The type value matches one of: random, passthrough, noop, stage_router, llm_classifier
3. All type-specific required fields are present and correctly typed
4. Target references (target, targets.*, *_target) resolve to defined target configurations

Invalid configurations will typically fail at server startup with schema validation errors in the Switchyard logs.

Key Source References

The following files in NVIDIA-NeMo/Switchyard define the canonical schema:

• switchyard/cli/defaults/openrouter.toml — Production llm_classifier deployment with full field documentation
• dev-server/config.toml — Comprehensive examples of all five route types in development context
• benchmark/server-configs/*.toml — Specialized configurations for performance evaluation

Summary

• Universal fields: id and type are mandatory for every route
• Type-specific requirements vary by routing algorithm:
  • random: requires targets array
  • passthrough: requires single target
  • noop: no additional fields
  • stage_router: requires five configuration parameters for two-stage routing
  • llm_classifier: requires six parameters for classifier-based intelligent routing
• Reference implementations in dev-server/config.toml and openrouter.toml provide validated templates for production use

Frequently Asked Questions

What happens if I omit a required field in a Switchyard route configuration?

The Switchyard server will fail to start and emit a configuration validation error. The error message typically identifies the missing field and the problematic route table. Always validate against the source examples in dev-server/config.toml before deployment.

Can I use the same id value for multiple routes?

No. The id field must be globally unique across all routes in a configuration. Duplicate IDs will cause routing ambiguity and server startup failures. Use descriptive, namespaced identifiers like "prod/us-east/classifier" to ensure uniqueness.

How do I choose between stage_router and llm_classifier route types?

Use stage_router when you have clear efficiency/capability tradeoffs and want deterministic fallback logic based on configurable thresholds. Use llm_classifier when you need learned, query-adaptive routing that can improve over time with better classifiers—the llm_classifier requires training data and a dedicated classifier model but achieves finer-grained routing decisions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}