How Switchyard Validates Version-1 TOML Deployment Configurations: Schema, LLM Clients, and Routing Rules

Switchyard validates version-1 TOML deployments by deserializing the file into a typed Config struct, verifying the schema_version matches SUPPORTED_SCHEMA_VERSION (value 1), then checking that all targets reference valid LLM clients and all routes reference valid targets while enforcing weight consistency.

NVIDIA NeMo Switchyard uses a strict version-1 TOML deployment schema to wire together LLM clients, model targets, and routing rules. When a configuration file is loaded, the validation pipeline ensures syntactic correctness and semantic consistency before the router processes any requests, preventing runtime errors through upfront schema enforcement.

The Config Struct and Schema Versioning

At the core of validation is the Config struct defined in [crates/switchyard-runner/src/config.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-runner/src/config.rs). This struct requires a mandatory schema_version: u32 field alongside three top-level tables:

schema_version = 1

[llm_clients.inference_hub]
type = "inference_hub"
url = "https://api.nvidia.com/v1"

[targets.capable]
clients = ["inference_hub"]

[routes.random]
targets = ["capable", "efficient"]
weights = [1, 2]

The Config::validate method first compares the supplied schema_version against the constant SUPPORTED_SCHEMA_VERSION (hardcoded to 1). If the values differ, the validator immediately returns an error such as "unsupported schema_version X; expected 1", blocking the loading of incompatible older or newer formats.

Deserialization and Unique Name Enforcement

Switchyard leverages the toml crate to parse the configuration file into the Config struct. Each section—llm_clients, targets, and routes—is stored as a HashMap<String, _>, which provides automatic duplicate key rejection during deserialization. This ensures every client, target, and route possesses a distinct identifier without requiring explicit uniqueness checks in the validation logic.

Cross-Reference Validation

After successful deserialization, Config::validate performs semantic checks to verify that references between components resolve correctly.

Target-to-Client References

Every target must specify a clients array containing names of LLM clients defined in the llm_clients table. The validator iterates over each target's client list and confirms that each referenced name exists as a key in the llm_clients map. If a target references "inference_hub" but no such client is defined, the validator returns an error like "unknown client 'inference_hub' in target 'capable'".

Route-to-Target References

Similarly, routes declare which targets they can forward requests to via a targets array. The validation routine checks that every target name appearing in a route's configuration exists in the global targets map. Missing references trigger descriptive errors that pinpoint the specific route containing the invalid reference.

Client-Specific and Route Constraints

Beyond structural validation, Switchyard enforces domain-specific rules on routing weights and client configurations.

Weight Consistency Checks

Routes utilizing the weights array for probabilistic routing must satisfy two constraints validated by Config::validate:

• The weights array length must exactly match the targets array length
• The sum of all weights must be a positive number (greater than 0)

These checks guarantee that the routing algorithm can compute a valid probability distribution across target endpoints.

Client Type Validation

Each entry under llm_clients deserializes into a concrete client-type struct (e.g., InferenceHubClient, MockClient). The validator invokes each client's native validate method, which verifies type-specific requirements such as valid endpoint URLs, authentication token presence, and model-specific parameters. This modular approach allows new client types to be added by simply implementing the validate trait method, without modifying the core Config validation logic.

Runtime Integration and Error Handling

Validation executes synchronously when Runner::new_from_file(path) or Config::load_from_path(path) is called. The typical invocation pattern follows this sequence:

let cfg = Config::load_from_path("deployment.toml")?;  // Parses TOML into Config
cfg.validate()?;                                       // Executes all checks
let runner = Runner::new(cfg);                       // Builds RuntimeModels graph

If any validation step fails—whether from schema version mismatches, unresolved references, or invalid weights—the validate() method returns an Err with a specific message, preventing the Runner from instantiating the RuntimeModels object. This fail-fast behavior ensures that configuration errors surface during startup rather than during request processing.

Summary

• Switchyard requires schema_version = 1 in every deployment file, rejecting configurations with mismatched versions immediately.
• The Config struct in crates/switchyard-runner/src/config.rs uses HashMap structures to enforce unique naming across llm_clients, targets, and routes.
• Cross-reference validation ensures targets only reference existing LLM clients and routes only reference existing targets.
• Weight arrays in routes must align with target counts and sum to positive values to support valid probability distributions.
• Client-specific validation occurs through modular validate methods implemented per client type.
• All checks run synchronously before the Runner builds the routing graph, ensuring fail-fast error detection at startup.

Frequently Asked Questions

What happens if I use the wrong schema_version in my Switchyard TOML file?

Switchyard will reject the configuration with an explicit error message indicating the unsupported version number and stating that version 1 is expected. The Runner will not initialize, and you must update the schema_version field to match the supported constant before the deployment can load.

How does Switchyard prevent duplicate client or target names?

The TOML deserializer stores llm_clients, targets, and routes as HashMap<String, _> structures, which inherently reject duplicate keys during parsing. This eliminates the possibility of naming collisions without requiring additional validation logic in the Rust code.

Can a route reference targets that don't exist in the configuration?

No. The Config::validate method explicitly checks every target name listed in a route's targets array against the global targets map. If any reference is undefined, validation fails with an error specifying the route name and the missing target, preventing the router from starting with invalid routing rules.

Are client-specific settings like API URLs validated during schema validation?

Yes. After structural validation, Switchyard calls the validate method on each client struct (such as InferenceHubClient) to verify that required fields like url, authentication tokens, and model parameters are present and properly formatted. This ensures that connection failures are caught during configuration loading rather than at runtime.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}