How to Define a Target Model Endpoint in Switchyard’s TOML Configuration

To define a target model endpoint in Switchyard, create a [targets.<name>] table in your TOML file that specifies the id (upstream model identifier) and llm_client (connection configuration), then reference this target name in your route definitions.

Switchyard uses a TOML-based deployment configuration to route LLM requests to upstream providers. According to the NVIDIA-NeMo/Switchyard source code, a target model endpoint acts as the bridge between routing logic and physical model instances, encapsulating the exact model identifier and connection parameters required to reach providers like OpenAI, Anthropic, or OpenRouter.

Understanding the Target Configuration Schema

In docs/reference/toml_schema.md, the [targets.<name>] table defines concrete model endpoints that routes can forward requests to. Each target maps a logical name to a physical upstream model through three key concepts: LLM clients (connection configurations), targets (model endpoints), and routes (selection algorithms).

Required Configuration Fields

Every target entry must include two mandatory fields:

• id: The exact model identifier string that the upstream provider expects (for example, anthropic/claude-sonnet-4.5 or openai/gpt-4o). This value is passed directly to the provider's API.
• llm_client: The name of a previously defined [llm_clients.<name>] entry that specifies the base URL, authentication method, and wire format for the upstream connection.

Optional Parameters

Targets support an optional extra_body field containing a JSON object that Switchyard merges into every request sent to that model. Use this for provider-specific options such as chat_template_kwargs, temperature settings, or other vendor extensions.

Minimal Configuration Example

The following TOML demonstrates the complete three-layer architecture: client definition, target endpoint creation, and route assignment:

schema_version = 1

# 1. Define the LLM client (upstream connection)

[llm_clients.openrouter]
format      = "openai_chat"
base_url    = "https://openrouter.ai/api/v1"
api_key_env = "OPENROUTER_API_KEY"

# 2. Define the target model endpoint

[targets.strong]
id         = "anthropic/claude-sonnet-4.5"
llm_client = "openrouter"

# 3. Reference the target in a route

[routes.default]
id     = "switchyard"
type   = "passthrough"
target = "strong"

In this configuration, the [llm_clients.openrouter] block establishes the connection parameters to OpenRouter using the OpenAI Chat wire format. The [targets.strong] block creates a model endpoint named strong that points to Claude Sonnet 4.5 via the OpenRouter client. Finally, the [routes.default] block creates a passthrough route that forwards all requests to the strong target.

Advanced Target Configuration Patterns

Provider-Specific Options with extra_body

When upstream providers support custom request fields not covered by standard parameters, use the extra_body table to inject provider-specific metadata. This is particularly useful for vLLM deployments or proprietary API extensions.

[targets.strong]
id         = "provider/model"
llm_client = "provider"
extra_body = { temperature = 0.7, chat_template_kwargs = { enable_thinking = false } }

As implemented in crates/switchyard-server/CONFIGURATION.md, the extra_body content is deep-merged into the request payload before transmission to the upstream provider.

Multiple Target Definitions for Tiered Routing

Switchyard supports defining multiple targets to enable intelligent routing between model tiers (weak/strong) or cost optimization strategies:

schema_version = 1

[llm_clients.openrouter]
format      = "openai_chat"
base_url    = "https://openrouter.ai/api/v1"
api_key_env = "OPENROUTER_API_KEY"

# Weak (cost-optimized) target

[targets.weak]
id         = "openrouter/llama-2-7b"
llm_client = "openrouter"

# Strong (capability-optimized) target

[targets.strong]
id         = "anthropic/claude-sonnet-4.5"
llm_client = "openrouter"
extra_body = { temperature = 0.7 }

# Classifier route that selects between targets

[routes.chat]
id                = "my-chat"
type              = "llm_classifier"
mode              = "capability"
weak_target       = "weak"
strong_target     = "strong"
classifier_target = "strong"

This configuration defines two distinct model endpoints—weak and strong—that the classifier route selects between based on request complexity.

Internal Architecture of Target Resolution

When Switchyard loads the TOML file via switchyard-server --config routes.toml, the runtime performs a three-phase initialization:

1. Client Parsing: The server parses [llm_clients] tables to construct client objects containing base URLs, headers, and authentication modes.
2. Target Resolution: Each [targets] entry is validated against its referenced llm_client. The system builds a TargetConfig structure containing the model id and a reference to the resolved client.
3. Route Binding: Routes reference targets by name. At request time, the routing algorithm selects a target, and the server forwards the request through the associated client while injecting the target's extra_body parameters into the payload.

Thus, defining a target creates the concrete mapping between Switchyard's logical routing layer and the physical upstream model endpoint.

Summary

• Target model endpoints are defined in [targets.<name>] tables within the Switchyard TOML configuration.
• Every target requires an id (upstream model identifier) and llm_client (connection configuration name).
• Use extra_body to inject provider-specific parameters into every request sent to that target.
• Targets bridge the gap between routes (logical request handlers) and LLM clients (physical connection configurations).
• Refer to docs/reference/toml_schema.md for the complete schema definition and validation rules.

Frequently Asked Questions

What is the difference between an LLM client and a target in Switchyard?

An LLM client defines how Switchyard communicates with an upstream provider—the base URL, authentication method, and wire format (OpenAI Chat, Anthropic, etc.). A target is a specific model endpoint that uses that client to reach a concrete model instance. One client can support multiple targets (for example, various models hosted on the same provider), but each target must reference exactly one client.

Can I use the same target in multiple routes?

Yes. Once defined in the [targets] section, a target name can be referenced by any number of routes via the target, strong_target, or weak_target fields. This allows you to define a model endpoint once and reuse it across different routing strategies (passthrough, classifier-based, or load-balanced) without duplicating configuration.

How do I pass dynamic authentication through to the upstream model?

To forward the original caller's credentials rather than using a static API key, enable authentication forwarding on the client definition using forward_auth = true, then reference any target that uses that client. The target definition remains unchanged; the credential forwarding behavior is controlled at the client level.

What happens if I specify an invalid llm_client name in a target?

Switchyard validates the TOML configuration at startup. If a target references an llm_client that is not defined in the [llm_clients] section, the server will fail to initialize and report a configuration error. This validation ensures that all targets have a valid transport mechanism before accepting requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}