# How to Define a Target Model Endpoint in Switchyard’s TOML Configuration > Learn to define a target model endpoint in Switchyard using TOML configuration. Set up your upstream model identifier and client connection for seamless routing. - Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard) - Tags: how-to-guide - Published: 2026-08-22 --- **To define a target model endpoint in Switchyard, create a `[targets.]` table in your TOML file that specifies the `id` (upstream model identifier) and `llm_client` (connection configuration), then reference this target name in your route definitions.** Switchyard uses a TOML-based deployment configuration to route LLM requests to upstream providers. According to the NVIDIA-NeMo/Switchyard source code, a **target model endpoint** acts as the bridge between routing logic and physical model instances, encapsulating the exact model identifier and connection parameters required to reach providers like OpenAI, Anthropic, or OpenRouter. ## Understanding the Target Configuration Schema In [`docs/reference/toml_schema.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/reference/toml_schema.md), the `[targets.]` table defines concrete model endpoints that routes can forward requests to. Each target maps a logical name to a physical upstream model through three key concepts: **LLM clients** (connection configurations), **targets** (model endpoints), and **routes** (selection algorithms). ### Required Configuration Fields Every target entry must include two mandatory fields: - **`id`**: The exact model identifier string that the upstream provider expects (for example, `anthropic/claude-sonnet-4.5` or `openai/gpt-4o`). This value is passed directly to the provider's API. - **`llm_client`**: The name of a previously defined `[llm_clients.]` entry that specifies the base URL, authentication method, and wire format for the upstream connection. ### Optional Parameters Targets support an optional **`extra_body`** field containing a JSON object that Switchyard merges into every request sent to that model. Use this for provider-specific options such as `chat_template_kwargs`, temperature settings, or other vendor extensions. ## Minimal Configuration Example The following TOML demonstrates the complete three-layer architecture: client definition, target endpoint creation, and route assignment: ```toml schema_version = 1 # 1. Define the LLM client (upstream connection) [llm_clients.openrouter] format = "openai_chat" base_url = "https://openrouter.ai/api/v1" api_key_env = "OPENROUTER_API_KEY" # 2. Define the target model endpoint [targets.strong] id = "anthropic/claude-sonnet-4.5" llm_client = "openrouter" # 3. Reference the target in a route [routes.default] id = "switchyard" type = "passthrough" target = "strong" ``` In this configuration, the `[llm_clients.openrouter]` block establishes the connection parameters to OpenRouter using the OpenAI Chat wire format. The `[targets.strong]` block creates a model endpoint named `strong` that points to Claude Sonnet 4.5 via the OpenRouter client. Finally, the `[routes.default]` block creates a passthrough route that forwards all requests to the `strong` target. ## Advanced Target Configuration Patterns ### Provider-Specific Options with extra_body When upstream providers support custom request fields not covered by standard parameters, use the `extra_body` table to inject provider-specific metadata. This is particularly useful for vLLM deployments or proprietary API extensions. ```toml [targets.strong] id = "provider/model" llm_client = "provider" extra_body = { temperature = 0.7, chat_template_kwargs = { enable_thinking = false } } ``` As implemented in [`crates/switchyard-server/CONFIGURATION.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/CONFIGURATION.md), the `extra_body` content is deep-merged into the request payload before transmission to the upstream provider. ### Multiple Target Definitions for Tiered Routing Switchyard supports defining multiple targets to enable intelligent routing between model tiers (weak/strong) or cost optimization strategies: ```toml schema_version = 1 [llm_clients.openrouter] format = "openai_chat" base_url = "https://openrouter.ai/api/v1" api_key_env = "OPENROUTER_API_KEY" # Weak (cost-optimized) target [targets.weak] id = "openrouter/llama-2-7b" llm_client = "openrouter" # Strong (capability-optimized) target [targets.strong] id = "anthropic/claude-sonnet-4.5" llm_client = "openrouter" extra_body = { temperature = 0.7 } # Classifier route that selects between targets [routes.chat] id = "my-chat" type = "llm_classifier" mode = "capability" weak_target = "weak" strong_target = "strong" classifier_target = "strong" ``` This configuration defines two distinct model endpoints—`weak` and `strong`—that the classifier route selects between based on request complexity. ## Internal Architecture of Target Resolution When Switchyard loads the TOML file via `switchyard-server --config routes.toml`, the runtime performs a three-phase initialization: 1. **Client Parsing**: The server parses `[llm_clients]` tables to construct client objects containing base URLs, headers, and authentication modes. 2. **Target Resolution**: Each `[targets]` entry is validated against its referenced `llm_client`. The system builds a `TargetConfig` structure containing the model `id` and a reference to the resolved client. 3. **Route Binding**: Routes reference targets by name. At request time, the routing algorithm selects a target, and the server forwards the request through the associated client while injecting the target's `extra_body` parameters into the payload. Thus, defining a target creates the concrete mapping between Switchyard's logical routing layer and the physical upstream model endpoint. ## Summary - **Target model endpoints** are defined in `[targets.]` tables within the Switchyard TOML configuration. - Every target requires an **`id`** (upstream model identifier) and **`llm_client`** (connection configuration name). - Use **`extra_body`** to inject provider-specific parameters into every request sent to that target. - Targets bridge the gap between **routes** (logical request handlers) and **LLM clients** (physical connection configurations). - Refer to [`docs/reference/toml_schema.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/reference/toml_schema.md) for the complete schema definition and validation rules. ## Frequently Asked Questions ### What is the difference between an LLM client and a target in Switchyard? An **LLM client** defines how Switchyard communicates with an upstream provider—the base URL, authentication method, and wire format (OpenAI Chat, Anthropic, etc.). A **target** is a specific model endpoint that uses that client to reach a concrete model instance. One client can support multiple targets (for example, various models hosted on the same provider), but each target must reference exactly one client. ### Can I use the same target in multiple routes? Yes. Once defined in the `[targets]` section, a target name can be referenced by any number of routes via the `target`, `strong_target`, or `weak_target` fields. This allows you to define a model endpoint once and reuse it across different routing strategies (passthrough, classifier-based, or load-balanced) without duplicating configuration. ### How do I pass dynamic authentication through to the upstream model? To forward the original caller's credentials rather than using a static API key, enable authentication forwarding on the **client** definition using `forward_auth = true`, then reference any target that uses that client. The target definition remains unchanged; the credential forwarding behavior is controlled at the client level. ### What happens if I specify an invalid llm_client name in a target? Switchyard validates the TOML configuration at startup. If a target references an `llm_client` that is not defined in the `[llm_clients]` section, the server will fail to initialize and report a configuration error. This validation ensures that all targets have a valid transport mechanism before accepting requests.