Switchyard Standalone Server vs. Embedded Runtime: A Complete Deployment Guide

The switchyard-server binary runs as an independent HTTP proxy process, while switchyard-libsy embeds the same routing core as a Rust library for direct programmatic use.

Switchyard, NVIDIA's Rust-based LLM routing layer, provides two interchangeable runtime surfaces that share identical protocol definitions and routing algorithms. Your choice between them depends on whether you need a self-contained network service or tight integration within an existing application stack.

Standalone Rust Proxy Server

The switchyard-server binary delivers Switchyard as a complete HTTP service. It loads a native TOML deployment configuration, translates provider-neutral requests into shared protocol types, applies configured routing algorithms, and forwards calls to upstream LLM clients.

How the Standalone Server Works

In crates/switchyard-server/src/main.rs, the server initializes from command-line arguments:

# Build and install the Rust binary

cargo install --locked switchyard-server

# Launch with TOML configuration

switchyard-server --config examples/deployment.toml --port 4000

The server exposes OpenAI/Anthropic-compatible HTTP endpoints. External clients interact with it as they would any standard LLM API:

import openai

openai.api_key = "irrelevant"
openai.base_url = "http://localhost:4000/v1"

resp = openai.ChatCompletion.create(
    model="my-route",  # route ID defined in the TOML

    messages=[{"role": "user", "content": "Hello"}],
)
print(resp)

When to Choose the Standalone Server

• Service-oriented architectures where Switchyard runs as a separate deployment unit
• Multi-language environments requiring OpenAI-compatible HTTP access
• Operations teams preferring configuration-driven deployment via TOML files
• Network isolation scenarios where proxy and client must reside on different hosts

Embedded Runtime Library

The switchyard-libsy crate exposes the identical routing core through a Rust API. Applications programmatically construct targets, algorithms, and routes, then invoke routing logic directly without HTTP overhead.

How the Embedded Runtime Works

In crates/libsy/src/lib.rs, the library provides a direct programmatic interface:

from switchyard_rust import libsy

# Initialize router from TOML (same format as standalone)

router = libsy.Router.from_toml_path("examples/deployment.toml")

# Route without HTTP layer

request = {
    "model": "my-route",
    "messages": [{"role": "user", "content": "Hello"}],
}
response = router.handle_chat_completion(request)
print(response)

The Python binding in switchyard_rust/libsy.py wraps the native Rust library through PyO3, enabling embedded use from Python applications.

When to Choose the Embedded Runtime

• Rust or Python applications requiring sub-millisecond routing overhead
• Custom request/response handling where you manage upstream calls yourself
• Tight lifecycle control over the routing component
• Library-style integration within larger ML serving frameworks

Key Architectural Differences

Dimension	Standalone Server (switchyard-server)	Embedded Runtime (switchyard-libsy)
Deployment unit	Binary process	Rust crate / shared library
Interface	HTTP endpoints	Function calls
Configuration	TOML file loaded at startup	TOML file or programmatic construction
Request handling	Fully managed by server	Delegated to host or handled internally
Network overhead	TCP/HTTP latency	In-process memory calls
Multi-language access	Any HTTP client	Rust native, Python via PyO3

Both surfaces import identical protocol definitions from switchyard-protocol and apply the same routing algorithms—only the deployment model differs.

Code-Level Implementation Details

Server Binary Structure

Per crates/switchyard-server/README.md, the server binary:

1. Parses CLI arguments via clap (see docs/cli_reference.md lines 7-53)
2. Loads and validates TOML deployment configuration
3. Initializes the Tokio runtime for async request handling
4. Binds to the configured TCP port
5. Translates incoming HTTP requests to internal protocol types before routing

Library Integration Points

Per crates/libsy/README.md, the library provides:

• Router::from_toml_path() — load configuration from file
• Router::handle_chat_completion() — synchronous routing entry point
• Direct access to target selection algorithms for custom implementations
• Optional upstream delegation via callback interfaces

Python Binding Implementation

The switchyard_rust/libsy.py module bridges both worlds. For embedded use, it exposes the native library directly. For server use, switchyard/cli/launchers/native_server.py hosts the Rust binary through PyO3 bindings—this powers the switchyard launch command.

# From switchyard/cli/launchers/native_server.py

# Launches the standalone server via PyO3 binding

Summary

• switchyard-server deploys Switchyard as an independent HTTP proxy—ideal for service-oriented architectures and OpenAI-compatible integrations.
• switchyard-libsy embeds the routing core as a library—optimal for latency-sensitive applications requiring programmatic control.
• Both share switchyard-protocol for type definitions and routing algorithms, ensuring behavioral consistency across deployment modes.
• Choose based on whether network isolation or in-process integration better serves your architecture.

Frequently Asked Questions

Can I switch between standalone and embedded without changing my routing configuration?

Yes. Both runtime surfaces accept identical TOML deployment files. The examples/deployment.toml format works with switchyard-server --config and libsy.Router.from_toml_path() without modification.

Does the embedded runtime support all routing algorithms available in the standalone server?

Yes. Both implementations import algorithms from the shared switchyard-protocol crate. Per docs/core_concepts.md lines 10-18, the runtime surfaces differ only in invocation mechanism, not routing capability.

What performance difference exists between the two deployment models?

The embedded runtime eliminates HTTP serialization and TCP overhead. For high-throughput scenarios, switchyard-libsy avoids network stack latency entirely—relevant when routing decisions must complete in microseconds rather than milliseconds.

Can I embed Switchyard in languages other than Rust and Python?

Currently, only Rust (native) and Python (via PyO3 bindings in switchyard_rust/libsy.py) are officially supported. The standalone server remains the universal integration point for other languages through its HTTP interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}