How to Run the Native Rust Server Standalone Without the Python Launcher in Switchyard

You can run Switchyard's native Rust server directly by building the switchyard-server binary from crates/switchyard-server and invoking it with a TOML configuration file, completely bypassing Python launchers.

Switchyard's core traffic-orchestration engine is written in Rust and packaged as the switchyard-server crate. This binary handles LLM routing, algorithm selection, and load balancing through an Axum-based HTTP server. Running it standalone gives you direct control over configuration, networking, and observability without Python wrapper overhead.

Prerequisites

You need the Rust toolchain installed. If you don't have it:

curl https://sh.rustup.rs -sSf | sh

Follow the prompts, then reload your shell or run source $HOME/.cargo/env.

Building the Native Rust Server

Step 1: Clone the Repository

git clone https://github.com/NVIDIA-NeMo/Switchyard.git
cd Switchyard

Step 2: Build the Release Binary

cargo build -p switchyard-server --release

The compiled binary appears at target/release/switchyard-server.

Build times vary with your hardware. For development iterations, omit --release for faster compiles at slightly reduced runtime performance.

Creating a Server Configuration

The Rust server requires a TOML configuration file that declares:

• LLM clients (backend model endpoints)
• Targets (logical groupings of clients)
• Algorithm routes (routing rules and weights)

A minimal example exists at dev-server/config.toml. The configuration schema is defined in crates/switchyard-server/src/config.rs and parsed through load_server_state.

# Example structure from dev-server/config.toml

[global]
log_level = "info"

[[client]]
name = "gpt-4"
provider = "openai"
api_key = "${OPENAI_API_KEY}"
model = "gpt-4"

[[target]]
name = "default"
clients = ["gpt-4"]

[[route]]
target = "default"
algorithm = "round_robin"

Environment variable substitution (e.g., ${OPENAI_API_KEY}) is supported.

Running the Standalone Rust Server

Basic Invocation

./target/release/switchyard-server \
    --config dev-server/config.toml \
    --host 0.0.0.0 \
    --port 4000

Available Command-Line Flags

The binary uses clap for argument parsing with the ServerArgs struct in crates/switchyard-server/src/cli.rs:

Flag	Description	Default
--config <PATH>	Path to TOML configuration file	(required)
--host <ADDR>	Bind address	0.0.0.0
--port <PORT>	Listen port	4000
--backlog <N>	TCP listen backlog	1024
--shutdown-timeout <SEC>	Graceful shutdown timeout	30
--dry-run	Validate config without binding socket	disabled
--routing-log-file <PATH>	Path for request routing logs	none
--tls-cert <PATH>	TLS certificate file	none
--tls-key <PATH>	TLS private key file	none

Configuration Validation

Test your configuration without starting the HTTP server:

./target/release/switchyard-server \
    --config path/to/routes.toml \
    --dry-run

This parses the TOML, validates client connectivity rules, and exits with code 0 on success.

TLS-Enabled Deployment

./target/release/switchyard-server \
    --config production.toml \
    --host 0.0.0.0 \
    --port 443 \
    --tls-cert /etc/ssl/certs/switchyard.crt \
    --tls-key /etc/ssl/private/switchyard.key

Programmatic Server Execution

You can also embed the server in Rust tests or custom binaries using the public API from crates/switchyard-server/src/lib.rs:

use switchyard_server::cli::ServerArgs;
use switchyard_server::run_server;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let args = ServerArgs {
        config: "dev-server/config.toml".into(),
        host: "127.0.0.1".parse()?,
        port: 4000,
        backlog: switchyard_server::DEFAULT_LISTEN_BACKLOG,
        shutdown_timeout: Default::default(),
        dry_run: false,
        routing_log_file: None,
        tls_cert: None,
        tls_key: None,
    };
    
    let (state, options) = args.into_runtime()?;
    run_server(state, options).await?;
    
    Ok(())
}

The into_runtime() method (defined in cli.rs) converts CLI arguments into the internal ServerState and ServerOptions structures required by run_server.

Key Source Files for Standalone Operation

File	Purpose
crates/switchyard-server/src/main.rs	Binary entry point—initializes tracing and delegates to CLI runner
crates/switchyard-server/src/cli.rs	ServerArgs struct and clap CLI definition
crates/switchyard-server/src/config.rs	TOML parsing via load_server_state
crates/switchyard-server/src/server.rs	Axum HTTP server implementation with route handlers
crates/switchyard-server/Cargo.toml	Declares the [[bin]] target and crate dependencies

Summary

• Build the standalone binary with cargo build -p switchyard-server --release
• Configure using a TOML file specifying clients, targets, and routes
• Execute directly via ./target/release/switchyard-server --config <path>
• Validate configurations with --dry-run before production deployment
• Embed programmatically using ServerArgs and run_server for custom integrations

Running the native Rust server standalone eliminates Python interpreter overhead and gives you precise control over Switchyard's routing and algorithm layer.

Frequently Asked Questions

What is the difference between the Python launcher and the native Rust server?

The Python launcher provides convenience wrappers and ecosystem integration, but adds interpreter startup latency. The native switchyard-server binary starts faster, uses less memory, and exposes all configuration options directly through command-line flags without abstraction layers.

Can I use environment variables in the TOML configuration?

Yes. Switchyard's configuration parser in crates/switchyard-server/src/config.rs supports ${VAR_NAME} substitution for environment variables, enabling secure credential management without hardcoding secrets.

What happens if the configuration file is invalid?

When running with --dry-run, the server validates the TOML schema, checks client definitions, and reports errors with file locations before exiting. Without --dry-run, validation still occurs but errors surface during server startup after socket binding may have already begun.

Is TLS termination required or can I use a reverse proxy?

TLS is optional. You can run the Rust server on plain HTTP behind an external TLS-terminating proxy (nginx, Envoy, cloud load balancer) or enable native TLS with --tls-cert and --tls-key for end-to-end encryption.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}