# How to Run the Native Rust Server Standalone Without the Python Launcher in Switchyard

> Learn to run Switchyard's native Rust server standalone without Python launchers. Build the switchyard server binary and use a TOML config for direct invocation.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**You can run Switchyard's native Rust server directly by building the `switchyard-server` binary from `crates/switchyard-server` and invoking it with a TOML configuration file, completely bypassing Python launchers.**

Switchyard's core traffic-orchestration engine is written in Rust and packaged as the **`switchyard-server`** crate. This binary handles LLM routing, algorithm selection, and load balancing through an Axum-based HTTP server. Running it standalone gives you direct control over configuration, networking, and observability without Python wrapper overhead.

## Prerequisites

You need the Rust toolchain installed. If you don't have it:

```bash
curl https://sh.rustup.rs -sSf | sh

```

Follow the prompts, then reload your shell or run `source $HOME/.cargo/env`.

## Building the Native Rust Server

### Step 1: Clone the Repository

```bash
git clone https://github.com/NVIDIA-NeMo/Switchyard.git
cd Switchyard

```

### Step 2: Build the Release Binary

```bash
cargo build -p switchyard-server --release

```

The compiled binary appears at **`target/release/switchyard-server`**.

Build times vary with your hardware. For development iterations, omit `--release` for faster compiles at slightly reduced runtime performance.

## Creating a Server Configuration

The Rust server requires a **TOML configuration file** that declares:

- **LLM clients** (backend model endpoints)
- **Targets** (logical groupings of clients)
- **Algorithm routes** (routing rules and weights)

A minimal example exists at [`dev-server/config.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/dev-server/config.toml). The configuration schema is defined in [`crates/switchyard-server/src/config.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/config.rs) and parsed through **`load_server_state`**.

```toml

# Example structure from dev-server/config.toml

[global]
log_level = "info"

[[client]]
name = "gpt-4"
provider = "openai"
api_key = "${OPENAI_API_KEY}"
model = "gpt-4"

[[target]]
name = "default"
clients = ["gpt-4"]

[[route]]
target = "default"
algorithm = "round_robin"

```

Environment variable substitution (e.g., `${OPENAI_API_KEY}`) is supported.

## Running the Standalone Rust Server

### Basic Invocation

```bash
./target/release/switchyard-server \
    --config dev-server/config.toml \
    --host 0.0.0.0 \
    --port 4000

```

### Available Command-Line Flags

The binary uses **`clap`** for argument parsing with the **`ServerArgs`** struct in [`crates/switchyard-server/src/cli.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/cli.rs):

| Flag | Description | Default |
|------|-------------|---------|
| `--config <PATH>` | Path to TOML configuration file | *(required)* |
| `--host <ADDR>` | Bind address | `0.0.0.0` |
| `--port <PORT>` | Listen port | `4000` |
| `--backlog <N>` | TCP listen backlog | `1024` |
| `--shutdown-timeout <SEC>` | Graceful shutdown timeout | `30` |
| `--dry-run` | Validate config without binding socket | disabled |
| `--routing-log-file <PATH>` | Path for request routing logs | none |
| `--tls-cert <PATH>` | TLS certificate file | none |
| `--tls-key <PATH>` | TLS private key file | none |

### Configuration Validation

Test your configuration without starting the HTTP server:

```bash
./target/release/switchyard-server \
    --config path/to/routes.toml \
    --dry-run

```

This parses the TOML, validates client connectivity rules, and exits with code 0 on success.

### TLS-Enabled Deployment

```bash
./target/release/switchyard-server \
    --config production.toml \
    --host 0.0.0.0 \
    --port 443 \
    --tls-cert /etc/ssl/certs/switchyard.crt \
    --tls-key /etc/ssl/private/switchyard.key

```

## Programmatic Server Execution

You can also embed the server in Rust tests or custom binaries using the public API from [`crates/switchyard-server/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/lib.rs):

```rust
use switchyard_server::cli::ServerArgs;
use switchyard_server::run_server;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let args = ServerArgs {
        config: "dev-server/config.toml".into(),
        host: "127.0.0.1".parse()?,
        port: 4000,
        backlog: switchyard_server::DEFAULT_LISTEN_BACKLOG,
        shutdown_timeout: Default::default(),
        dry_run: false,
        routing_log_file: None,
        tls_cert: None,
        tls_key: None,
    };
    
    let (state, options) = args.into_runtime()?;
    run_server(state, options).await?;
    
    Ok(())
}

```

The **`into_runtime()`** method (defined in [`cli.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/cli.rs)) converts CLI arguments into the internal **`ServerState`** and **`ServerOptions`** structures required by **`run_server`**.

## Key Source Files for Standalone Operation

| File | Purpose |
|------|---------|
| [`crates/switchyard-server/src/main.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/main.rs) | Binary entry point—initializes tracing and delegates to CLI runner |
| [`crates/switchyard-server/src/cli.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/cli.rs) | **`ServerArgs`** struct and **`clap`** CLI definition |
| [`crates/switchyard-server/src/config.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/config.rs) | TOML parsing via **`load_server_state`** |
| [`crates/switchyard-server/src/server.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/server.rs) | Axum HTTP server implementation with route handlers |
| [`crates/switchyard-server/Cargo.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/Cargo.toml) | Declares the **`[[bin]]`** target and crate dependencies |

## Summary

- **Build** the standalone binary with `cargo build -p switchyard-server --release`
- **Configure** using a TOML file specifying clients, targets, and routes
- **Execute** directly via `./target/release/switchyard-server --config <path>`
- **Validate** configurations with `--dry-run` before production deployment
- **Embed** programmatically using `ServerArgs` and `run_server` for custom integrations

Running the native Rust server standalone eliminates Python interpreter overhead and gives you precise control over Switchyard's routing and algorithm layer.

## Frequently Asked Questions

### What is the difference between the Python launcher and the native Rust server?

The Python launcher provides convenience wrappers and ecosystem integration, but adds interpreter startup latency. The native `switchyard-server` binary starts faster, uses less memory, and exposes all configuration options directly through command-line flags without abstraction layers.

### Can I use environment variables in the TOML configuration?

Yes. Switchyard's configuration parser in [`crates/switchyard-server/src/config.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/config.rs) supports `${VAR_NAME}` substitution for environment variables, enabling secure credential management without hardcoding secrets.

### What happens if the configuration file is invalid?

When running with `--dry-run`, the server validates the TOML schema, checks client definitions, and reports errors with file locations before exiting. Without `--dry-run`, validation still occurs but errors surface during server startup after socket binding may have already begun.

### Is TLS termination required or can I use a reverse proxy?

TLS is optional. You can run the Rust server on plain HTTP behind an external TLS-terminating proxy (nginx, Envoy, cloud load balancer) or enable native TLS with `--tls-cert` and `--tls-key` for end-to-end encryption.