# Switchyard Standalone Server vs. Embedded Runtime: A Complete Deployment Guide

> Understand Switchyard standalone server vs embedded runtime. Learn how to deploy Switchyard as an independent HTTP proxy or integrate its routing core directly into your Rust applications.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**The `switchyard-server` binary runs as an independent HTTP proxy process, while `switchyard-libsy` embeds the same routing core as a Rust library for direct programmatic use.**

Switchyard, NVIDIA's Rust-based LLM routing layer, provides two interchangeable runtime surfaces that share identical protocol definitions and routing algorithms. Your choice between them depends on whether you need a self-contained network service or tight integration within an existing application stack.

## Standalone Rust Proxy Server

The **`switchyard-server`** binary delivers Switchyard as a complete HTTP service. It loads a native TOML deployment configuration, translates provider-neutral requests into shared protocol types, applies configured routing algorithms, and forwards calls to upstream LLM clients.

### How the Standalone Server Works

In [`crates/switchyard-server/src/main.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/main.rs), the server initializes from command-line arguments:

```bash

# Build and install the Rust binary

cargo install --locked switchyard-server

# Launch with TOML configuration

switchyard-server --config examples/deployment.toml --port 4000

```

The server exposes OpenAI/Anthropic-compatible HTTP endpoints. External clients interact with it as they would any standard LLM API:

```python
import openai

openai.api_key = "irrelevant"
openai.base_url = "http://localhost:4000/v1"

resp = openai.ChatCompletion.create(
    model="my-route",  # route ID defined in the TOML

    messages=[{"role": "user", "content": "Hello"}],
)
print(resp)

```

### When to Choose the Standalone Server

- **Service-oriented architectures** where Switchyard runs as a separate deployment unit
- **Multi-language environments** requiring OpenAI-compatible HTTP access
- **Operations teams** preferring configuration-driven deployment via TOML files
- **Network isolation** scenarios where proxy and client must reside on different hosts

## Embedded Runtime Library

The **`switchyard-libsy`** crate exposes the identical routing core through a Rust API. Applications programmatically construct targets, algorithms, and routes, then invoke routing logic directly without HTTP overhead.

### How the Embedded Runtime Works

In [`crates/libsy/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/lib.rs), the library provides a direct programmatic interface:

```python
from switchyard_rust import libsy

# Initialize router from TOML (same format as standalone)

router = libsy.Router.from_toml_path("examples/deployment.toml")

# Route without HTTP layer

request = {
    "model": "my-route",
    "messages": [{"role": "user", "content": "Hello"}],
}
response = router.handle_chat_completion(request)
print(response)

```

The Python binding in [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py) wraps the native Rust library through PyO3, enabling embedded use from Python applications.

### When to Choose the Embedded Runtime

- **Rust or Python applications** requiring sub-millisecond routing overhead
- **Custom request/response handling** where you manage upstream calls yourself
- **Tight lifecycle control** over the routing component
- **Library-style integration** within larger ML serving frameworks

## Key Architectural Differences

| Dimension | Standalone Server (`switchyard-server`) | Embedded Runtime (`switchyard-libsy`) |
|-----------|----------------------------------------|---------------------------------------|
| **Deployment unit** | Binary process | Rust crate / shared library |
| **Interface** | HTTP endpoints | Function calls |
| **Configuration** | TOML file loaded at startup | TOML file or programmatic construction |
| **Request handling** | Fully managed by server | Delegated to host or handled internally |
| **Network overhead** | TCP/HTTP latency | In-process memory calls |
| **Multi-language access** | Any HTTP client | Rust native, Python via PyO3 |

Both surfaces import identical protocol definitions from `switchyard-protocol` and apply the same routing algorithms—only the deployment model differs.

## Code-Level Implementation Details

### Server Binary Structure

Per [`crates/switchyard-server/README.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/README.md), the server binary:

1. Parses CLI arguments via `clap` (see [`docs/cli_reference.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/cli_reference.md) lines 7-53)
2. Loads and validates TOML deployment configuration
3. Initializes the Tokio runtime for async request handling
4. Binds to the configured TCP port
5. Translates incoming HTTP requests to internal protocol types before routing

### Library Integration Points

Per [`crates/libsy/README.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/README.md), the library provides:

- `Router::from_toml_path()` — load configuration from file
- `Router::handle_chat_completion()` — synchronous routing entry point
- Direct access to target selection algorithms for custom implementations
- Optional upstream delegation via callback interfaces

## Python Binding Implementation

The [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py) module bridges both worlds. For embedded use, it exposes the native library directly. For server use, [`switchyard/cli/launchers/native_server.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/launchers/native_server.py) hosts the Rust binary through PyO3 bindings—this powers the `switchyard launch` command.

```python

# From switchyard/cli/launchers/native_server.py

# Launches the standalone server via PyO3 binding

```

## Summary

- **`switchyard-server`** deploys Switchyard as an independent HTTP proxy—ideal for service-oriented architectures and OpenAI-compatible integrations.
- **`switchyard-libsy`** embeds the routing core as a library—optimal for latency-sensitive applications requiring programmatic control.
- Both share `switchyard-protocol` for type definitions and routing algorithms, ensuring behavioral consistency across deployment modes.
- Choose based on whether network isolation or in-process integration better serves your architecture.

## Frequently Asked Questions

### Can I switch between standalone and embedded without changing my routing configuration?

Yes. Both runtime surfaces accept identical TOML deployment files. The [`examples/deployment.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/deployment.toml) format works with `switchyard-server --config` and `libsy.Router.from_toml_path()` without modification.

### Does the embedded runtime support all routing algorithms available in the standalone server?

Yes. Both implementations import algorithms from the shared `switchyard-protocol` crate. Per [`docs/core_concepts.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/core_concepts.md) lines 10-18, the runtime surfaces differ only in invocation mechanism, not routing capability.

### What performance difference exists between the two deployment models?

The embedded runtime eliminates HTTP serialization and TCP overhead. For high-throughput scenarios, `switchyard-libsy` avoids network stack latency entirely—relevant when routing decisions must complete in microseconds rather than milliseconds.

### Can I embed Switchyard in languages other than Rust and Python?

Currently, only Rust (native) and Python (via PyO3 bindings in [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py)) are officially supported. The standalone server remains the universal integration point for other languages through its HTTP interface.