# How to Run the Examples in the Switchyard Repository

> Run Switchyard examples easily. Execute Python LibSy driver for tests or deploy LiteLLM routing plugin with Docker Compose for full proxy integration. Get started now.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: getting-started
- Published: 2026-09-11

---

**You can run Switchyard examples by either executing the pure-Python LibSy driver directly for minimal routing tests or deploying the LiteLLM routing plugin via Docker-Compose for a full proxy integration with OpenRouter.**

The NVIDIA-NeMo/Switchyard repository provides two self-contained example suites that demonstrate its core routing APIs. Whether you want to test the decision-only library or see how the framework integrates with LiteLLM as a routing plugin, you will need Python 3.12 or newer and optionally a Rust toolchain for the server components. Both approaches use actual source files from the codebase to show how to run the examples in Switchyard.

## Running the LibSy Python Example

The LibSy example located at [`examples/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py) is a minimal, pure-Python driver that streams a random routing algorithm and prints the selected model and response. It demonstrates how to import the Python bindings, create an algorithm, and handle the `Step.CallModel` and `Step.Done` events.

### Installation Prerequisites

Because the published `nemo-switchyard` package does not yet contain the decision-only API used by this example, you must install Switchyard directly from the repository source. From your terminal, run:

```bash
pip install git+https://github.com/NVIDIA-NeMo/Switchyard.git

```

This installation requires Python 3.12 or newer. The optional Rust toolchain is only necessary if you plan to build the server binary from source.

### Executing the Driver Script

The [`examples/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py) file contains a `Step` pattern-matching loop that drives the algorithm and uses an `EchoClient` to mock responses. Execute the driver with:

```bash
python examples/libsy.py

```

You should see output similar to:

```

Decision: efficient
Response: {'model': 'efficient', 'outputs': [{'role': 'assistant', 'content': [{'type': 'text', 'text': 'Hello'}]}]}

```

The script uses `algorithms.random` to pick between the two dummy targets **fast** and **quality** with weights of 1 and 3, respectively. The underlying flow initializes a `stage_router` or `random` algorithm and consumes the `run_stream` iterator, matching on each `Step` variant to process routing decisions.

## Running the LiteLLM Routing Example

The LiteLLM example demonstrates the full plug-in stack: LiteLLM → Switchyard routing plugin → algorithm → model selection → proxy → OpenRouter provider. This deployment requires Docker-Compose because it launches a local OpenRouter-backed proxy alongside a LiteLLM router container.

### Configure Environment Variables

Navigate to `examples/litellm/deployment/` and create a `.env` file from the provided template. The only external credential required is an **OpenRouter API key**:

```bash
cp deployment/.env.example deployment/.env

```

Edit `deployment/.env` to add your key:

```text
OPENROUTER_API_KEY=sk-...

```

### Start the Proxy with Docker-Compose

The default profile uses **stage** (efficient-first) routing. From the `examples/litellm/` directory, start the services:

```bash
docker compose -f deployment/compose.yaml up -d --build --wait

```

The [`compose.yaml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/compose.yaml) file mounts the `stage` profile from `deployment/profiles/stage/` and sets the `SWITCHYARD_LITELLM_CONFIG` environment variable so that the Switchyard routing plugin loads the corresponding [`switchyard.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard.toml) configuration.

### Send Test Requests via curl

Any OpenAI-compatible client can hit the proxy at `http://127.0.0.1:4000`. To test the routing decision with a simple HTTP request:

```bash
curl -i http://127.0.0.1:4000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "switchyard",
    "messages": [{"role": "user", "content": "Reply with the word hello."}],
    "max_tokens": 64
  }'

```

Check the response header `x-litellm-model-name` to see which concrete provider model Switchyard selected (for example, `openrouter/openai/gpt-5.6-sol`).

### Run the Python Driver Alternative

For a programmatic approach, use the script in [`examples/litellm/examples/python_router.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/litellm/examples/python_router.py). This file constructs a LiteLLM `Router`, registers the `StageRoutingPlugin`, and prints the selected model and answer. Run it using the repository's locked Python environment:

```bash
uv sync --locked --python 3.12
uv run --locked --env-file deployment/.env python examples/litellm/examples/python_router.py

```

### Stopping the Service

When you have finished testing, tear down the containers:

```bash
docker compose -f deployment/compose.yaml down

```

## Switching Between Routing Profiles

The LiteLLM integration ships with two profiles: **stage** and **random**. To run the **random** profile instead of the default stage profile, prefix the Docker-Compose command with the `SWITCHYARD_LITELLM_PROFILE` environment variable:

```bash
SWITCHYARD_LITELLM_PROFILE=random docker compose -f deployment/compose.yaml up -d --build --wait

```

Each profile consists of a [`litellm.yaml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/litellm.yaml) (defining the model inventory) and a [`switchyard.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard.toml) (configuring the algorithm). These files reside in `deployment/profiles/<profile_name>/`.

## Summary

- **Install from source** using `pip install git+https://github.com/NVIDIA-NeMo/Switchyard.git` to access the decision-only API required by the LibSy example.
- **Run the LibSy driver** with `python examples/libsy.py` to see the `Step` pattern-matching loop and `EchoClient` in action.
- **Deploy the LiteLLM example** using Docker-Compose from `examples/litellm/deployment/` after configuring your OpenRouter API key in `.env`.
- **Switch routing algorithms** by setting `SWITCHYARD_LITELLM_PROFILE=random` when starting the Docker containers.
- **Inspect routing decisions** via the `x-litellm-model-name` response header or by running [`examples/litellm/examples/python_router.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/litellm/examples/python_router.py).

## Frequently Asked Questions

### What Python version is required to run Switchyard examples?

Switchyard requires Python 3.12 or newer to run the examples. The LibSy example in [`switchyard/libsy/__init__.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/__init__.py) and the LiteLLM Python driver both depend on features available in this version.

### Do I need an API key to run the examples?

You only need an API key for the LiteLLM example, which requires an **OpenRouter API key** stored in `examples/litellm/deployment/.env`. The LibSy example at [`examples/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py) uses a mock `EchoClient` and requires no external credentials.

### What is the difference between the LibSy and LiteLLM examples in Switchyard?

The **LibSy example** is a minimal, pure-Python driver that demonstrates the core routing API without network dependencies. The **LiteLLM example** is a full integration test that runs a local proxy using Docker-Compose, showing how Switchyard acts as a routing plugin within the LiteLLM ecosystem to select models from OpenRouter.

### How do I switch between stage and random routing profiles?

Set the environment variable `SWITCHYARD_LITELLM_PROFILE` to your desired algorithm before running Docker-Compose. For example, use `SWITCHYARD_LITELLM_PROFILE=random docker compose -f deployment/compose.yaml up` to use the random selection algorithm instead of the default stage (efficient-first) algorithm.