How to Run the Examples in the Switchyard Repository

You can run Switchyard examples by either executing the pure-Python LibSy driver directly for minimal routing tests or deploying the LiteLLM routing plugin via Docker-Compose for a full proxy integration with OpenRouter.

The NVIDIA-NeMo/Switchyard repository provides two self-contained example suites that demonstrate its core routing APIs. Whether you want to test the decision-only library or see how the framework integrates with LiteLLM as a routing plugin, you will need Python 3.12 or newer and optionally a Rust toolchain for the server components. Both approaches use actual source files from the codebase to show how to run the examples in Switchyard.

Running the LibSy Python Example

The LibSy example located at examples/libsy.py is a minimal, pure-Python driver that streams a random routing algorithm and prints the selected model and response. It demonstrates how to import the Python bindings, create an algorithm, and handle the Step.CallModel and Step.Done events.

Installation Prerequisites

Because the published nemo-switchyard package does not yet contain the decision-only API used by this example, you must install Switchyard directly from the repository source. From your terminal, run:

pip install git+https://github.com/NVIDIA-NeMo/Switchyard.git

This installation requires Python 3.12 or newer. The optional Rust toolchain is only necessary if you plan to build the server binary from source.

Executing the Driver Script

The examples/libsy.py file contains a Step pattern-matching loop that drives the algorithm and uses an EchoClient to mock responses. Execute the driver with:

python examples/libsy.py

You should see output similar to:

Decision: efficient
Response: {'model': 'efficient', 'outputs': [{'role': 'assistant', 'content': [{'type': 'text', 'text': 'Hello'}]}]}

The script uses algorithms.random to pick between the two dummy targets fast and quality with weights of 1 and 3, respectively. The underlying flow initializes a stage_router or random algorithm and consumes the run_stream iterator, matching on each Step variant to process routing decisions.

Running the LiteLLM Routing Example

The LiteLLM example demonstrates the full plug-in stack: LiteLLM → Switchyard routing plugin → algorithm → model selection → proxy → OpenRouter provider. This deployment requires Docker-Compose because it launches a local OpenRouter-backed proxy alongside a LiteLLM router container.

Configure Environment Variables

Navigate to examples/litellm/deployment/ and create a .env file from the provided template. The only external credential required is an OpenRouter API key:

cp deployment/.env.example deployment/.env

Edit deployment/.env to add your key:

OPENROUTER_API_KEY=sk-...

Start the Proxy with Docker-Compose

The default profile uses stage (efficient-first) routing. From the examples/litellm/ directory, start the services:

docker compose -f deployment/compose.yaml up -d --build --wait

The compose.yaml file mounts the stage profile from deployment/profiles/stage/ and sets the SWITCHYARD_LITELLM_CONFIG environment variable so that the Switchyard routing plugin loads the corresponding switchyard.toml configuration.

Send Test Requests via curl

Any OpenAI-compatible client can hit the proxy at http://127.0.0.1:4000. To test the routing decision with a simple HTTP request:

curl -i http://127.0.0.1:4000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "switchyard",
    "messages": [{"role": "user", "content": "Reply with the word hello."}],
    "max_tokens": 64
  }'

Check the response header x-litellm-model-name to see which concrete provider model Switchyard selected (for example, openrouter/openai/gpt-5.6-sol).

Run the Python Driver Alternative

For a programmatic approach, use the script in examples/litellm/examples/python_router.py. This file constructs a LiteLLM Router, registers the StageRoutingPlugin, and prints the selected model and answer. Run it using the repository's locked Python environment:

uv sync --locked --python 3.12
uv run --locked --env-file deployment/.env python examples/litellm/examples/python_router.py

Stopping the Service

When you have finished testing, tear down the containers:

docker compose -f deployment/compose.yaml down

Switching Between Routing Profiles

The LiteLLM integration ships with two profiles: stage and random. To run the random profile instead of the default stage profile, prefix the Docker-Compose command with the SWITCHYARD_LITELLM_PROFILE environment variable:

SWITCHYARD_LITELLM_PROFILE=random docker compose -f deployment/compose.yaml up -d --build --wait

Each profile consists of a litellm.yaml (defining the model inventory) and a switchyard.toml (configuring the algorithm). These files reside in deployment/profiles/<profile_name>/.

Summary

• Install from source using pip install git+https://github.com/NVIDIA-NeMo/Switchyard.git to access the decision-only API required by the LibSy example.
• Run the LibSy driver with python examples/libsy.py to see the Step pattern-matching loop and EchoClient in action.
• Deploy the LiteLLM example using Docker-Compose from examples/litellm/deployment/ after configuring your OpenRouter API key in .env.
• Switch routing algorithms by setting SWITCHYARD_LITELLM_PROFILE=random when starting the Docker containers.
• Inspect routing decisions via the x-litellm-model-name response header or by running examples/litellm/examples/python_router.py.

Frequently Asked Questions

What Python version is required to run Switchyard examples?

Switchyard requires Python 3.12 or newer to run the examples. The LibSy example in switchyard/libsy/__init__.py and the LiteLLM Python driver both depend on features available in this version.

Do I need an API key to run the examples?

You only need an API key for the LiteLLM example, which requires an OpenRouter API key stored in examples/litellm/deployment/.env. The LibSy example at examples/libsy.py uses a mock EchoClient and requires no external credentials.

What is the difference between the LibSy and LiteLLM examples in Switchyard?

The LibSy example is a minimal, pure-Python driver that demonstrates the core routing API without network dependencies. The LiteLLM example is a full integration test that runs a local proxy using Docker-Compose, showing how Switchyard acts as a routing plugin within the LiteLLM ecosystem to select models from OpenRouter.

How do I switch between stage and random routing profiles?

Set the environment variable SWITCHYARD_LITELLM_PROFILE to your desired algorithm before running Docker-Compose. For example, use SWITCHYARD_LITELLM_PROFILE=random docker compose -f deployment/compose.yaml up to use the random selection algorithm instead of the default stage (efficient-first) algorithm.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}