# How to Set Up Prometheus Metrics Collection in Switchyard Server: Complete Configuration Guide

> Learn how to set up Prometheus metrics collection in Switchyard server easily. Configure the metrics endpoint and start monitoring your server's performance with this quick guide.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**Enable Prometheus metrics in Switchyard by passing the `--metrics-bind` flag when starting the server, which exposes a `/metrics` endpoint on a configurable port for scraper access.**

Switchyard, NVIDIA's modular ML serving framework, includes a native Prometheus exporter built directly into the `switchyard-server` binary. This integration eliminates the need for external exporters or sidecar containers—you simply enable the metrics endpoint via command-line flag or environment variable, and the server handles the rest. The implementation leverages the Rust `prometheus` crate to collect and serve operational telemetry with minimal overhead.

## Understanding Switchyard's Metrics Architecture

The metrics system resides in the `switchyard-server` crate and follows a **registry-based pattern** common to Prometheus client libraries. When enabled, the server spawns an independent async HTTP listener that exclusively serves the `/metrics` endpoint. This separation ensures that metrics scraping never interferes with the main inference API traffic.

According to the NVIDIA-NeMo/Switchyard source code, the key components are:

- **[`crates/switchyard-server/src/main.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/main.rs)** — Parses CLI arguments including `--metrics-bind`, validates the configuration, and initializes the metrics subsystem before the main server loop starts.
- **[`crates/switchyard-server/src/metrics.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/metrics.rs)** — Creates the Prometheus `Registry`, defines metric families (counters, histograms, gauges), and runs the HTTP handler that serializes registry state into the Prometheus exposition format.
- **[`crates/switchyard-server/Cargo.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/Cargo.toml)** — Declares the `prometheus` crate dependency with default features enabled for text format encoding.

## Enabling the Metrics Endpoint

Switchyard ships with metrics **disabled by default**. You have two equivalent methods to activate the exporter:

### Method 1: Command-Line Flag

Pass `--metrics-bind` with a socket address:

```bash
switchyard-server \
  --config routes.toml \
  --port 4000 \
  --metrics-bind 0.0.0.0:9090

```

This starts the metrics server on all interfaces at port **9090**. The endpoint responds at `http://0.0.0.0:9090/metrics`.

### Method 2: Environment Variable

Set `SWITCHYARD_METRICS_BIND` before launching:

```bash
export SWITCHYARD_METRICS_BIND="0.0.0.0:9090"
switchyard-server --config routes.toml --port 4000

```

Environment variables take precedence over CLI defaults but can be overridden by explicit flags.

### Optional: Customizing the Metrics Path

To serve metrics at a non-standard path, export `SWITCHYARD_METRICS_PATH`:

```bash
export SWITCHYARD_METRICS_PATH="/custom/metrics"

```

The default value is `/metrics` when unspecified.

## Configuring Prometheus to Scrape Switchyard

Add a scrape job to your [`prometheus.yml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/prometheus.yml):

```yaml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'switchyard_server'
    static_configs:
      - targets: ['localhost:9090']
    metrics_path: '/metrics'
    scrape_interval: 10s

```

For Kubernetes deployments, use **service discovery**:

```yaml
scrape_configs:
  - job_name: 'switchyard_k8s'
    kubernetes_sd_configs:
      - role: pod
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_label_app]
        action: keep
        regex: switchyard-server
      - source_labels: [__address__]
        target_label: __address__
        replacement: ${1}:9090

```

## Metrics Available

The Switchyard exporter exposes standard server-side telemetry. Based on the implementation in [`crates/switchyard-server/src/metrics.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/metrics.rs), expect these metric families:

| Metric Name | Type | Description |
|-------------|------|-------------|
| `switchyard_requests_total` | Counter | Total requests processed since server start |
| `switchyard_request_latency_seconds` | Histogram | Request duration distribution across configurable buckets |
| `switchyard_active_connections` | Gauge | Currently open client connections |
| `switchyard_errors_total` | Counter | Cumulative error responses by category |

All metrics include default labels for the server instance and, where applicable, route identifiers from your [`routes.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/routes.toml) configuration.

## Verification and Troubleshooting

### Quick Health Check

```bash

# Test metrics endpoint directly

curl http://localhost:9090/metrics

# Verify specific metric presence

curl -s http://localhost:9090/metrics | grep switchyard_requests_total

```

### Common Issues

- **Connection refused**: Confirm the `--metrics-bind` flag was passed and the port is not blocked by firewall rules. Check `netstat -tlnp` for the listening process.
- **Empty response**: The metrics module initializes lazily—send a few requests through the main API to populate counters before scraping.
- **Port conflicts**: Switchyard binds the metrics listener separately from the main API port; ensure `9090` (or your chosen port) is available.

### Debugging Startup

Launch with verbose logging to confirm metrics initialization:

```bash
RUST_LOG=switchyard_server=debug switchyard-server --metrics-bind 0.0.0.0:9090

```

Look for log lines containing `metrics::init` and `prometheus registry` to verify successful setup.

## Performance Considerations

The Switchyard metrics implementation uses **lock-free data structures** where possible and serializes the registry only during scrape requests. Observed overhead in production deployments is typically **<1% of CPU** under moderate load. For high-throughput scenarios:

- Increase `scrape_interval` in Prometheus to reduce serialization frequency
- Enable metric aggregation via recording rules rather than raw histogram queries
- Consider the `--metrics-buffer-size` flag (if available in your version) to tune response buffering

## Summary

- Switchyard server includes a **built-in Prometheus exporter** in [`crates/switchyard-server/src/metrics.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/metrics.rs)—no external components required
- Enable with **`--metrics-bind <address>`** or **`SWITCHYARD_METRICS_BIND`** environment variable
- Default endpoint is **`/metrics`** but customizable via `SWITCHYARD_METRICS_PATH`
- Source files: [`main.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/main.rs) for CLI handling, [`metrics.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/metrics.rs) for registry and HTTP server, [`Cargo.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/Cargo.toml) for crate dependencies

## Frequently Asked Questions

### What port should I use for Switchyard Prometheus metrics?

Use any available TCP port that your Prometheus installation can reach. Port **9090** is conventional and matches Prometheus's own default, but Switchyard accepts any valid socket address. Ensure the port is exposed in your container or firewall configuration.

### Can I disable specific metrics to reduce overhead?

The current Switchyard implementation registers a fixed set of metric families during initialization. Granular disabling requires code modification in [`crates/switchyard-server/src/metrics.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/metrics.rs) to conditionally register collectors based on configuration flags.

### Does Switchyard support the Prometheus remote write protocol?

No. Switchyard's native exporter serves the **Prometheus text format** over HTTP for scraper pull-based collection. For remote write functionality, deploy a Prometheus instance as a sidecar that scrapes Switchyard and forwards via remote write to your backend.