Switchyard Server Health Check Endpoints: Complete 2024 Guide

The Switchyard server exposes a single /health endpoint that returns {"status":"ok"} to indicate the server is operational.

The NVIDIA‑NeMo Switchyard project provides a lightweight, Rust‑based inference routing server. Understanding its health check implementation is critical for deploying reliable production monitoring and load‑balancing setups. This guide examines the exact health endpoint exposed by switchyard‑server, its implementation details, and how to integrate it into your observability stack.

Health Endpoint Overview

The Switchyard server registers exactly one built‑in health check route. This minimal design prioritizes simplicity while providing sufficient signal for container orchestrators and load balancers.

HTTP Route Registration

In crates/switchyard-server/src/lib.rs at line 509, the Axum router attaches the health handler:

.route("/health", get(health))

This registers a GET handler for the /health path. No other health‑related routes—such as /ready or /live—are defined in the core server implementation.

Handler Implementation

The health function at line 1126 in the same file constructs the response:

async fn health() -> Json<HealthResponse> {
    Json(HealthResponse {
        status: "ok".to_string(),
    })
}

The response struct serializes to:

{"status":"ok"}

HTTP status code is 200 OK. The server returns this payload immediately without checking downstream dependencies, making it suitable for basic liveness probes.

Testing the Health Endpoint

The Switchyard test suite validates this behavior in crates/switchyard-server/tests/server.rs at lines 1615‑1617:

let resp = client.get("/health").send().await;
assert_eq!(resp.status(), 200);
assert_eq!(resp.json::<serde_json::Value>().await.unwrap(), json!({"status":"ok"}));

Command-Line Verification

Use curl to quickly verify a running instance:

curl -s http://localhost:4000/health | jq .

Expected output:

{
  "status": "ok"
}

Programmatic Health Checks

For Python‑based monitoring or Kubernetes readiness scripts:

import asyncio
import httpx

async def check_switchyard_health(url: str = "http://localhost:4000/health") -> bool:
    """Return True if Switchyard server reports healthy."""
    async with httpx.AsyncClient(timeout=5.0) as client:
        try:
            response = await client.get(url)
            response.raise_for_status()
            data = response.json()
            return data.get("status") == "ok"
        except (httpx.ConnectError, httpx.HTTPStatusError, KeyError):
            return False

# Example usage

if __name__ == "__main__":
    healthy = asyncio.run(check_switchyard_health())
    print(f"Switchyard healthy: {healthy}")

Production Monitoring Patterns

Since Switchyard provides only basic liveness, implement these complementary patterns:

• Liveness probe: Use /health directly. Kubernetes example:

  livenessProbe:
    httpGet:
      path: /health
      port: 4000
    initialDelaySeconds: 5
    periodSeconds: 10

• Readiness probe: Extend with custom middleware or sidecar if dependency health (e.g., upstream model servers) is required. The core /health endpoint does not validate downstream connectivity.

• Metrics exposure: Switchyard does not expose Prometheus metrics by default. Integrate a separate metrics sidecar or patch the server to add /metrics if observability beyond health status is needed.

Summary

• Switchyard server exposes one health check endpoint: GET /health
• Returns 200 OK with JSON body {"status":"ok"}
• Implementation located in crates/switchyard-server/src/lib.rs (route at line 509, handler at line 1126)
• Validated by test suite in crates/switchyard-server/tests/server.rs
• No built‑in readiness, liveness split, or metrics endpoints—extend externally if required

Frequently Asked Questions

Does Switchyard support a readiness probe separate from liveness?

No. The server exposes only the /health endpoint, which performs no dependency checks. For readiness that validates upstream model availability, implement a custom wrapper or sidecar that queries Switchyard's routing state before reporting ready.

What HTTP status codes does the health endpoint return?

The handler always returns 200 OK when the server process is running. There is no scenario where /health returns an error status from the core implementation—failure to respond indicates the server has crashed or is unreachable.

Can I customize the health response format?

Not without modifying the source. The HealthResponse struct in lib.rs hardcodes the status field with string value "ok". To add fields like version or timestamp, fork the repository and extend the struct and handler accordingly.

Is there a /metrics endpoint for Prometheus monitoring?

No. The base Switchyard server does not expose Prometheus‑compatible metrics. Monitor at the infrastructure level (CPU, memory, network) or contribute a metrics middleware patch to the Axum router in crates/switchyard-server/src/lib.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}