# How Switchyard Handles LLM Request Metadata: Session ID and Correlation ID

> Learn how Switchyard handles LLM request metadata, normalizing HTTP headers into a Metadata struct for session and correlation IDs, enabling distributed tracing and end-to-end observability.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: internals
- Published: 2026-08-21

---

**Switchyard normalizes heterogeneous HTTP headers into a canonical `Metadata` struct that extracts session IDs and correlation IDs, attaches them to internal request objects for distributed tracing, and propagates them downstream to LLM backends to maintain end-to-end observability.**

Switchyard is NVIDIA's open-source inference router for large language models. Understanding how Switchyard handles LLM request metadata like session ID and correlation ID is critical for debugging multi-turn conversations and maintaining trace context across distributed services. The system implements a vendor-agnostic normalization layer that reconciles conflicting header conventions from OpenAI, Claude, Codex, and internal NVIDIA services into a unified metadata envelope.

## From HTTP Headers to Structured Metadata

The normalization process begins in [[`crates/protocol/src/metadata.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/protocol/src/metadata.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/protocol/src/metadata.rs), where the `Metadata::from_headers` function transforms incoming HTTP headers into a structured representation. This function consults a **precedence table** defined in `HEADER_CONFIG` to resolve conflicts, ensuring that explicit `x-switchyard-*` headers take priority over vendor-specific aliases. The implementation also handles **nested JSON values** in headers (such as `x-codex-turn-metadata.session_id`) and falls back to flat header values when JSON parsing yields empty results.

## Session ID and Correlation ID Extraction

Switchyard extracts two critical identifiers from the normalized headers to track requests through the system.

### Session ID for Conversation State

The **`session_id`** field provides a stable identifier for multi-turn conversations, enabling Switchyard to maintain context across related requests. The extractor maps multiple vendor headers to this canonical field, including `x-switchyard-session-id`, `x-claude-code-session-id`, `x-nemo-relay-session-id`, and the generic `session-id`. This aliasing allows clients using different LLM providers to integrate without modifying their header conventions.

### Correlation ID for Distributed Tracing

The **`correlation_id`** (also referenced as `request_id` in tracing contexts) enables end-to-end request tracing across service boundaries. Switchyard extracts this value from `x-switchyard-request-id` or its aliases `x-request-id` and `x-client-request-id`, then injects it into OpenTelemetry spans and Prometheus metrics.

## Server-Side Metadata Processing

When requests arrive at the Rust HTTP server, the metadata integration happens at the entry point and continues through the observability stack.

### Request Lifecycle Initialization

In [[`crates/switchyard-server/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/lib.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/lib.rs), the inbound `HeaderMap` is immediately converted into a `Metadata` instance using `Metadata::from_headers`. This metadata is then attached to the internal `switchyard_protocol::Request` object, making the identifiers available to all downstream processing stages including routing algorithms and sub-agent dispatchers.

### Observability Integration

The [[`crates/switchyard-server/src/observability.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/observability.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/observability.rs) module creates a tracing span for each incoming request that explicitly records `session_id` and `correlation_id`. This integration ensures that logs and metrics emitted during request processing carry these identifiers, allowing operators to correlate events across Switchyard instances and external services.

## Routing and Downstream Propagation

Switchyard does not merely extract metadata for internal use; it actively propagates these identifiers to maintain context across service boundaries.

### Sub-Agent State Management

The `Metadata` struct drives sub-agent routing decisions through fields like `is_subagent` and `is_subagent_work`. The **session ID** serves as a key for per-session state in Switchyard's routing algorithms, enabling sticky routing where subsequent requests in a conversation reach the same backend or context pool.

### Backend Header Forwarding

When forwarding requests to LLM backends via the `libsy-llm-client` crate, specifically in [[`crates/libsy-llm-client/src/client.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/src/client.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/src/client.rs), Switchyard includes the correlation ID in outbound HTTP headers. This propagation ensures that backend services can participate in the same distributed trace, maintaining observability continuity from the initial client request through the final LLM provider response.

## Implementing Metadata Handling in Python

Switchyard exposes its metadata handling through Python bindings defined in [[`crates/switchyard-py/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-py/src/lib.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-py/src/lib.rs). The following example demonstrates how to construct a request with explicit session and correlation identifiers:

```python
import switchyard
from switchyard import Request, Metadata

# Build a request with explicit metadata headers

metadata_headers = [
    ("x-switchyard-session-id", "session-1234"),
    ("x-switchyard-request-id", "trace-5678"),
]
metadata = Metadata.from_headers(metadata_headers)

req = Request(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello"}],
    metadata=metadata,
)

# Send through Switchyard (the Python façade forwards to the Rust server)

response = switchyard.run(req)
print(response.metadata.session_id)      # → "session-1234"

print(response.metadata.correlation_id) # → "trace-5678"

```

## Summary

- Switchyard normalizes vendor-specific HTTP headers into a canonical `Metadata` struct using `Metadata::from_headers` in [`crates/protocol/src/metadata.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/protocol/src/metadata.rs).
- **Session IDs** enable multi-turn conversation tracking across requests, extracted from headers like `x-switchyard-session-id` and `x-claude-code-session-id`.
- **Correlation IDs** facilitate distributed tracing, sourced from `x-switchyard-request-id` or its aliases and recorded in OpenTelemetry spans via the observability module.
- The server attaches metadata to internal `Request` objects at the entry point in [`crates/switchyard-server/src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/lib.rs).
- Downstream propagation to LLM backends occurs through [`crates/libsy-llm-client/src/client.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/src/client.rs), ensuring end-to-end trace continuity.

## Frequently Asked Questions

### What HTTP headers does Switchyard accept for session identification?

Switchyard accepts multiple header aliases for session IDs, including `x-switchyard-session-id`, `x-claude-code-session-id`, `x-nemo-relay-session-id`, and the generic `session-id`. The system uses a precedence table where explicit Switchyard headers take priority over vendor-specific alternatives.

### How does Switchyard resolve conflicting metadata headers from different LLM providers?

The `Metadata::from_headers` function implements a precedence hierarchy defined in `HEADER_CONFIG`, ensuring that `x-switchyard-*` headers override vendor aliases like OpenAI or Claude headers. For nested JSON values (e.g., `x-codex-turn-metadata.session_id`), the parser extracts internal fields while falling back to flat header values when JSON is absent or empty.

### Can correlation IDs be traced through downstream LLM providers?

Yes. When Switchyard forwards requests via [`crates/libsy-llm-client/src/client.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy-llm-client/src/client.rs), it includes the correlation ID in outbound HTTP headers, allowing backend LLM services to participate in the same distributed trace. This propagation ensures that logs and metrics from both Switchyard and the upstream provider share the same request identifier.

### How is session state maintained across multiple requests in Switchyard?

Switchyard uses the normalized `session_id` field as a key for per-session state in its routing algorithms, particularly when handling sub-agent requests. This enables sticky routing and context preservation across multiple turns of a conversation, even when requests traverse different server instances.