# What Is the Purpose of the `switchyard-translation` Crate for Wire Format Conversion? > Discover how switchyard-translation converts LLM API wire formats like OpenAI and Anthropic into a neutral representation. Enable seamless cross-provider request handling in Switchyard. - Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard) - Tags: internals - Published: 2026-08-15 --- **The `switchyard-translation` crate converts provider-specific LLM API wire formats—such as OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages—into a provider-neutral intermediate representation (IR) defined by `switchyard-protocol`, enabling seamless cross-provider request and response handling in the Switchyard inference router.** This crate sits at the heart of NVIDIA's Switchyard project, acting as a pure-Rust translation layer that abstracts away vendor-specific JSON schemas. Whether you're routing a request from an OpenAI-compatible client to an Anthropic backend, or streaming responses between heterogeneous providers, `switchyard-translation` handles the format conversion without requiring external SDKs, HTTP servers, or Python dependencies. --- ## Core Purpose: Provider-Agnostic Wire Format Translation The fundamental challenge in multi-provider LLM routing is that each vendor defines slightly different request and response schemas. OpenAI's Chat Completions API uses a `messages` array with `role` and `content` fields. Anthropic's Messages API separates `system` prompts from `messages` and structures tool use differently. Legacy OpenAI Responses endpoints have yet another layout. `switchyard-translation` solves this by establishing a **canonical intermediate representation**. As defined in [`src/engine.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/engine.rs), the `TranslationEngine` provides methods to decode any supported provider format into neutral IR types like `LlmRequest` and `AggLlmResponse`, then re-encode to any other supported format. ```rust // From src/engine.rs - the core translation interface pub struct TranslationEngine { registry: Arc, stream_registry: Arc, } impl TranslationEngine { pub fn translate_request( &self, from: FormatId, to: FormatId, body: &Value, policy: &TranslationPolicy, ) -> Result; } ``` According to the [crate README](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/README.md), this design makes the crate "usable from Python (via PyO3) as well as from pure-Rust services" without pulling in heavy dependencies. --- ## Architecture Components ### Format Registry and Codecs The `TranslationEngine` delegates actual serialization work to a `FormatRegistry` containing `FormatCodec` implementations for each supported provider. As shown in [[`src/engine.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/engine.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/engine.rs), codecs are registered at engine construction and looked up dynamically during translation. The built-in codec modules live under `src/codecs/`: | Codec Module | Handles | Source | |-------------|---------|--------| | [`openai_chat.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/openai_chat.rs) | OpenAI Chat Completions v1 | [[`src/codecs/openai_chat.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/codecs/openai_chat.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/codecs/openai_chat.rs) | | [`openai_responses.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/openai_responses.rs) | OpenAI legacy Responses endpoint | [[`src/codecs/openai_responses.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/codecs/openai_responses.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/codecs/openai_responses.rs) | | [`anthropic.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/anthropic.rs) | Anthropic Messages API | [[`src/codecs/anthropic.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/codecs/anthropic.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/codecs/anthropic.rs) | Each codec implements two core traits: `RequestEncoder`/`RequestDecoder` for the initial request, and `ResponseEncoder`/`ResponseDecoder` for the final response. The registry pattern allows new providers to be added without modifying the engine itself. ### Streaming Translation Real-time LLM responses require special handling. The `StreamFormatRegistry` manages `StreamCodec` implementations that handle Server-Sent Events (SSE) and other streaming formats. The [`StreamTranslationState`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/stream.rs) tracks partial messages across chunks, enabling conversion of live streams between formats. ```rust // Stream translation preserves raw JSON when formats match for efficiency pub fn translate_stream_event( &self, state: &mut StreamTranslationState, from: FormatId, to: FormatId, event: Value, ) -> Result; ``` ### Policy-Driven Translation Safety Translation isn't just mechanical JSON mapping. The [`TranslationPolicy`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/policy.rs) struct allows callers to inject safety and compatibility rules: - **Tool ID sanitization**: Anthropic tool IDs must match specific patterns - **Message truncation**: Enforcing token limits across providers with different context windows - **Schema validation**: Rejecting requests with unsupported features for the target provider Policy is passed through every translation method and consulted by codecs at decision points. --- ## Practical Usage Examples ### Translating Requests Between Providers This example demonstrates converting an OpenAI Chat Completions request to Anthropic Messages format using the `TranslationEngine`: ```rust use switchyard_translation::{ engine::TranslationEngine, format::FormatId, policy::TranslationPolicy, error::Result, }; use serde_json::json; fn route_to_anthropic() -> Result { let engine = TranslationEngine::default(); // Incoming OpenAI-compatible request let openai_request = json!({ "model": "gpt-4o", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Explain quantum computing"} ], "temperature": 0.7, "tools": [{ "type": "function", "function": { "name": "search", "description": "Search for information", "parameters": {"type": "object", "properties": {}} } }] }); // Translate to Anthropic format let translated = engine.translate_request( FormatId::OpenAiChat, // Source: OpenAI Chat Completions FormatId::AnthropicMessages, // Target: Anthropic Messages API &openai_request, &TranslationPolicy::default(), )?; // ReturnsAnthropic-compatible JSON with proper field mapping Ok(translated.body) } ``` The `TranslatedRequest` struct returned by `translate_request` contains: - `body`: The re-encoded JSON value - `diagnostics`: Vec of `TranslationDiagnostic` for logging and debugging - `metadata`: Provider-specific routing hints extracted during translation ### Handling Streaming Responses Cross-provider streaming requires maintaining state across SSE chunks. The `StreamTranslationState` tracks partial tool calls and message content: ```rust use switchyard_translation::{ engine::TranslationEngine, format::FormatId, stream::StreamTranslationState, }; struct StreamingRouter { engine: TranslationEngine, state: StreamTranslationState, } impl StreamingRouter { fn forward_chunk( &mut self, chunk: serde_json::Value, // Incoming OpenAI SSE data ) -> Result> { // Decode preserving format-specific metadata let preserved = self.engine.decode_stream_event( &mut self.state, FormatId::OpenAiChat, chunk, )?; // Re-encode for Anthropic client let anthropic_chunks = self.engine.encode_stream_event( &mut self.state, FormatId::AnthropicMessages, preserved, )?; // Serialize to bytes for HTTP response Ok(serde_json::to_vec(&anthropic_chunks)?) } } ``` ### Python Integration via PyO3 The crate exposes Python bindings in [[`src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/lib.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/lib.rs), making the same engine available to the Switchyard Python launchers: ```python import switchyard_translation as st # Create engine instance (calls Rust constructor) engine = st.TranslationEngine() # Define translation policy policy = st.TranslationPolicy() policy.sanitize_anthropic_tool_ids = True policy.max_input_tokens = 128000 # Translate request from dict result = engine.translate_request( source="openai_chat", target="anthropic", body={ "model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}] }, policy=policy ) # Access translated body and diagnostics print(result.body) # Dict in Anthropic format print(result.diagnostics) # List of translation notes ``` The PyO3 layer adds minimal overhead by holding the `TranslationEngine` in a Python-exposed struct and converting between `serde_json::Value` and Python dicts. --- ## Key Design Decisions ### Zero External Dependencies for Core Logic Per the [README](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/README.md), the crate deliberately excludes: - HTTP client/server libraries - Provider SDKs (no `openai`, `anthropic` crates) - Python runtime dependencies - Async runtimes (Tokio is optional for stream utilities) This constraint ensures the translation logic can run in constrained environments—including inside the Python process without GIL contention issues—and keeps compile times minimal. ### Diagnostics-First Error Handling Rather than failing fast on schema mismatches, the engine accumulates `TranslationDiagnostic` entries and attempts partial translation. This enables: - Logging and observability of "fixable" incompatibilities - Graceful degradation when optional features aren't supported - A/B testing of translation strategies ### Format ID Type Safety The `FormatId` enum defined in [`src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/lib.rs) provides compile-time guarantees about supported providers: ```rust pub enum FormatId { OpenAiChat, OpenAiResponses, AnthropicMessages, } ``` Attempting to translate between unsupported format pairs results in a clear error at the registry lookup stage, not deep in codec logic. --- ## Integration with Switchyard Ecosystem `switchyard-translation` consumes types from `switchyard-protocol` and provides services to higher-level crates: - **`switchyard-core`**: Uses the engine for incoming request normalization before routing decisions - **`switchyard-llm`**: Invokes translation for backend-specific request crafting - **`switchyard-py`**: Exposes Python bindings wrapping the engine The crate sits in the middle of the dependency stack: protocol types below, application logic above. As noted in [[`src/lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/lib.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/lib.rs), public exports are carefully curated to expose only the stable API surface. --- ## Summary - **`switchyard-translation` provides wire format conversion** between OpenAI Chat, OpenAI Responses, and Anthropic Messages APIs using a provider-neutral IR from `switchyard-protocol`. - **Core abstraction is the `TranslationEngine`** in [`src/engine.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/engine.rs), which orchestrates codec-based encoding/decoding with policy enforcement and diagnostic collection. - **Zero-dependency design** enables deployment in both native Rust services and Python environments via PyO3 bindings. - **Streaming support** via `StreamTranslationState` and dedicated stream codecs handles real-time SSE conversion between formats. - **Extensible architecture** through the `FormatRegistry` pattern allows new LLM providers to be added without engine modifications. - **Policy-driven translation** ensures safety and compatibility through configurable rules for tool ID sanitization, truncation, and validation. --- ## Frequently Asked Questions ### What wire formats does `switchyard-translation` currently support? The crate supports **OpenAI Chat Completions**, **OpenAI legacy Responses**, and **Anthropic Messages** formats as of the current implementation. Each format has dedicated codec modules under `src/codecs/`. The registry-based architecture allows additional providers to be registered by implementing the `FormatCodec` trait, though built-in support is limited to these three major APIs. See [[`src/codecs/mod.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/codecs/mod.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/codecs/mod.rs) for the module structure. ### How does streaming translation handle partial messages across formats? The `StreamTranslationState` struct in [[`src/stream.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/stream.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/stream.rs) maintains internal buffers for incomplete tool calls, function arguments, and message content. When decoding a provider-specific SSE event, the engine accumulates partial data; when encoding for the target format, it emits complete events according to that provider's streaming protocol. Raw JSON is preserved when source and target formats match, avoiding unnecessary re-serialization. ### Can I use `switchyard-translation` without the rest of Switchyard? Yes. The crate is **self-contained** and exposes a minimal public API through [`lib.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/lib.rs). You can depend on it directly from crates.io (or a Git source) to perform format conversion in standalone tools. The engine requires only `switchyard-protocol` for IR types and has no mandatory async runtime or HTTP dependencies. This design is intentional per the [README](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/README.md) documentation. ### What happens when a feature isn't supported by the target provider? The `TranslationPolicy` determines behavior for incompatibilities. By default, the engine attempts **best-effort translation** and records diagnostics. For example, if an OpenAI request includes `logprobs` but Anthropic doesn't support this feature, the codec may strip the field and emit a warning diagnostic rather than failing. Stricter policies can be configured to reject such requests at translation time. Check [[`src/policy.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/src/policy.rs)](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/policy.rs) for available policy options.