What Is the Purpose of the `switchyard-translation` Crate for Wire Format Conversion?

The switchyard-translation crate converts provider-specific LLM API wire formats—such as OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages—into a provider-neutral intermediate representation (IR) defined by switchyard-protocol, enabling seamless cross-provider request and response handling in the Switchyard inference router.

This crate sits at the heart of NVIDIA's Switchyard project, acting as a pure-Rust translation layer that abstracts away vendor-specific JSON schemas. Whether you're routing a request from an OpenAI-compatible client to an Anthropic backend, or streaming responses between heterogeneous providers, switchyard-translation handles the format conversion without requiring external SDKs, HTTP servers, or Python dependencies.

---

Core Purpose: Provider-Agnostic Wire Format Translation

The fundamental challenge in multi-provider LLM routing is that each vendor defines slightly different request and response schemas. OpenAI's Chat Completions API uses a messages array with role and content fields. Anthropic's Messages API separates system prompts from messages and structures tool use differently. Legacy OpenAI Responses endpoints have yet another layout.

switchyard-translation solves this by establishing a canonical intermediate representation. As defined in src/engine.rs, the TranslationEngine provides methods to decode any supported provider format into neutral IR types like LlmRequest and AggLlmResponse, then re-encode to any other supported format.

// From src/engine.rs - the core translation interface
pub struct TranslationEngine {
    registry: Arc<FormatRegistry>,
    stream_registry: Arc<StreamFormatRegistry>,
}

impl TranslationEngine {
    pub fn translate_request(
        &self,
        from: FormatId,
        to: FormatId,
        body: &Value,
        policy: &TranslationPolicy,
    ) -> Result<TranslatedRequest>;
}

According to the crate README, this design makes the crate "usable from Python (via PyO3) as well as from pure-Rust services" without pulling in heavy dependencies.

---

Architecture Components

Format Registry and Codecs

The TranslationEngine delegates actual serialization work to a FormatRegistry containing FormatCodec implementations for each supported provider. As shown in [src/engine.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/engine.rs), codecs are registered at engine construction and looked up dynamically during translation.

The built-in codec modules live under src/codecs/:

Codec Module	Handles	Source
openai_chat.rs	OpenAI Chat Completions v1	[src/codecs/openai_chat.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/codecs/openai_chat.rs)
openai_responses.rs	OpenAI legacy Responses endpoint	[src/codecs/openai_responses.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/codecs/openai_responses.rs)
anthropic.rs	Anthropic Messages API	[src/codecs/anthropic.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/codecs/anthropic.rs)

Each codec implements two core traits: RequestEncoder/RequestDecoder for the initial request, and ResponseEncoder/ResponseDecoder for the final response. The registry pattern allows new providers to be added without modifying the engine itself.

Streaming Translation

Real-time LLM responses require special handling. The StreamFormatRegistry manages StreamCodec implementations that handle Server-Sent Events (SSE) and other streaming formats. The StreamTranslationState tracks partial messages across chunks, enabling conversion of live streams between formats.

// Stream translation preserves raw JSON when formats match for efficiency
pub fn translate_stream_event(
    &self,
    state: &mut StreamTranslationState,
    from: FormatId,
    to: FormatId,
    event: Value,
) -> Result<Value>;

Policy-Driven Translation Safety

Translation isn't just mechanical JSON mapping. The TranslationPolicy struct allows callers to inject safety and compatibility rules:

• Tool ID sanitization: Anthropic tool IDs must match specific patterns
• Message truncation: Enforcing token limits across providers with different context windows
• Schema validation: Rejecting requests with unsupported features for the target provider

Policy is passed through every translation method and consulted by codecs at decision points.

---

Practical Usage Examples

Translating Requests Between Providers

This example demonstrates converting an OpenAI Chat Completions request to Anthropic Messages format using the TranslationEngine:

use switchyard_translation::{
    engine::TranslationEngine,
    format::FormatId,
    policy::TranslationPolicy,
    error::Result,
};
use serde_json::json;

fn route_to_anthropic() -> Result<serde_json::Value> {
    let engine = TranslationEngine::default();
    
    // Incoming OpenAI-compatible request
    let openai_request = json!({
        "model": "gpt-4o",
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Explain quantum computing"}
        ],
        "temperature": 0.7,
        "tools": [{
            "type": "function",
            "function": {
                "name": "search",
                "description": "Search for information",
                "parameters": {"type": "object", "properties": {}}
            }
        }]
    });

    // Translate to Anthropic format
    let translated = engine.translate_request(
        FormatId::OpenAiChat,      // Source: OpenAI Chat Completions
        FormatId::AnthropicMessages, // Target: Anthropic Messages API
        &openai_request,
        &TranslationPolicy::default(),
    )?;

    // ReturnsAnthropic-compatible JSON with proper field mapping
    Ok(translated.body)
}

The TranslatedRequest struct returned by translate_request contains:

• body: The re-encoded JSON value
• diagnostics: Vec of TranslationDiagnostic for logging and debugging
• metadata: Provider-specific routing hints extracted during translation

Handling Streaming Responses

Cross-provider streaming requires maintaining state across SSE chunks. The StreamTranslationState tracks partial tool calls and message content:

use switchyard_translation::{
    engine::TranslationEngine,
    format::FormatId,
    stream::StreamTranslationState,
};

struct StreamingRouter {
    engine: TranslationEngine,
    state: StreamTranslationState,
}

impl StreamingRouter {
    fn forward_chunk(
        &mut self,
        chunk: serde_json::Value,  // Incoming OpenAI SSE data
    ) -> Result<Vec<u8>> {
        // Decode preserving format-specific metadata
        let preserved = self.engine.decode_stream_event(
            &mut self.state,
            FormatId::OpenAiChat,
            chunk,
        )?;

        // Re-encode for Anthropic client
        let anthropic_chunks = self.engine.encode_stream_event(
            &mut self.state,
            FormatId::AnthropicMessages,
            preserved,
        )?;

        // Serialize to bytes for HTTP response
        Ok(serde_json::to_vec(&anthropic_chunks)?)
    }
}

Python Integration via PyO3

The crate exposes Python bindings in [src/lib.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/lib.rs), making the same engine available to the Switchyard Python launchers:

import switchyard_translation as st

# Create engine instance (calls Rust constructor)

engine = st.TranslationEngine()

# Define translation policy

policy = st.TranslationPolicy()
policy.sanitize_anthropic_tool_ids = True
policy.max_input_tokens = 128000

# Translate request from dict

result = engine.translate_request(
    source="openai_chat",
    target="anthropic",
    body={
        "model": "gpt-4o",
        "messages": [{"role": "user", "content": "Hello"}]
    },
    policy=policy
)

# Access translated body and diagnostics

print(result.body)           # Dict in Anthropic format

print(result.diagnostics)    # List of translation notes

The PyO3 layer adds minimal overhead by holding the TranslationEngine in a Python-exposed struct and converting between serde_json::Value and Python dicts.

---

Key Design Decisions

Zero External Dependencies for Core Logic

Per the README, the crate deliberately excludes:

• HTTP client/server libraries
• Provider SDKs (no openai, anthropic crates)
• Python runtime dependencies
• Async runtimes (Tokio is optional for stream utilities)

This constraint ensures the translation logic can run in constrained environments—including inside the Python process without GIL contention issues—and keeps compile times minimal.

Diagnostics-First Error Handling

Rather than failing fast on schema mismatches, the engine accumulates TranslationDiagnostic entries and attempts partial translation. This enables:

• Logging and observability of "fixable" incompatibilities
• Graceful degradation when optional features aren't supported
• A/B testing of translation strategies

Format ID Type Safety

The FormatId enum defined in src/lib.rs provides compile-time guarantees about supported providers:

pub enum FormatId {
    OpenAiChat,
    OpenAiResponses,
    AnthropicMessages,
}

Attempting to translate between unsupported format pairs results in a clear error at the registry lookup stage, not deep in codec logic.

---

Integration with Switchyard Ecosystem

switchyard-translation consumes types from switchyard-protocol and provides services to higher-level crates:

• switchyard-core: Uses the engine for incoming request normalization before routing decisions
• switchyard-llm: Invokes translation for backend-specific request crafting
• switchyard-py: Exposes Python bindings wrapping the engine

The crate sits in the middle of the dependency stack: protocol types below, application logic above. As noted in [src/lib.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/lib.rs), public exports are carefully curated to expose only the stable API surface.

---

Summary

• switchyard-translation provides wire format conversion between OpenAI Chat, OpenAI Responses, and Anthropic Messages APIs using a provider-neutral IR from switchyard-protocol.

• Core abstraction is the TranslationEngine in src/engine.rs, which orchestrates codec-based encoding/decoding with policy enforcement and diagnostic collection.

• Zero-dependency design enables deployment in both native Rust services and Python environments via PyO3 bindings.

• Streaming support via StreamTranslationState and dedicated stream codecs handles real-time SSE conversion between formats.

• Extensible architecture through the FormatRegistry pattern allows new LLM providers to be added without engine modifications.

• Policy-driven translation ensures safety and compatibility through configurable rules for tool ID sanitization, truncation, and validation.

---

Frequently Asked Questions

What wire formats does switchyard-translation currently support?

The crate supports OpenAI Chat Completions, OpenAI legacy Responses, and Anthropic Messages formats as of the current implementation. Each format has dedicated codec modules under src/codecs/. The registry-based architecture allows additional providers to be registered by implementing the FormatCodec trait, though built-in support is limited to these three major APIs. See [src/codecs/mod.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/codecs/mod.rs) for the module structure.

How does streaming translation handle partial messages across formats?

The StreamTranslationState struct in [src/stream.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/stream.rs) maintains internal buffers for incomplete tool calls, function arguments, and message content. When decoding a provider-specific SSE event, the engine accumulates partial data; when encoding for the target format, it emits complete events according to that provider's streaming protocol. Raw JSON is preserved when source and target formats match, avoiding unnecessary re-serialization.

Can I use switchyard-translation without the rest of Switchyard?

Yes. The crate is self-contained and exposes a minimal public API through lib.rs. You can depend on it directly from crates.io (or a Git source) to perform format conversion in standalone tools. The engine requires only switchyard-protocol for IR types and has no mandatory async runtime or HTTP dependencies. This design is intentional per the README documentation.

What happens when a feature isn't supported by the target provider?

The TranslationPolicy determines behavior for incompatibilities. By default, the engine attempts best-effort translation and records diagnostics. For example, if an OpenAI request includes logprobs but Anthropic doesn't support this feature, the codec may strip the field and emit a warning diagnostic rather than failing. Stricter policies can be configured to reject such requests at translation time. Check [src/policy.rs](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-translation/src/policy.rs) for available policy options.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}