How to Implement Custom LLM Transformers for Provider Compatibility in AxonHub

Implementing custom LLM transformers in AxonHub requires creating a struct that satisfies the transformer.Outbound interface by defining TransformRequest and TransformResponse methods to translate between AxonHub's standard OpenAI-compatible schema and your target provider's native format.

AxonHub, the open-source LLM gateway developed by looplj/axonhub, isolates provider-specific quirks behind a unified transformation layer. When integrating a new Large Language Model vendor that isn't natively supported, you must implement custom LLM transformers that handle bidirectional translation between AxonHub's canonical OpenAI-compatible requests and the provider's proprietary API format.

Understanding the Transformer Architecture

The Outbound Interface

The core contract for outbound transformations is defined in llm/transformer/interfaces.go. Every custom transformer must implement the transformer.Outbound interface, which requires two methods: TransformRequest and TransformResponse. These methods handle the conversion from AxonHub's generic *transformer.Request to an *http.Request ready for the provider, and back from the provider's *http.Response to AxonHub's generic *transformer.Response.

Key Source Files

Several files provide the foundation for implementing custom LLM transformers:

• llm/transformer/interfaces.go – Defines the Outbound interface, error types like ErrInvalidRequest, and shared structs.
• llm/transformer/openai/shared.go – Contains helper functions such as NormalizeBaseURL and common payload definitions that many providers reuse.
• llm/transformer/openai/outbound.go – Serves as the canonical reference implementation for providers that natively speak OpenAI's schema.
• llm/transformer/zai/outbound.go – Shows how a transformer can add extra validation and model-specific logic while delegating to the OpenAI transformer internally.
• internal/server/orchestrator/outbound.go – Routes requests to the appropriate transformer based on channel configuration.

Implementing a Custom LLM Transformer

Step 1: Define Configuration and Constructor

Begin by creating a configuration struct that mirrors the YAML fields your provider requires. Provide two constructor functions: NewOutboundTransformer for quick testing and NewOutboundTransformerWithConfig for production use by the orchestrator.

package myai

import (
	"context"
	"fmt"
	"net/http"

	"github.com/looplj/axonhub/llm/transformer"
)

type Config struct {
	BaseURL string `yaml:"base_url"`
	APIKey  string `yaml:"api_key"`
}

type OutboundTransformer struct {
	baseURL string
	apiKey  string
}

func NewOutboundTransformer(baseURL, apiKey string) (transformer.Outbound, error) {
	return &OutboundTransformer{
		baseURL: transformer.NormalizeBaseURL(baseURL, ""),
		apiKey:  apiKey,
	}, nil
}

func NewOutboundTransformerWithConfig(cfg *Config) (transformer.Outbound, error) {
	if cfg.BaseURL == "" || cfg.APIKey == "" {
		return nil, fmt.Errorf("%w: missing configuration", transformer.ErrInvalidRequest)
	}
	return NewOutboundTransformer(cfg.BaseURL, cfg.APIKey)
}

Step 2: Transform Requests to Provider Format

Implement TransformRequest to convert the generic *transformer.Request into an *http.Request targeting your provider's endpoint. Validate required fields, marshal the payload, and set authentication headers.

func (t *OutboundTransformer) TransformRequest(ctx context.Context, req *transformer.Request) (*http.Request, error) {
	if req.Model == "" {
		return nil, fmt.Errorf("%w: model is required", transformer.ErrInvalidRequest)
	}
	
	payload := map[string]interface{}{
		"model":    req.Model,
		"messages": req.Messages,
		"stream":   req.Stream,
	}
	
	body, err := json.Marshal(payload)
	if err != nil {
		return nil, fmt.Errorf("%w: failed to marshal request", transformer.ErrInvalidRequest)
	}
	
	httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, 
		t.baseURL+"/chat/completions", bytes.NewReader(body))
	if err != nil {
		return nil, err
	}
	
	httpReq.Header.Set("Authorization", "Bearer "+t.apiKey)
	httpReq.Header.Set("Content-Type", "application/json")
	
	return httpReq, nil
}

Step 3: Transform Responses to Canonical Format

Implement TransformResponse to parse the provider's HTTP response and convert it into AxonHub's generic *transformer.Response. Handle error status codes and unmarshal the JSON body into the standard schema.

func (t *OutboundTransformer) TransformResponse(_ context.Context, resp *http.Response) (*transformer.Response, error) {
	defer resp.Body.Close()
	
	if resp.StatusCode >= 400 {
		return nil, fmt.Errorf("%w: provider returned %d", transformer.ErrInvalidResponse, resp.StatusCode)
	}
	
	var raw struct {
		Choices []struct {
			Message transformer.Message `json:"message"`
		} `json:"choices"`
		Usage struct {
			PromptTokens     int `json:"prompt_tokens"`
			CompletionTokens int `json:"completion_tokens"`
		} `json:"usage"`
	}
	
	if err := json.NewDecoder(resp.Body).Decode(&raw); err != nil {
		return nil, fmt.Errorf("%w: failed to decode response", transformer.ErrInvalidResponse)
	}
	
	return &transformer.Response{
		Choices: raw.Choices,
		Usage: transformer.Usage{
			PromptTokens:     raw.Usage.PromptTokens,
			CompletionTokens: raw.Usage.CompletionTokens,
		},
	}, nil
}

Step 4: Register and Test

Place your package under llm/transformer/myai/ and ensure it is importable. The orchestrator in internal/server/orchestrator/outbound.go automatically discovers transformers based on the type field in channel configuration, so no explicit registration is required beyond making the package available.

Create a test file outbound_test.go following the pattern in llm/transformer/openai/outbound_test.go to validate request transformation and response parsing.

Wiring Your Transformer into AxonHub

Once your custom LLM transformer is implemented, integrate it into the AxonHub gateway by updating your channel configuration. The orchestrator selects the appropriate transformer based on the type field in your YAML configuration.

channels:
  - name: myai-production
    type: myai
    config:
      base_url: "https://api.myai.com/v1"
      api_key: "${MYAI_API_KEY}"

The orchestrator in internal/server/orchestrator/outbound.go routes requests to your transformer by matching the type value to your package name. As long as your package exposes the standard constructor functions, it is automatically discovered without explicit registration.

Reusing OpenAI-Compatible Structures

Many providers accept OpenAI-compatible JSON schemas. You can import and reuse structures from llm/transformer/openai/shared.go or llm/transformer/openai/outbound.go to avoid duplicating field definitions. This approach works best when your provider mirrors OpenAI's chat completion format but requires custom authentication headers or endpoint paths.

The transformer.FromOpenAI helper function converts OpenAI-style responses into the generic transformer.Response type, centralizing mapping logic and avoiding code duplication across providers.

Summary

• Custom LLM transformers in AxonHub implement the transformer.Outbound interface defined in llm/transformer/interfaces.go to bridge provider-specific APIs with AxonHub's standard OpenAI-compatible schema.
• You must provide two methods: TransformRequest to convert generic requests to provider HTTP requests, and TransformResponse to parse provider responses back into the canonical format.
• Follow the established pattern by defining a Config struct, providing NewOutboundTransformer and NewOutboundTransformerWithConfig constructors, and placing your code under llm/transformer/<provider>/.
• The orchestrator automatically discovers your transformer based on the type field in channel configuration, requiring no explicit registration beyond making the package available.

Frequently Asked Questions

What interface must a custom LLM transformer implement in AxonHub?

Every custom transformer must implement the transformer.Outbound interface defined in llm/transformer/interfaces.go. This interface requires two methods: TransformRequest, which converts a *transformer.Request into an *http.Request ready for the provider, and TransformResponse, which parses the provider's *http.Response back into a *transformer.Response.

How does AxonHub route requests to my custom transformer?

The orchestrator in internal/server/orchestrator/outbound.go routes requests based on the type field in your channel's YAML configuration. When you set type: myai, the orchestrator looks for a transformer package named myai under llm/transformer/. As long as your package exposes the standard constructor functions, it is automatically discovered without explicit registration.

Can I reuse existing OpenAI payload structures for my custom provider?

Yes, many providers accept OpenAI-compatible JSON schemas. You can import and reuse structures from llm/transformer/openai/shared.go or llm/transformer/openai/outbound.go to avoid duplicating field definitions. This approach works best when your provider mirrors OpenAI's chat completion format but requires custom authentication headers or endpoint paths.

What error types should my transformer return for consistent handling?

Use the exported error variables from llm/transformer/interfaces.go such as transformer.ErrInvalidRequest and transformer.ErrInvalidResponse. Returning these standardized errors ensures that the orchestrator and logging systems handle provider-specific failures consistently, maintaining uniform error responses for downstream clients regardless of which transformer generated the error.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}