How to Implement Request Overrides with Templates in Channel Configuration

AxonHub enables dynamic request customization through Go template-based overrides defined in channel configurations, allowing runtime modification of JSON payloads and HTTP headers before they reach LLM providers.

The looplj/axonhub repository implements a flexible request override system that lets you implement request overrides with templates in channel configuration to transform outbound LLM calls. This mechanism uses Go's text/template engine to render dynamic values at request time, supporting everything from simple variable substitution to complex conditional logic.

Understanding Channel Configuration and Override Templates

Channel configurations in AxonHub define how requests are routed and transformed before reaching specific LLM providers. The override system extends this by allowing per-channel template definitions that modify the request body or headers dynamically.

ChannelSettings Structure

The configuration schema resides in objects/channel_settings.go and defines the storage structure for override rules:

• OverrideParameters: A JSON field containing template strings for body modifications
• OverrideHeaders: A structured list of header key-value pairs where values are templates

These settings are parsed into actionable operations when a channel is loaded into the orchestrator state.

OverrideOperation Definition

Each override translates into an OverrideOperation struct that specifies:

• Path: The JSON path or header name to modify
• Value: The Go template string to render
• Operation: The type of modification (set, append, or delete)

Template Rendering Context and Syntax

Templates execute against a RenderContext populated at request time, providing access to runtime variables and request metadata.

RenderContext Fields

The context available in templates includes:

• Prompt: The raw user prompt text
• Model: The resolved model name for the request
• SessionID: The current session identifier
• UserID: The authenticated user identifier
• ApiKey: The provider-specific API key (sourced from profiles)
• Tags: Arbitrary string tags attached to the request
• UserSettings: UI-controlled parameters like temperature
• Metadata: Extra data passed via the API call

Go Template Syntax Examples

Templates support all standard Go template functions:

// Simple variable substitution
"temperature": "{{.UserSettings.temp}}"

// Pipeline operations
"max_tokens": "{{.Prompt | len}}"

// Conditional logic
"model": "{{if eq .Model \"fast\"}}gpt-3.5-turbo{{else}}gpt-4{{end}}"

// Default values
"user": "{{default .UserID \"anonymous\"}}"

Implementing Request Body Overrides

The applyOverrideRequestBody function in internal/server/orchestrator/override.go handles JSON payload modifications. This middleware intercepts the raw request body, parses it as JSON, applies the rendered templates, and re-serializes the payload.

When implementing body overrides:

1. Define the JSON structure in OverrideParameters as a string containing template expressions
2. Ensure template paths match the target JSON keys exactly
3. Use type-appropriate functions (e.g., atoi for integers) if the provider expects specific types

Example configuration:

overrideParameters: |
  {
    "temperature": 0.7,
    "max_tokens": 2048,
    "user": "{{.UserID}}",
    "metadata": {
      "session": "{{.SessionID}}",
      "source": "axonhub"
    }
  }

Implementing Header Overrides

Header modifications follow a similar pattern through applyOverrideRequestHeaders. This middleware updates HTTP headers before the request leaves AxonHub, supporting authentication token rotation, custom tagging, and provider-specific headers.

Key capabilities include:

• Dynamic Authorization: Inject per-user API keys using {{.ApiKey}}
• Header Removal: Set empty values to strip default headers
• Conditional Headers: Add headers only when specific tags are present

Configuration example:

overrideHeaders:
  - key: "Authorization"
    value: "Bearer {{.ApiKey}}"
  - key: "X-Request-ID"
    value: "{{.SessionID}}"
  - key: "X-User-Tags"
    value: "{{range .Tags}}{{.}},{{end}}"

Middleware Pipeline Integration

The orchestrator assembles these overrides into the request pipeline through internal/server/orchestrator/transform_options.go. When applyTransformOptions processes a request, it copies the channel's TransformOptions into the llm.Request structure and populates the orchestrator state with any defined overrides.

The pipeline insertion occurs in internal/server/orchestrator/override.go where two critical middlewares register:

1. override-request-body: Executes during the raw request phase to transform JSON payloads
2. override-request-headers: Executes just before dispatch to modify HTTP headers

Both middlewares access the override operations stored in the orchestrator state (defined in internal/server/orchestrator/state.go), ensuring per-channel isolation and thread-safe template rendering.

Configuration Examples and Edge Cases

Complete channel configuration demonstrating template capabilities:

id: "prod-openai"
type: "openai"
defaultModel: "gpt-4"
overrideParameters: |
  {
    "temperature": "{{.UserSettings.temp}}",
    "max_tokens": "{{.UserSettings.maxTokens}}",
    "user": "{{.UserID}}",
    "metadata": {
      "session_id": "{{.SessionID}}",
      "request_tags": "{{.Tags | join \",\"}}"
    }
  }
overrideHeaders:
  - key: "Authorization"
    value: "Bearer {{.ApiKey}}"
  - key: "X-Custom-Header"
    value: "{{if .Tags}}tagged{{else}}untagged{{end}}"

Edge case handling as implemented in internal/server/orchestrator/override_test.go:

Situation	Behaviour
Missing template variable	Renders as empty string; field retains empty value (use {{default .Var "value"}} for fallbacks)
Invalid JSON after override	Middleware logs warning and returns 400 Bad Request to prevent malformed provider calls
Header override with empty value	Header is removed from the outgoing request, useful for stripping default authentication
Template execution error	Logged as orchestrator warning; original request value preserved to maintain request flow

Summary

• Request overrides in AxonHub use Go templates defined in ChannelSettings to dynamically modify outbound LLM requests.
• OverrideParameters targets JSON body fields while OverrideHeaders manipulates HTTP headers before dispatch.
• The RenderContext provides runtime variables including Prompt, UserID, SessionID, ApiKey, and UserSettings for template substitution.
• Middleware in internal/server/orchestrator/override.go applies these transformations through applyOverrideRequestBody and applyOverrideRequestHeaders.
• Configuration uses standard YAML with JSON-encoded template strings, supporting conditional logic, pipelines, and default values.

Frequently Asked Questions

How do I reference the user's API key in an override template?

Use the {{.ApiKey}} variable in your template string. This value is sourced from the user's profile or channel configuration and injected into the RenderContext at request time. For example: "Authorization": "Bearer {{.ApiKey}}" dynamically sets the authentication header per-request.

What happens if a template variable is missing during rendering?

If a template references a non-existent field like {{.MissingVar}}, Go's template engine renders an empty string for that value. The override will still apply, but with an empty value. To provide fallbacks, use the default function: {{default .MissingVar "fallback_value"}}.

Can I use conditionals in override templates?

Yes, AxonHub supports all standard Go template syntax including conditionals, loops, and pipelines. You can conditionally set values using {{if}} blocks, iterate over slices with {{range}}, or chain functions like {{.Tags | join ","}} to format arrays as comma-separated strings.

Where are the override templates stored and processed?

Override templates are stored in the OverrideParameters and OverrideHeaders fields of the ChannelSettings struct defined in objects/channel_settings.go. At runtime, the orchestrator loads these into the pipeline state (internal/server/orchestrator/state.go) and processes them through middleware functions in internal/server/orchestrator/override.go before dispatching to the LLM provider.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}