Which AI Model Providers Does the Caveman Proxy Support? A Complete Technical Guide

The Caveman proxy supports major commercial AI providers including OpenAI, Azure OpenAI, Anthropic, Amazon Bedrock, Google Vertex AI, and Google Gemini, along with an OpenAI-compatible abstraction layer that unifies access to all backends through a single interface.

JuliusBrussee/caveman is an open-source Go-based proxy layer that standardizes access to diverse large language model APIs. The project implements provider-specific adapters under proxy/providers/ to handle authentication, request formatting, and response parsing for each service. Understanding which AI model providers the Caveman proxy supports helps developers route traffic to the optimal backend without modifying client code.

Supported AI Model Providers

The proxy layer organizes each provider into its own sub-directory under proxy/providers/. Each adapter manages provider-specific details like token counting, streaming, content compression, and authentication.

OpenAI

The OpenAI adapter connects to the official OpenAI API, supporting chat completions, streaming responses, token counting, and response caching.

• Implementation: proxy/providers/openai/openai.go
• Capabilities: Direct integration with GPT-4o, GPT-4o-mini, and legacy GPT models
• Features: Handles rate limiting, content compression, and automatic retries

Azure OpenAI

The Azure OpenAI adapter routes requests to Azure-hosted OpenAI endpoints using Azure-specific authentication and endpoint formatting.

• Implementation: proxy/providers/azureopenai/azureopenai.go
• Capabilities: Supports regional Azure endpoints and managed identity authentication
• Routing: Maps model names to Azure deployment names internally

Anthropic

The Anthropic adapter connects to Claude models via the official Anthropic API, handling the distinct message format and token accounting required by Claude.

• Implementation: proxy/providers/anthropic/anthropic.go
• Capabilities: Full support for Claude 3.5 Sonnet, Claude 3 Opus, and Claude 3 Haiku
• Features: Manages distinct request/response schemas and streaming implementations

Amazon Bedrock

The Amazon Bedrock adapter provides access to AWS Bedrock models including Claude, Titan, and Cohere, with automatic AWS request signing.

• Implementation: proxy/providers/bedrock/bedrock.go
• Capabilities: Supports amazon.titan-text-lite-v1, anthropic.claude-*, and other Bedrock model IDs
• Features: Handles AWS Signature Version 4 request signing and regional routing

Google Vertex AI

The Google Vertex AI adapter integrates with Google Cloud's Vertex AI platform, managing model naming conventions and token usage tracking.

• Implementation: proxy/providers/vertex/vertex.go
• Capabilities: Supports Gemini and PaLM models hosted on Vertex AI
• Routing: Uses vertexai/ prefix in model strings to distinguish from direct Gemini API

Google Gemini

The Google Gemini adapter provides direct API access to Gemini models through Google's generative AI endpoint.

• Implementation: proxy/providers/gemini/gemini.go
• Capabilities: Direct Gemini 1.5 Pro and Gemini 1.5 Flash support
• Features: Implements content compression and Gemini-specific token counting

OpenAI Compatibility Layer

Beyond individual provider adapters, Caveman implements an OpenAI-compatible abstraction that exposes a unified REST interface. This layer accepts standard OpenAI-formatted requests and routes them to any supported backend based on the model identifier.

• Implementation: proxy/providers/openaicompat/openaicompat.go
• Purpose: Allows existing OpenAI client SDKs to interact with Anthropic, Bedrock, or Vertex AI without code changes
• Routing Logic: Inspects the model field in request payloads to determine the appropriate provider adapter

Generic and Utility Adapters

The proxy includes additional adapters for specialized use cases:

• JSON Splice (proxy/providers/jsonsplice/jsonsplice.go): Generic JSON-based adapter for custom or experimental providers
• AgentPath (proxy/providers/agentpath.go): Routes requests to external agent services implementing the required contract
• Toolzone (proxy/providers/toolzone.go): Internal utility for routing to auxiliary services within the proxy stack
• Base Adapter (proxy/providers/adapter.go): Core abstraction that all provider modules extend, enabling minimal boilerplate for new backends

How Model Routing Works

Caveman determines the target provider by inspecting the model string in incoming requests. The provider catalog (shared/provider-catalog/catalog.go) maintains a central registry mapping model identifiers to specific provider implementations.

For example:

• gpt-4o-mini routes to the OpenAI adapter
• claude-3-5-sonnet-20241022 routes to the Anthropic adapter
• amazon.titan-text-lite-v1 routes to the Bedrock adapter
• vertexai/gemini-1.5-pro routes to the Vertex AI adapter

Code Examples

The following examples demonstrate how to invoke different providers through the Caveman proxy. All examples assume the proxy runs on localhost:8080 and that API keys are configured via environment variables.

OpenAI Request

import (
    "bytes"
    "encoding/json"
    "net/http"
)

func callOpenAI() {
    payload := map[string]any{
        "model": "gpt-4o-mini",
        "messages": []map[string]string{
            {"role": "user", "content": "Explain quantum tunneling in simple terms."},
        },
    }
    body, _ := json.Marshal(payload)

    req, _ := http.NewRequest("POST", "http://localhost:8080/v1/chat/completions", bytes.NewReader(body))
    req.Header.Set("Content-Type", "application/json")
    // The proxy reads the OPENAI_API_KEY env var automatically
    resp, _ := http.DefaultClient.Do(req)
    defer resp.Body.Close()
    // Process resp...
}

Anthropic Request

payload := map[string]any{
    "model": "claude-3-5-sonnet-20241022",
    "messages": []map[string]string{
        {"role": "user", "content": "Write a haiku about sunrise."},
    },
}
// POST to http://localhost:8080/v1/chat/completions
// Proxy routes to Anthropic based on model prefix

Amazon Bedrock Request

payload := map[string]any{
    "model": "amazon.titan-text-lite-v1",
    "messages": []map[string]string{
        {"role": "user", "content": "Summarize the plot of 'Pride and Prejudice'."},
    },
}
// POST to proxy endpoint; Bedrock adapter handles AWS request signing automatically

Google Vertex AI Request

payload := map[string]any{
    "model": "vertexai/gemini-1.5-pro",
    "messages": []map[string]string{
        {"role": "user", "content": "Translate this to Japanese: 'Good morning'"},
    },
}
// Proxy routes to Vertex AI based on the vertexai/ prefix

Implementation Architecture

The extensible provider ecosystem relies on two critical components:

• Provider Catalog (shared/provider-catalog/catalog.go): Central registry that maps model identifiers to provider implementations
• Base Adapter (proxy/providers/adapter.go): Core abstraction layer that enforces a consistent interface across all providers, enabling new backends to integrate with minimal boilerplate

Each provider adapter implements the base interface while handling provider-specific concerns such as authentication headers, request payload transformation, and response parsing.

Summary

• Caveman supports six major commercial providers: OpenAI, Azure OpenAI, Anthropic, Amazon Bedrock, Google Vertex AI, and Google Gemini
• Each provider has a dedicated adapter under proxy/providers/ handling authentication, formatting, and streaming
• The OpenAI-compatible layer (proxy/providers/openaicompat/openaicompat.go) allows existing OpenAI SDKs to interact with any supported backend
• Routing occurs automatically based on the model string, managed by the provider catalog in shared/provider-catalog/catalog.go
• Generic adapters like JSON Splice and AgentPath enable custom provider integration without core code changes

Frequently Asked Questions

How does Caveman handle authentication for different AI providers?

Caveman reads provider API keys from environment variables (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, AWS_ACCESS_KEY_ID). Each adapter in proxy/providers/ retrieves the appropriate credentials during request construction, ensuring keys never transit through client requests.

Can I use the same client code to call both OpenAI and Anthropic models?

Yes. By targeting the OpenAI-compatible endpoint at /v1/chat/completions and changing only the model string in your request payload, the same HTTP client code can invoke OpenAI, Anthropic, Bedrock, or Vertex AI models. The proxy routes requests based on the model identifier.

What is the difference between the Vertex AI and Gemini adapters?

The Vertex AI adapter (proxy/providers/vertex/vertex.go) connects to models hosted on Google Cloud's Vertex AI platform, while the Gemini adapter (proxy/providers/gemini/gemini.go) connects directly to the Google Gemini API. Use vertexai/ prefixes in model names to route to Vertex AI, or use the Gemini adapter for direct API access without Google Cloud project requirements.

How do I add support for a new AI provider not currently listed?

Implement the base adapter interface defined in proxy/providers/adapter.go and register your provider in shared/provider-catalog/catalog.go. You can also use the JSON Splice adapter (proxy/providers/jsonsplice/jsonsplice.go) for quick prototyping with custom JSON-based endpoints without writing Go code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}