Freebuff LLM Providers: Complete Guide to Supported Services and Integration

Freebuff supports any LLM service that implements the OpenAI API specification—including OpenAI, Azure OpenAI, OpenRouter, and custom endpoints—through a unified OpenAI-compatible shim.

The Freebuff framework provides a flexible abstraction layer for integrating large language models via its dedicated LLM-providers package. Rather than maintaining separate adapters for each provider, Freebuff ships with a single, specification-compliant shim that enables seamless provider swapping by changing only configuration parameters.

The OpenAI-Compatible Architecture

Freebuff's provider strategy centers on a universal OpenAI-compatible shim rather than vendor-specific SDKs. This architecture is implemented in packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts, where the createOpenAICompatible function generates a provider instance that conforms to the OpenAI API contract.

The shim exposes a standardized interface for language models, chat models, completions, embeddings, and image generation. Because it relies on the OpenAPI specification rather than proprietary client libraries, any service exposing a compatible HTTP JSON interface works without code modifications.

Supported LLM Providers

Freebuff officially supports any provider implementing the OpenAI API contract. The following services are verified to work with the OpenAI-compatible shim:

• OpenAI (Official) – The native API including GPT-4, GPT-3.5, and embedding models.
• Azure OpenAI – Microsoft's enterprise-hosted OpenAI service requiring deployment-specific endpoints.
• OpenRouter – A unified gateway accessing multiple models through a single OpenAI-compatible interface.
• Custom Endpoints – Self-hosted inference servers (e.g., vLLM, LM Studio, or Ollama with OpenAI compatibility layers).

Implementation Details

The core provider logic lives in packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts. This file defines the runtime behavior and type signatures ensuring consistent API behavior across all supported services.

The public API surface is exported from packages/llm-providers/src/openai-compatible/index.ts, which re-exports the provider factory and associated model classes. The package.json at packages/llm-providers/package.json declares the package as @freebuff/llm-providers, tying it into the monorepo structure.

Provider Factory Signature

The createOpenAICompatible function accepts configuration objects containing apiKey and baseURL parameters. Runtime logic within the provider normalizes requests to the OpenAI schema regardless of which underlying service receives the actual HTTP call.

Configuring LLM Providers in Freebuff

Instantiate providers by importing createOpenAICompatible from the @freebuff/llm-providers package and passing the appropriate endpoint credentials.

import { createOpenAICompatible } from '@freebuff/llm-providers';

// OpenAI (official)
const openai = createOpenAICompatible({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: 'https://api.openai.com/v1',
});

// Azure OpenAI
const azure = createOpenAICompatible({
  apiKey: process.env.AZURE_OPENAI_KEY,
  baseURL: 'https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME',
});

// OpenRouter (multi-provider gateway)
const openRouter = createOpenAICompatible({
  apiKey: process.env.OPENROUTER_API_KEY,
  baseURL: 'https://openrouter.ai/api/v1',
});

Using Chat Models

All configured providers expose identical methods for text generation. The chat model interface remains consistent regardless of the underlying service:

const response = await openai.chatModel('gpt-4o-mini').generate({
  messages: [{ role: 'user', content: 'Explain the OpenAI-compatible shim.' }],
});

console.log(response.text);

Supported Model Capabilities

The OpenAI-compatible shim guarantees uniform behavior across five core capability areas:

1. Chat Models – Conversational interfaces supporting system prompts and multi-turn contexts.
2. Completions – Legacy text completion endpoints for single-turn generation tasks.
3. Embeddings – Vector representation generation for similarity search and clustering.
4. Image Generation – DALL-E compatible endpoints for programmatic image synthesis.
5. Language Models – Base language model interfaces for non-chat inference tasks.

Summary

• Freebuff utilizes a single OpenAI-compatible shim located in packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts to interface with all LLM services.
• Any provider implementing the OpenAI API specification is supported, including OpenAI, Azure OpenAI, OpenRouter, and custom inference endpoints.
• The createOpenAICompatible factory function accepts apiKey and baseURL parameters to instantiate provider-specific clients.
• All providers expose identical interfaces for chat, embeddings, completions, and image generation, enabling zero-code provider migration.

Frequently Asked Questions

Can I use local LLM servers with Freebuff?

Yes. Freebuff supports local inference servers such as vLLM, LM Studio, or Ollama (when configured with OpenAI compatibility mode). Point the baseURL parameter to your local endpoint (typically http://localhost:port/v1) and provide the appropriate API key if required by the local server.

Does Freebuff support Anthropic Claude or Google Gemini directly?

The current codebase does not include provider-specific shims for Anthropic or Google APIs. However, you can access Claude, Gemini, and other models through OpenRouter, which aggregates multiple providers behind a single OpenAI-compatible interface. Direct native SDK support would require additional provider implementations beyond the OpenAI-compatible shim.

How do I switch between different LLM providers in the same application?

Switching providers requires only changing the configuration object passed to createOpenAICompatible. Since all providers implement the same interface methods, you can instantiate multiple provider objects simultaneously or swap the baseURL and credentials without modifying your application logic. The shim normalizes all requests to ensure consistent behavior across services.

What authentication methods are supported by the LLM providers package?

The OpenAI-compatible shim supports Bearer token authentication via the apiKey configuration parameter. This covers standard API key authentication used by OpenAI, Azure OpenAI (using key-based authentication), and most OpenAI-compatible gateways. Azure AD-based authentication or OAuth flows would require extending the base provider configuration in openai-compatible-provider.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}