# Freebuff LLM Providers: Complete Guide to Supported Services and Integration

> Explore Freebuff LLM providers. Integrate seamlessly with OpenAI, Azure OpenAI, OpenRouter & more via the OpenAI API. Simplify your LLM services.

- Repository: [Codebuff/freebuff](https://github.com/CodebuffAI/freebuff)
- Tags: getting-started
- Published: 2026-09-01

---

**Freebuff supports any LLM service that implements the OpenAI API specification—including OpenAI, Azure OpenAI, OpenRouter, and custom endpoints—through a unified OpenAI-compatible shim.**

The Freebuff framework provides a flexible abstraction layer for integrating large language models via its dedicated **LLM-providers** package. Rather than maintaining separate adapters for each provider, Freebuff ships with a single, specification-compliant shim that enables seamless provider swapping by changing only configuration parameters.

## The OpenAI-Compatible Architecture

Freebuff's provider strategy centers on a universal **OpenAI-compatible** shim rather than vendor-specific SDKs. This architecture is implemented in [`packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts), where the `createOpenAICompatible` function generates a provider instance that conforms to the OpenAI API contract.

The shim exposes a standardized interface for **language models**, **chat models**, **completions**, **embeddings**, and **image generation**. Because it relies on the OpenAPI specification rather than proprietary client libraries, any service exposing a compatible HTTP JSON interface works without code modifications.

## Supported LLM Providers

Freebuff officially supports any provider implementing the OpenAI API contract. The following services are verified to work with the OpenAI-compatible shim:

- **OpenAI** (Official) – The native API including GPT-4, GPT-3.5, and embedding models.
- **Azure OpenAI** – Microsoft's enterprise-hosted OpenAI service requiring deployment-specific endpoints.
- **OpenRouter** – A unified gateway accessing multiple models through a single OpenAI-compatible interface.
- **Custom Endpoints** – Self-hosted inference servers (e.g., vLLM, LM Studio, or Ollama with OpenAI compatibility layers).

## Implementation Details

The core provider logic lives in [`packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts). This file defines the runtime behavior and type signatures ensuring consistent API behavior across all supported services.

The public API surface is exported from [`packages/llm-providers/src/openai-compatible/index.ts`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/src/openai-compatible/index.ts), which re-exports the provider factory and associated model classes. The [`package.json`](https://github.com/CodebuffAI/freebuff/blob/main/package.json) at [`packages/llm-providers/package.json`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/package.json) declares the package as `@freebuff/llm-providers`, tying it into the monorepo structure.

### Provider Factory Signature

The `createOpenAICompatible` function accepts configuration objects containing `apiKey` and `baseURL` parameters. Runtime logic within the provider normalizes requests to the OpenAI schema regardless of which underlying service receives the actual HTTP call.

## Configuring LLM Providers in Freebuff

Instantiate providers by importing `createOpenAICompatible` from the `@freebuff/llm-providers` package and passing the appropriate endpoint credentials.

```typescript
import { createOpenAICompatible } from '@freebuff/llm-providers';

// OpenAI (official)
const openai = createOpenAICompatible({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: 'https://api.openai.com/v1',
});

// Azure OpenAI
const azure = createOpenAICompatible({
  apiKey: process.env.AZURE_OPENAI_KEY,
  baseURL: 'https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME',
});

// OpenRouter (multi-provider gateway)
const openRouter = createOpenAICompatible({
  apiKey: process.env.OPENROUTER_API_KEY,
  baseURL: 'https://openrouter.ai/api/v1',
});

```

### Using Chat Models

All configured providers expose identical methods for text generation. The chat model interface remains consistent regardless of the underlying service:

```typescript
const response = await openai.chatModel('gpt-4o-mini').generate({
  messages: [{ role: 'user', content: 'Explain the OpenAI-compatible shim.' }],
});

console.log(response.text);

```

## Supported Model Capabilities

The OpenAI-compatible shim guarantees uniform behavior across five core capability areas:

1. **Chat Models** – Conversational interfaces supporting system prompts and multi-turn contexts.
2. **Completions** – Legacy text completion endpoints for single-turn generation tasks.
3. **Embeddings** – Vector representation generation for similarity search and clustering.
4. **Image Generation** – DALL-E compatible endpoints for programmatic image synthesis.
5. **Language Models** – Base language model interfaces for non-chat inference tasks.

## Summary

- Freebuff utilizes a single **OpenAI-compatible shim** located in [`packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts) to interface with all LLM services.
- **Any provider** implementing the OpenAI API specification is supported, including OpenAI, Azure OpenAI, OpenRouter, and custom inference endpoints.
- The `createOpenAICompatible` factory function accepts `apiKey` and `baseURL` parameters to instantiate provider-specific clients.
- All providers expose identical interfaces for chat, embeddings, completions, and image generation, enabling zero-code provider migration.

## Frequently Asked Questions

### Can I use local LLM servers with Freebuff?

Yes. Freebuff supports local inference servers such as **vLLM**, **LM Studio**, or **Ollama** (when configured with OpenAI compatibility mode). Point the `baseURL` parameter to your local endpoint (typically `http://localhost:port/v1`) and provide the appropriate API key if required by the local server.

### Does Freebuff support Anthropic Claude or Google Gemini directly?

The current codebase does not include provider-specific shims for Anthropic or Google APIs. However, you can access Claude, Gemini, and other models through **OpenRouter**, which aggregates multiple providers behind a single OpenAI-compatible interface. Direct native SDK support would require additional provider implementations beyond the OpenAI-compatible shim.

### How do I switch between different LLM providers in the same application?

Switching providers requires only changing the configuration object passed to `createOpenAICompatible`. Since all providers implement the same interface methods, you can instantiate multiple provider objects simultaneously or swap the `baseURL` and credentials without modifying your application logic. The shim normalizes all requests to ensure consistent behavior across services.

### What authentication methods are supported by the LLM providers package?

The OpenAI-compatible shim supports **Bearer token authentication** via the `apiKey` configuration parameter. This covers standard API key authentication used by OpenAI, Azure OpenAI (using key-based authentication), and most OpenAI-compatible gateways. Azure AD-based authentication or OAuth flows would require extending the base provider configuration in [`openai-compatible-provider.ts`](https://github.com/CodebuffAI/freebuff/blob/main/openai-compatible-provider.ts).