# How to Integrate a New LLM Provider into Freebuff: A Step-by-Step Guide

> Learn to integrate a new LLM provider into Freebuff easily. Follow our step-by-step guide extending OpenAICompatibleProvider, registering your provider, and configuring SDK options for seamless integration.

- Repository: [Codebuff/freebuff](https://github.com/CodebuffAI/freebuff)
- Tags: how-to-guide
- Published: 2026-08-20

---

**Integrating a new LLM provider into Freebuff requires extending the `OpenAICompatibleProvider` base class, registering your provider in the `PROVIDERS` map, and exposing configuration options in the SDK validation schema.**

Freebuff's plugin-based architecture treats every language model service as a modular component that implements a standardized **OpenAI-compatible interface**. This design allows developers to add support for any LLM with an HTTP API without modifying core engine code. This guide walks through the three essential implementation steps based on the `CodebuffAI/freebuff` source code.

---

## Create a Provider Class Extending OpenAICompatibleProvider

Every LLM provider in Freebuff inherits from the base class defined in [`packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts). Your implementation must override the core methods for completions, chat, embeddings, and image generation.

### Required Method Implementations

- **`complete(prompt, opts)`**: Handle text completions and return a `CompletionResponse` with translated fields.
- **`chat(messages, opts)`**: Process chat-style conversations and return a `ChatResponse`.
- **Translation methods**: Map provider-specific response structures to OpenAI-compatible shapes including `choices`, `usage`, and `created` timestamps.

### Example Provider Implementation

```typescript
import { OpenAICompatibleProvider } from '../openai-compatible-provider';
import type { CompletionResponse, ChatResponse } from '../openai-compatible';

export class MyLLMProvider extends OpenAICompatibleProvider {
  async complete(prompt: string, opts: any): Promise<CompletionResponse> {
    const res = await fetch(`${this.baseUrl}/completions`, {
      method: 'POST',
      headers: { Authorization: `Bearer ${this.apiKey}` },
      body: JSON.stringify({ prompt, ...opts })
    });
    const data = await res.json();
    return this.translateCompletion(data);
  }

  async chat(messages: any[], opts: any): Promise<ChatResponse> {
    const res = await fetch(`${this.baseUrl}/chat`, {
      method: 'POST',
      headers: { Authorization: `Bearer ${this.apiKey}` },
      body: JSON.stringify({ messages, ...opts })
    });
    const data = await res.json();
    return this.translateChat(data);
  }
}

```

Export your class from an [`index.ts`](https://github.com/CodebuffAI/freebuff/blob/main/index.ts) file at your provider's package root so Freebuff's module resolver can locate it by name.

---

## Register the Provider in the Runtime Registry

Freebuff discovers available providers through a centralized registry located at [`packages/llm-providers/src/openai-compatible/internal/index.ts`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/src/openai-compatible/internal/index.ts). This file exports a `PROVIDERS` object that maps string keys to provider class constructors.

### Adding Your Provider Entry

Open [`internal/index.ts`](https://github.com/CodebuffAI/freebuff/blob/main/internal/index.ts) and append your provider to the existing map:

```typescript
import { MyLLMProvider } from './my-llm-provider';

export const PROVIDERS = {
  openai: OpenAICompatibleProvider,
  anthropic: AnthropicProvider,
  'my-llm': MyLLMProvider,   // ← new provider registration
};

```

The string key (`'my-llm'` in this example) becomes the identifier users specify in their Freebuff configuration. Choose a descriptive, lowercase key with hyphens for consistency with built-in providers.

---

## Expose Configuration Options for Users

Users need a way to configure API credentials, endpoint URLs, and provider-specific parameters. Freebuff validates these settings through schema definitions in [`sdk/src/validate-agents.ts`](https://github.com/CodebuffAI/freebuff/blob/main/sdk/src/validate-agents.ts).

### Configuration Schema Updates

Define a TypeScript interface describing your provider's required and optional fields. Then extend the validation logic to recognize your provider key and enforce the schema. Users can then configure your provider in their `freebuff` configuration file:

```json
{
  "llmProvider": "my-llm",
  "my-llm": {
    "apiKey": "YOUR_API_KEY",
    "baseUrl": "https://api.my-llm.com/v1"
  }
}

```

The runtime uses the `llmProvider` key to look up the corresponding class in the `PROVIDERS` registry, instantiates it with the provided options, and routes all generation calls through your implementation.

---

## Key Integration Files

| Path | Purpose |
|------|---------|
| [`packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts) | Base class defining the OpenAI-compatible contract |
| [`packages/llm-providers/src/openai-compatible/internal/index.ts`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/src/openai-compatible/internal/index.ts) | Provider registry with the `PROVIDERS` map |
| [`sdk/src/validate-agents.ts`](https://github.com/CodebuffAI/freebuff/blob/main/sdk/src/validate-agents.ts) | Configuration schema and validation logic |

---

## Summary

- **Extend `OpenAICompatibleProvider`** in `packages/llm-providers/src/openai-compatible/` to implement provider-specific API calls and response translation.
- **Register in [`internal/index.ts`](https://github.com/CodebuffAI/freebuff/blob/main/internal/index.ts)** by adding a key-value pair to the `PROVIDERS` exported object.
- **Add configuration schema** in [`sdk/src/validate-agents.ts`](https://github.com/CodebuffAI/freebuff/blob/main/sdk/src/validate-agents.ts) so users can specify credentials and endpoint settings.
- **Use the OpenAI-compatible response shape** (`choices`, `usage`, `created`) to ensure seamless integration with Freebuff's agent system.

---

## Frequently Asked Questions

### What interface must my LLM provider implement?

Your provider must extend `OpenAICompatibleProvider` from [`openai-compatible-provider.ts`](https://github.com/CodebuffAI/freebuff/blob/main/openai-compatible-provider.ts) and implement `complete()`, `chat()`, and other generation methods. These methods handle HTTP requests to your LLM's API and translate responses into OpenAI-compatible structures.

### Where do I register a new provider so Freebuff can find it?

Add your provider class to the `PROVIDERS` map in [`packages/llm-providers/src/openai-compatible/internal/index.ts`](https://github.com/CodebuffAI/freebuff/blob/main/packages/llm-providers/src/openai-compatible/internal/index.ts). The string key you choose becomes the `llmProvider` value users set in their configuration.

### How do users configure my custom LLM provider?

Users specify your provider key in their Freebuff config along with provider-specific options like `apiKey` and `baseUrl`. You must extend the validation schema in [`sdk/src/validate-agents.ts`](https://github.com/CodebuffAI/freebuff/blob/main/sdk/src/validate-agents.ts) to recognize and validate these settings.

### Can I integrate a provider that doesn't follow OpenAI's API format?

Yes, provided you handle translation in your provider class. The `translateCompletion()` and `translateChat()` methods map any response structure to the standardized OpenAI-compatible format expected by Freebuff's agents.