How to Integrate a New LLM Provider into Freebuff: A Step-by-Step Guide

Integrating a new LLM provider into Freebuff requires extending the OpenAICompatibleProvider base class, registering your provider in the PROVIDERS map, and exposing configuration options in the SDK validation schema.

Freebuff's plugin-based architecture treats every language model service as a modular component that implements a standardized OpenAI-compatible interface. This design allows developers to add support for any LLM with an HTTP API without modifying core engine code. This guide walks through the three essential implementation steps based on the CodebuffAI/freebuff source code.

---

Create a Provider Class Extending OpenAICompatibleProvider

Every LLM provider in Freebuff inherits from the base class defined in packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts. Your implementation must override the core methods for completions, chat, embeddings, and image generation.

Required Method Implementations

• complete(prompt, opts): Handle text completions and return a CompletionResponse with translated fields.
• chat(messages, opts): Process chat-style conversations and return a ChatResponse.
• Translation methods: Map provider-specific response structures to OpenAI-compatible shapes including choices, usage, and created timestamps.

Example Provider Implementation

import { OpenAICompatibleProvider } from '../openai-compatible-provider';
import type { CompletionResponse, ChatResponse } from '../openai-compatible';

export class MyLLMProvider extends OpenAICompatibleProvider {
  async complete(prompt: string, opts: any): Promise<CompletionResponse> {
    const res = await fetch(`${this.baseUrl}/completions`, {
      method: 'POST',
      headers: { Authorization: `Bearer ${this.apiKey}` },
      body: JSON.stringify({ prompt, ...opts })
    });
    const data = await res.json();
    return this.translateCompletion(data);
  }

  async chat(messages: any[], opts: any): Promise<ChatResponse> {
    const res = await fetch(`${this.baseUrl}/chat`, {
      method: 'POST',
      headers: { Authorization: `Bearer ${this.apiKey}` },
      body: JSON.stringify({ messages, ...opts })
    });
    const data = await res.json();
    return this.translateChat(data);
  }
}

Export your class from an index.ts file at your provider's package root so Freebuff's module resolver can locate it by name.

---

Register the Provider in the Runtime Registry

Freebuff discovers available providers through a centralized registry located at packages/llm-providers/src/openai-compatible/internal/index.ts. This file exports a PROVIDERS object that maps string keys to provider class constructors.

Adding Your Provider Entry

Open internal/index.ts and append your provider to the existing map:

import { MyLLMProvider } from './my-llm-provider';

export const PROVIDERS = {
  openai: OpenAICompatibleProvider,
  anthropic: AnthropicProvider,
  'my-llm': MyLLMProvider,   // ← new provider registration
};

The string key ('my-llm' in this example) becomes the identifier users specify in their Freebuff configuration. Choose a descriptive, lowercase key with hyphens for consistency with built-in providers.

---

Expose Configuration Options for Users

Users need a way to configure API credentials, endpoint URLs, and provider-specific parameters. Freebuff validates these settings through schema definitions in sdk/src/validate-agents.ts.

Configuration Schema Updates

Define a TypeScript interface describing your provider's required and optional fields. Then extend the validation logic to recognize your provider key and enforce the schema. Users can then configure your provider in their freebuff configuration file:

{
  "llmProvider": "my-llm",
  "my-llm": {
    "apiKey": "YOUR_API_KEY",
    "baseUrl": "https://api.my-llm.com/v1"
  }
}

The runtime uses the llmProvider key to look up the corresponding class in the PROVIDERS registry, instantiates it with the provided options, and routes all generation calls through your implementation.

---

Key Integration Files

Path	Purpose
packages/llm-providers/src/openai-compatible/openai-compatible-provider.ts	Base class defining the OpenAI-compatible contract
packages/llm-providers/src/openai-compatible/internal/index.ts	Provider registry with the PROVIDERS map
sdk/src/validate-agents.ts	Configuration schema and validation logic

---

Summary

• Extend OpenAICompatibleProvider in packages/llm-providers/src/openai-compatible/ to implement provider-specific API calls and response translation.
• Register in internal/index.ts by adding a key-value pair to the PROVIDERS exported object.
• Add configuration schema in sdk/src/validate-agents.ts so users can specify credentials and endpoint settings.
• Use the OpenAI-compatible response shape (choices, usage, created) to ensure seamless integration with Freebuff's agent system.

---

Frequently Asked Questions

What interface must my LLM provider implement?

Your provider must extend OpenAICompatibleProvider from openai-compatible-provider.ts and implement complete(), chat(), and other generation methods. These methods handle HTTP requests to your LLM's API and translate responses into OpenAI-compatible structures.

Where do I register a new provider so Freebuff can find it?

Add your provider class to the PROVIDERS map in packages/llm-providers/src/openai-compatible/internal/index.ts. The string key you choose becomes the llmProvider value users set in their configuration.

How do users configure my custom LLM provider?

Users specify your provider key in their Freebuff config along with provider-specific options like apiKey and baseUrl. You must extend the validation schema in sdk/src/validate-agents.ts to recognize and validate these settings.

Can I integrate a provider that doesn't follow OpenAI's API format?

Yes, provided you handle translation in your provider class. The translateCompletion() and translateChat() methods map any response structure to the standardized OpenAI-compatible format expected by Freebuff's agents.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}