# How to Add a New LLM Provider to pi-ai: A Complete Implementation Guide

> Learn to add a new LLM provider to pi-ai with this complete implementation guide. Extend types, implement streaming, register the API, and add model discovery.

- Repository: [Mario Zechner/pi-mono](https://github.com/badlogic/pi-mono)
- Tags: how-to-guide
- Published: 2026-02-16

---

**Adding a new LLM provider to pi-ai requires extending the type definitions in [`packages/ai/src/types.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/types.ts), implementing a streaming function in `src/providers/<provider>.ts`, registering the API in [`register-builtins.ts`](https://github.com/badlogic/pi-mono/blob/main/register-builtins.ts), and adding model discovery logic to [`scripts/generate-models.ts`](https://github.com/badlogic/pi-mono/blob/main/scripts/generate-models.ts).**

The pi-ai package in the `badlogic/pi-mono` repository provides a unified, type-safe interface for streaming LLM responses across multiple providers. Whether you need to integrate a proprietary API or a self-hosted model, adding a new LLM provider to pi-ai follows a consistent architectural pattern that maps providers to APIs and models through a thin abstraction layer.

## Understanding the pi-ai Provider Architecture

Pi-ai is built around a thin, type-safe layer that maps **providers** → **APIs** → **models**. The architecture consists of four main touchpoints:

- **Core type definitions** ([`packages/ai/src/types.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/types.ts)) – Union types for APIs, providers, and streaming options
- **Provider implementation** (`src/providers/<provider>.ts`) – The streaming logic that translates pi-ai's generic interface to provider-specific API calls
- **API registry** ([`src/providers/register-builtins.ts`](https://github.com/badlogic/pi-mono/blob/main/src/providers/register-builtins.ts)) – Global registration of the provider and its environment variable detection
- **Model discovery** ([`scripts/generate-models.ts`](https://github.com/badlogic/pi-mono/blob/main/scripts/generate-models.ts)) – Automated fetching of the provider's model catalog

## Step-by-Step Implementation Guide

### Extend Core Type Definitions

Begin by extending the union types in [`packages/ai/src/types.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/types.ts) to include your new provider and its API identifier:

```typescript
// packages/ai/src/types.ts
export type KnownApi = 
  | "openai-api"
  | "anthropic-api"
  | "amazon-bedrock-api"
  | "myprovider-api";  // Add your API here

export type KnownProvider = 
  | "openai"
  | "anthropic"
  | "amazon-bedrock"
  | "myprovider";  // Add your provider here

export interface MyProviderOptions extends StreamOptions {
  region?: string;          // Provider-specific flags
  toolChoice?: "auto" | "none";
}

```

### Implement the Provider Logic

Create a new file at [`packages/ai/src/providers/myprovider.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/providers/myprovider.ts) that implements the `stream<Provider>()` function. Use the existing [`amazon-bedrock.ts`](https://github.com/badlogic/pi-mono/blob/main/amazon-bedrock.ts) implementation as a reference for structure:

```typescript
// packages/ai/src/providers/myprovider.ts
import {
  AssistantMessageEventStream,
} from "../utils/event-stream.js";
import {
  Model,
  Context,
  StreamFunction,
  AssistantMessage,
  TextContent,
  ToolCall,
} from "../types.js";

export interface MyProviderOptions {
  region?: string;
  toolChoice?: "auto" | "none";
}

export const streamMyProvider: StreamFunction<"myprovider-api", MyProviderOptions> = (
  model: Model<"myprovider-api">,
  context: Context,
  opts: MyProviderOptions = {}
) => {
  const stream = new AssistantMessageEventStream();

  (async () => {
    const output: AssistantMessage = {
      role: "assistant",
      content: [],
      api: "myprovider-api",
      provider: model.provider,
      model: model.id,
      usage: { 
        input: 0, 
        output: 0, 
        cacheRead: 0, 
        cacheWrite: 0, 
        totalTokens: 0,
        cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 } 
      },
      stopReason: "stop",
      timestamp: Date.now(),
    };

    // Build request payload using helper functions
    const payload = {
      model: model.id,
      messages: context.messages,
      temperature: opts.temperature,
      max_tokens: opts.maxTokens,
      // Provider-specific parameters
      region: opts.region,
      tool_choice: opts.toolChoice,
    };
    opts.onPayload?.(payload);

    // Issue HTTP request
    const resp = await fetch("https://api.myprovider.com/v1/chat/completions", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${opts.apiKey ?? process.env.MYPROVIDER_API_KEY}`,
        ...opts.headers,
      },
      body: JSON.stringify(payload),
      signal: opts.signal,
    });

    // Parse streaming response
    const reader = resp.body!.getReader();
    const decoder = new TextDecoder();
    let buffer = "";
    
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      buffer += decoder.decode(value, { stream: true });
      const lines = buffer.split("\n");
      buffer = lines.pop() ?? "";
      
      for (const line of lines) {
        if (!line) continue;
        const event = JSON.parse(line);
        
        // Translate provider events to pi-ai streaming events
        if (event.type === "text") {
          const block: TextContent = { type: "text", text: event.content };
          output.content.push(block);
          stream.push({ 
            type: "text_delta", 
            delta: event.content, 
            contentIndex: output.content.length - 1 
          });
        } else if (event.type === "tool_call") {
          const call: ToolCall = {
            type: "toolCall",
            id: event.id,
            name: event.name,
            arguments: event.arguments,
          };
          output.content.push(call);
          stream.push({ type: "toolcall_end", toolCall: call });
        }
      }
    }

    stream.push({ type: "done", reason: output.stopReason, message: output });
    stream.end();
  })().catch(err => {
    stream.push({ 
      type: "error", 
      reason: "error", 
      error: { ...output, errorMessage: err.message } 
    });
    stream.end();
  });

  return stream;
};

```

### Register the API Provider

Register your implementation in [`packages/ai/src/providers/register-builtins.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/providers/register-builtins.ts) to make it available globally:

```typescript
// packages/ai/src/providers/register-builtins.ts
import { streamMyProvider } from "./myprovider.js";

registerApiProvider({
  api: "myprovider-api",
  stream: streamMyProvider,
  // Optional: provide a simplified streaming interface
  streamSimple: streamSimpleMyProvider,
});

```

### Add Environment Variable Detection

If your provider requires an API key, add detection logic in [`packages/ai/src/env-api-keys.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/env-api-keys.ts):

```typescript
// packages/ai/src/env-api-keys.ts
export function getApiKey(provider: KnownProvider): string | undefined {
  // ... existing providers ...
  
  if (provider === "myprovider") {
    return process.env.MYPROVIDER_API_KEY;
  }
  
  return undefined;
}

```

### Generate Model Metadata

Add a discovery function in [`packages/ai/scripts/generate-models.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/scripts/generate-models.ts) to fetch your provider's model catalog:

```typescript
// packages/ai/scripts/generate-models.ts
async function fetchMyProviderModels(): Promise<Model<any>[]> {
  const res = await fetch("https://api.myprovider.com/v1/models");
  const data = await res.json();
  
  return data.models
    .filter(m => m.tool_call)  // Keep only tool-capable models
    .map(m => ({
      id: m.id,
      name: m.name,
      api: "myprovider-api",
      provider: "myprovider",
      baseUrl: "https://api.myprovider.com/v1",
      reasoning: m.reasoning ?? false,
      input: m.modalities?.includes("image") ? ["text", "image"] : ["text"],
      cost: {
        input: m.pricing?.input ?? 0,
        output: m.pricing?.output ?? 0,
        cacheRead: m.pricing?.cacheRead ?? 0,
        cacheWrite: m.pricing?.cacheWrite ?? 0,
      },
      contextWindow: m.limit?.context ?? 4096,
      maxTokens: m.limit?.output ?? 4096,
    }));
}

```

## Optional Integration Points

### Coding Agent Defaults

If you want the coding agent to support your provider out of the box, add a default model ID in [`packages/coding-agent/src/core/model-resolver.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/model-resolver.ts).

### Documentation and Changelog

Update the **Supported Providers** table in [`packages/ai/README.md`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/README.md) with authentication requirements and supported features. Add an entry under `## [Unreleased]` in [`packages/ai/CHANGELOG.md`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/CHANGELOG.md) describing the new provider integration.

## Testing Your New Provider

Before submitting your implementation, ensure comprehensive test coverage. Copy an existing provider's test suite (such as [`packages/ai/test/stream.test.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/test/stream.test.ts)) and modify it to test your provider:

- **Basic streaming**: Verify text generation and event streaming
- **Tool usage**: Test function calling and tool result handling
- **Token accounting**: Validate input/output token counting and cost calculation
- **Abort handling**: Ensure `AbortSignal` properly cancels requests
- **Edge cases**: Empty payloads, context overflow, and image handling (if applicable)

Run the test suite with `npm run check` and targeted `vitest` runs to verify your implementation doesn't break existing providers.

## Summary

- **Type definitions**: Extend `KnownApi` and `KnownProvider` unions in [`packages/ai/src/types.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/types.ts) to include your provider
- **Implementation**: Create `packages/ai/src/providers/<provider>.ts` implementing the `StreamFunction` type with proper event streaming
- **Registration**: Call `registerApiProvider()` in [`packages/ai/src/providers/register-builtins.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/providers/register-builtins.ts) to expose your provider globally
- **Authentication**: Add environment variable detection in [`packages/ai/src/env-api-keys.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/env-api-keys.ts) if your provider requires API keys
- **Model discovery**: Implement a fetch function in [`packages/ai/scripts/generate-models.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/scripts/generate-models.ts) to pull available models and pricing data
- **Testing**: Copy existing provider tests and verify streaming, tool calls, token counting, and abort handling before submitting

## Frequently Asked Questions

### Do I need to modify the coding agent to add a new LLM provider to pi-ai?

No, modifying the coding agent is optional. The core provider integration only requires changes to the `packages/ai` module. However, if you want the coding agent to use your provider by default, you can add a sensible default model ID in [`packages/coding-agent/src/core/model-resolver.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/model-resolver.ts) so that the agent automatically selects your provider when no specific model is requested.

### How does pi-ai handle authentication for custom providers?

Pi-ai uses a centralized environment variable detection system located in [`packages/ai/src/env-api-keys.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/env-api-keys.ts). When adding a new provider, you implement a check for your specific provider identifier that returns the appropriate environment variable (such as `process.env.MYPROVIDER_API_KEY`). The streaming function can then access this key via `opts.apiKey` or fall back to the environment detection, ensuring secure credential handling without hardcoding secrets.

### Can I add a provider that doesn't support streaming responses?

Yes, though pi-ai is optimized for streaming. If your provider only supports batch responses, you can still implement the `StreamFunction` interface by buffering the complete response and then emitting it as a single `text_delta` event followed by `done`. However, for the best integration with pi-ai's event-driven architecture, you should implement true streaming if the provider supports it via Server-Sent Events (SSE) or similar mechanisms, parsing chunks as they arrive and pushing events to the `AssistantMessageEventStream`.

### What testing is required before submitting a new provider implementation?

You should copy an existing provider's test suite (such as [`packages/ai/test/stream.test.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/test/stream.test.ts)) and adapt it for your provider. Essential test coverage includes: basic streaming functionality to verify text generation, tool usage testing for function calling capabilities, token accounting validation to ensure accurate cost calculation, abort handling to verify `AbortSignal` cancellation works correctly, and edge cases like empty payloads or context overflow. Run `npm run check` and targeted `vitest` runs to ensure your changes don't break existing providers before submitting your pull request.