Integrating Multiple LLM Providers in the pi-ai Abstraction: A Complete Guide

The pi-ai abstraction integrates multiple LLM providers through a four-step pattern: implement dual streaming functions, export them with the StreamFunction type, register via registerApiProvider, and consume through generic high-level helpers that automatically resolve the correct implementation.

The badlogic/pi-mono repository provides a unified streaming interface for Large Language Models through its pi-ai package. When integrating multiple LLM providers within the pi-ai abstraction, developers follow a consistent pattern that treats every service as a streaming API conforming to common interfaces. This design enables seamless swapping between OpenAI, Anthropic, Google, or custom providers without changing consumer code.

The Four-Step Pattern for Integrating LLM Providers

Every provider integration follows an identical structural pattern that keeps provider-specific details isolated while sharing runtime plumbing for payload handling, abort signals, and usage tracking.

Step 1: Implement the Streaming Functions

Create two stream functions in a new provider file under packages/ai/src/providers/. The full-featured function handles complex options, while the simplified version derives from SimpleStreamOptions.

// packages/ai/src/providers/my-provider.ts
import type {
    Api,
    AssistantMessage,
    Context,
    Model,
    StreamFunction,
    StreamOptions,
    SimpleStreamOptions,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";

export interface MyProviderOptions extends StreamOptions {
    customField?: string;
}

export const streamMyProvider: StreamFunction<"my-api", MyProviderOptions> = (
    model: Model<"my-api">,
    context: Context,
    options?: MyProviderOptions,
): AssistantMessageEventStream => {
    const stream = new AssistantMessageEventStream();
    // Build request, handle chunks, push events
    return stream;
};

Step 2: Export with the StreamFunction Type

Export both implementations using the StreamFunction type to enable compile-time checking of provider-specific options.

export const streamSimpleMyProvider: StreamFunction<"my-api", SimpleStreamOptions> = (
    model,
    context,
    options,
) => {
    const base = buildBaseOptions(model, options, options?.apiKey ?? "");
    return streamMyProvider(model, context, { ...base } satisfies MyProviderOptions);
};

Step 3: Register with the Global Registry

Import the functions into packages/ai/src/providers/register-builtins.ts and call registerApiProvider to map the API identifier to your implementation.

// packages/ai/src/providers/register-builtins.ts
import { registerApiProvider } from "../api-registry.js";
import { streamMyProvider, streamSimpleMyProvider } from "./my-provider.js";

export function registerBuiltInApiProviders(): void {
    registerApiProvider({
        api: "my-api",
        stream: streamMyProvider,
        streamSimple: streamSimpleMyProvider,
    });
    // ... other providers
}

// Auto-execute on module load
registerBuiltInApiProviders();

Step 4: Consume Through High-Level Helpers

Application code consumes the provider through generic functions that automatically resolve the correct implementation based on the Model object's api field.

import { streamSimple, Model, Context } from "@mariozechner/pi-ai";

const model: Model<"my-api"> = {
    id: "my-model-1",
    name: "My-Model",
    api: "my-api",
    provider: "my-provider",
    baseUrl: "https://api.myprovider.com/v1",
    reasoning: true,
    input: ["text"],
    cost: { input: 0.001, output: 0.002, cacheRead: 0, cacheWrite: 0 },
    contextWindow: 8192,
    maxTokens: 2048,
    headers: { "X-Custom": "value" },
};

const ctx: Context = {
    systemPrompt: "You are an assistant.",
    messages: [{ role: "user", content: "Hello!", timestamp: Date.now() }],
};

const stream = streamSimple(model, ctx, { temperature: 0.5 });
stream.on("text_delta", ({ delta }) => console.log("Δ:", delta));
stream.on("done", ({ message }) => console.log("Finished:", message));

Core Types and Registry Architecture

The abstraction relies on three core files that define the contracts and wiring mechanisms.

File	Role
packages/ai/src/types.ts	Defines Api, Provider, StreamOptions, SimpleStreamOptions, StreamFunction, and AssistantMessageEventStream.
packages/ai/src/api-registry.ts	Holds the Map<Api, ApiProvider> registry and exports registerApiProvider for validation and wrapping.
packages/ai/src/stream.ts	Public entry points (stream, streamSimple, complete, completeSimple) that resolve providers from the registry.

The registration helper in packages/ai/src/api-registry.ts validates mappings and wraps stream functions to enforce the requested API:

export function registerApiProvider<TApi extends Api, TOptions extends StreamOptions>(
    provider: ApiProvider<TApi, TOptions>,
    sourceId?: string,
): void {
    apiProviderRegistry.set(provider.api, {
        provider: {
            api: provider.api,
            stream: wrapStream(provider.api, provider.stream),
            streamSimple: wrapStreamSimple(provider.api, provider.streamSimple),
        },
        sourceId,
    });
}

Built-in Provider Registration

All first-party providers are wired in packages/ai/src/providers/register-builtins.ts. This centralizes provider discovery and executes automatically on module load:

export function registerBuiltInApiProviders(): void {
    registerApiProvider({ api: "anthropic-messages", stream: streamAnthropic, streamSimple: streamSimpleAnthropic });
    registerApiProvider({ api: "openai-completions", stream: streamOpenAICompletions, streamSimple: streamSimpleOpenAICompletions });
    registerApiProvider({ api: "openai-responses", stream: streamOpenAIResponses, streamSimple: streamSimpleOpenAIResponses });
    registerApiProvider({ api: "google", stream: streamGoogle, streamSimple: streamSimpleGoogle });
    registerApiProvider({ api: "ollama", stream: streamOllama, streamSimple: streamSimpleOllama });
    registerApiProvider({ api: "openrouter", stream: streamOpenRouter, streamSimple: streamSimpleOpenRouter });
}

registerBuiltInApiProviders();

Summary

• Implement dual functions: Every provider needs a full-featured stream… and a simplified streamSimple… function that return AssistantMessageEventStream.
• Type safety: Export functions using the StreamFunction<Api, Options> type to enable compile-time checking of provider-specific options.
• Registration: Call registerApiProvider in packages/ai/src/providers/register-builtins.ts to map the API identifier to your implementation.
• Consumption: Use generic helpers like streamSimple or complete which automatically resolve the correct provider from the registry based on the Model.api field.
• Isolation: Provider-specific logic (request building, authentication, response parsing) remains isolated in dedicated files while sharing common runtime plumbing for abort signals and usage tracking.

Frequently Asked Questions

How does the registry know which provider implementation to use?

The registry in packages/ai/src/api-registry.ts maintains a Map that associates each Api identifier (like "openai-completions" or "anthropic-messages") with its corresponding ApiProvider object. When you call streamSimple(model, context, options), the function reads model.api, looks up the registered provider, and invokes its streamSimple implementation. This resolution happens at runtime, allowing you to switch providers by simply changing the Model configuration object.

What is the difference between the stream and streamSimple functions?

The stream function accepts full StreamOptions including advanced parameters like tools, toolChoice, responseFormat, and provider-specific extensions, giving you complete control over the LLM request. The streamSimple function accepts SimpleStreamOptions (typically just temperature, maxTokens, and apiKey) and internally constructs the full options object before delegating to the main stream function. This dual-layer design lets consumers choose between maximum flexibility and minimal configuration overhead.

Do I need to modify core library files to add a new LLM provider?

You only need to modify one core file: packages/ai/src/providers/register-builtins.ts. Create your provider implementation in a new file under packages/ai/src/providers/, then import and register it using registerApiProvider in the registration function. You do not need to touch packages/ai/src/stream.ts, packages/ai/src/api-registry.ts, or packages/ai/src/types.ts unless you are adding new global options or changing the core abstraction contracts.

How does error handling work when integrating multiple LLM providers?

Each provider implementation handles provider-specific errors (HTTP status codes, rate limits, malformed responses) internally within its stream function, converting them into standard AssistantMessageEvent error events or throwing exceptions that propagate through the AssistantMessageEventStream. The high-level helpers in packages/ai/src/stream.ts remain agnostic to these details, treating all providers uniformly. Consumers attach error listeners to the returned event stream to handle failures regardless of which underlying provider generated the error.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}