How to Integrate Custom LLM Providers Beyond the Default OpenAI-Compatible API

To integrate custom LLM providers, implement the LLMClient interface from packages/llms/src/types.ts to translate provider-specific API calls, then inject your implementation into the LLM class to replace the default OpenAIClient.

The alibaba/page-agent repository provides an LLM-agnostic agent framework that enables seamless integration with any model provider through a single TypeScript interface. Whether you need to connect to Anthropic's Claude, Google's Gemini, or a private inference endpoint, you can integrate custom LLM providers without altering the agent's core tool execution, retry logic, or event handling. The entire abstraction hinges on the LLMClient contract defined in packages/llms/src/types.ts.

Understanding the LLMClient Interface

The LLMClient interface defines the minimal surface area required to plug in a new provider. Located in packages/llms/src/types.ts, it requires a single method that all higher-level agent code uses to interact with language models:

export interface LLMClient {
  invoke(
    messages: Message[],
    tools: Record<string, Tool>,
    abortSignal?: AbortSignal,
    options?: InvokeOptions
  ): Promise<InvokeResult>
}

This interface decouples the agent's business logic from specific provider implementations. The default OpenAIClient in packages/llms/src/OpenAIClient.ts implements this contract for OpenAI-compatible endpoints. To support a different provider, you create a class that satisfies this same interface.

Implementing Your Custom Provider Client

Step 1: Scaffold the Client Class

Create a new module that imports the shared types and error classes from @page-agent/llms. Your class must accept a Required<LLMConfig> object in its constructor, which provides the baseURL, apiKey, and optional customFetch function:

// my-provider/src/MyProviderClient.ts
import type { LLMClient, Message, Tool, InvokeOptions, InvokeResult, LLMConfig } from '@page-agent/llms';
import { InvokeError, InvokeErrorType } from '@page-agent/llms';

export class MyProviderClient implements LLMClient {
  private readonly config: Required<LLMConfig>;
  private readonly fetch: typeof globalThis.fetch;

  constructor(config: Required<LLMConfig>) {
    this.config = config;
    this.fetch = config.customFetch ?? fetch;
  }

  async invoke(
    messages: Message[],
    tools: Record<string, Tool>,
    abortSignal?: AbortSignal,
    options?: InvokeOptions
  ): Promise<InvokeResult> {
    // Implementation detailed below
  }
}

Step 2: Translate Requests and Execute Tool Calls

Inside the invoke method, you must perform four critical operations: convert the generic message format to your provider's schema, execute the HTTP request, parse the response to extract tool calls, and validate arguments against the tool's inputSchema. This mirrors the pattern used in the reference OpenAIClient:

async invoke(
  messages: Message[],
  tools: Record<string, Tool>,
  abortSignal?: AbortSignal,
  options?: InvokeOptions
): Promise<InvokeResult> {
  // ① Convert messages + tools → provider request body
  const body = { 
    model: this.config.model,
    messages: messages.map(m => ({ role: m.role, content: m.content })),
    // Adapt tool definitions to provider format
    functions: Object.entries(tools).map(([name, tool]) => ({
      name,
      description: tool.description,
      parameters: tool.inputSchema
    }))
  };

  // ② Call the provider
  const response = await this.fetch(this.config.baseURL + '/chat/completions', {
    method: 'POST',
    headers: { 
      'Content-Type': 'application/json', 
      Authorization: `Bearer ${this.config.apiKey}` 
    },
    body: JSON.stringify(body),
    signal: abortSignal,
  });

  if (!response.ok) {
    // Translate HTTP errors into InvokeError (mirrors OpenAIClient)
    const err = await response.json().catch(() => ({}));
    throw new InvokeError(
      InvokeErrorType.NETWORK_ERROR,
      `Provider error ${response.status}`,
      err
    );
  }

  // ③ Parse provider-specific response and extract the tool call
  const data = await response.json();
  const toolName = data.calledTool;           // ← adapt to provider format
  const rawArgs = data.toolArguments;         // ← adapt to provider format

  const tool = tools[toolName];
  if (!tool) {
    throw new InvokeError(InvokeErrorType.UNKNOWN, `Tool "${toolName}" not registered`);
  }

  const args = JSON.parse(rawArgs);
  const validation = tool.inputSchema.safeParse(args);
  if (!validation.success) {
    throw new InvokeError(InvokeErrorType.INVALID_TOOL_ARGS, 'Argument validation failed');
  }

  // ④ Execute the tool and build the result
  const toolResult = await tool.execute(validation.data);
  return {
    toolCall: { name: toolName, args: validation.data },
    toolResult,
    usage: { promptTokens: 0, completionTokens: 0, totalTokens: 0 }, // optional
    rawResponse: data,
    rawRequest: body,
  };
}

Step 3: Handle Structured Errors

Page Agent expects specific error types to trigger appropriate retry logic. Always throw InvokeError from packages/llms/src/errors.ts when operations fail. Use InvokeErrorType.NETWORK_ERROR for connectivity issues, InvokeErrorType.INVALID_TOOL_ARGS for schema validation failures, and InvokeErrorType.UNKNOWN for unexpected states.

Wiring Your Client into Page Agent

The LLM class constructor in packages/llms/src/index.ts instantiates a default OpenAIClient, but you can override this after construction or through subclassing. Import parseLLMConfig to normalize your configuration object:

import { LLM, parseLLMConfig } from '@page-agent/llms';
import { MyProviderClient } from './my-provider/MyProviderClient';

const rawConfig = {
  baseURL: 'https://api.myprovider.com',
  apiKey: 'YOUR_KEY',
  model: 'my-model',
};
const llm = new LLM(rawConfig);
// Replace the default client with your custom one
(llm as any).client = new MyProviderClient(parseLLMConfig(rawConfig));

For a more robust approach, subclass LLM to ensure your client is used consistently:

import { LLM } from '@page-agent/llms';
import { MyProviderClient } from './my-provider/MyProviderClient';

export class MyLLM extends LLM {
  constructor(config) {
    super(config);
    this.client = new MyProviderClient(this.config);
  }
}

Once wired, all higher-level code—including the core agent, tool orchestration, and retry handlers—interacts with your provider transparently through the standard invoke method.

Key Source Files for Reference

• packages/llms/src/types.ts — Defines LLMClient, Message, Tool, InvokeResult, and InvokeOptions interfaces.
• packages/llms/src/OpenAIClient.ts — Reference implementation showing OpenAI/Chat-Completions integration patterns.
• packages/llms/src/index.ts — Contains the LLM wrapper class with retry logic and event handling.
• packages/llms/src/errors.ts — Structured error hierarchy (InvokeError, InvokeErrorType) required for custom implementations.
• packages/llms/src/utils.ts — Helper functions for patching model-specific payloads.

Summary

• Implement LLMClient: Create a class that satisfies the interface defined in packages/llms/src/types.ts with a single invoke method.
• Translate formats: Convert generic Message[] arrays and Tool records to your provider's specific REST API schema within the invoke method.
• Use structured errors: Throw InvokeError instances from packages/llms/src/errors.ts to ensure proper retry and error handling behavior.
• Inject the client: Replace the default client in the LLM class instance or subclass LLM to use your implementation throughout the agent stack.
• Leverage existing infrastructure: All retry logic, abort signal handling, and tool execution orchestration remain functional regardless of the underlying provider.

Frequently Asked Questions

Do I need to modify the Page Agent source code to add a new provider?

No. The architecture is designed for external extension. You implement the LLMClient interface in your own codebase, import the necessary types from @page-agent/llms, and inject your implementation into the LLM class instance. All changes remain in your project without requiring forks or patches to the original alibaba/page-agent repository.

How does the interface handle tool calling with non-OpenAI schemas?

The invoke method receives a Record<string, Tool> containing tool definitions with description, inputSchema, and execute methods. Your implementation translates these into the provider's native function-calling format (such as Anthropic's tools array or Google's functionDeclarations), parses the provider's response to identify which tool was called, validates arguments using tool.inputSchema.safeParse(), and returns the result wrapped in an InvokeResult object.

Can I reuse the retry logic with custom providers?

Yes. The retry logic resides in the LLM wrapper class in packages/llms/src/index.ts, which calls your LLMClient.invoke implementation. As long as you throw InvokeError with appropriate InvokeErrorType codes (such as NETWORK_ERROR for transient failures), the existing exponential backoff and circuit breaker mechanisms apply automatically.

What if my provider requires custom authentication headers beyond Bearer tokens?

The LLMConfig object passed to your client constructor includes all configuration values, including the apiKey. You can extend your client implementation to accept additional configuration parameters (such as custom headers or region settings) and apply them in the fetch call within your invoke method. The customFetch option also allows you to inject a pre-configured fetch instance that handles authentication middleware.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}