How Maka Integrates with OpenAI-Compatible Models Using `ai-sdk-backend.ts`

Maka integrates with OpenAI-compatible models by using the AiSdkBackend class to instantiate a ModelAdapter that delegates to the Vercel AI SDK's createOpenAICompatible constructor, automatically configuring provider-specific options and handling streaming, retries, and tool calls through a unified pipeline.

The Apache Maka runtime abstracts all LLM providers behind a single interface, allowing seamless switching between Anthropic, Google, OpenAI, and any OpenAI-compatible endpoint. At the core of this abstraction lies packages/runtime/src/ai-sdk-backend.ts, which orchestrates model initialization, streaming, and tool integration. When integrating with OpenAI-compatible models, the backend leverages the Vercel AI SDK via a dedicated factory pattern that resolves provider configurations and manages the full request lifecycle.

Architecture of the AI SDK Backend

Backend Initialization and ModelAdapter Creation

When AiSdkBackend is instantiated at line 1125 in packages/runtime/src/ai-sdk-backend.ts, it receives AiSdkBackendInput and immediately creates a ModelAdapter. This adapter holds the connection details, API key, model ID, and resolved provider options. The ModelAdapter serves as the abstraction layer that hides provider-specific differences from the rest of the runtime, normalizing calls to doGenerate and doStream regardless of the underlying SDK.

Provider Option Resolution

Before model creation, the backend resolves provider-specific configurations. At line 1224 in ai-sdk-backend.ts, it assigns:

this.resolvedProviderOptions = input.providerOptions ?? buildProviderOptions(...)

The buildProviderOptions function (lines 56-65 in packages/runtime/src/model-factory.ts) constructs a SharedV4ProviderOptions object containing OpenAI-compatible fields such as reasoningEffort and serviceTier.

Model Factory and OpenAI-Compatible SDK Integration

Selecting the SDK Constructor

The getAIModel function in packages/runtime/src/model-factory.ts (lines 70-86) acts as the factory for concrete model instances. For OpenAI-compatible providers, the switch case openai-compatible invokes:

createOpenAICompatible({ name, apiKey, baseURL, ... }).chatModel(modelId)

This returns a model implementing the LanguageModelV4 interface, which the backend uses for all subsequent operations.

Reasoning and Metadata Handling

For advanced capabilities like chain-of-thought reasoning, the factory composes request transforms. Lines 199-226 in model-factory.ts handle transformRequestBody for reasoning details. The withReasoningDetails proxy (lines 302-345) intercepts OpenAI-compatible reasoning_content and reasoning_details fields, injecting them into streamed responses when requested.

Streaming, Retries, and Tool Integration

Unified Event Streaming

The backend's send() method (lines 1912-1990 in ai-sdk-backend.ts) constructs an AsyncEventQueue and applies provider-agnostic streaming logic. It uses StreamWatchdog for timeout enforcement and respects providerRetryDelayMs for backoff strategies. These mechanisms work uniformly across all providers because the ModelAdapter normalizes the underlying SDK calls.

Tool Call Execution

After each generation step, tool calls are resolved via the ToolRuntime (lines 1430-1470). The backend processes OpenAI-compatible tool calls through generic logic that records token usage, enforces budget constraints, and re-injects results into subsequent conversation turns without requiring provider-specific handling.

Implementation Example

The following example demonstrates how to instantiate the backend and send a prompt to an OpenAI-compatible model:

import { AiSdkBackend } from '@maka/runtime/ai-sdk-backend';

// Prepare backend input
const backendInput = {
  sessionId: 'sess-01',
  header: { workspaceRoot: '/repo' },
  connection: {
    providerType: 'openai-compatible',
    slug: 'my-custom-relay'
  },
  apiKey: process.env.OPENAI_API_KEY!,
  modelId: 'gpt-4o-mini',
  tools: [],
  providerOptions: undefined // Auto-computed by buildProviderOptions
};

// Instantiate the backend
const backend = new AiSdkBackend(backendInput);

// Send a prompt - the backend handles SDK specifics internally
await backend.send({
  messages: [{ role: 'user', content: 'Write a short poem about clouds.' }]
});

This implementation creates a ModelAdapter that automatically selects createOpenAICompatible, applies resolved provider options, and manages the full lifecycle of the request including streaming and retries.

Summary

• AiSdkBackend serves as the universal entry point for all LLM providers in Apache Maka, abstracting provider differences behind a consistent API.
• OpenAI-compatible integration uses createOpenAICompatible from the Vercel AI SDK, selected via the factory pattern in model-factory.ts.
• buildProviderOptions automatically configures provider-specific settings like reasoningEffort and serviceTier when explicit options are not provided.
• The ModelAdapter normalizes streaming, retry logic, and tool call execution across all providers, including OpenAI-compatible endpoints.
• All implementation details are contained within packages/runtime/src/ai-sdk-backend.ts and packages/runtime/src/model-factory.ts.

Frequently Asked Questions

What file handles the core integration logic for OpenAI-compatible models in Maka?

The core integration logic resides in packages/runtime/src/ai-sdk-backend.ts, which orchestrates the ModelAdapter and streaming pipeline, and packages/runtime/src/model-factory.ts, which contains the getAIModel function that instantiates the Vercel AI SDK's createOpenAICompatible constructor for OpenAI-compatible providers.

How does Maka handle provider-specific options like reasoning effort for OpenAI-compatible models?

Maka automatically builds provider-specific options through the buildProviderOptions function located at lines 56-65 in model-factory.ts. This function constructs a SharedV4ProviderOptions object that includes OpenAI-compatible fields such as reasoningEffort and serviceTier, which are then passed to the SDK constructor.

Can I use custom base URLs with OpenAI-compatible models in Maka?

Yes, when the openai-compatible case is selected in getAIModel (lines 70-86 of model-factory.ts), the function passes your custom baseURL along with the apiKey and name to createOpenAICompatible, enabling integration with any OpenAI-compatible endpoint including custom relays and local deployments.

How does Maka manage streaming and retries for OpenAI-compatible models?

The AiSdkBackend class manages streaming through its send() method (lines 1912-1990), which creates an AsyncEventQueue and applies providerRetryDelayMs for backoff. A StreamWatchdog enforces timeouts. These mechanisms work uniformly across all providers because the ModelAdapter abstracts the underlying doStream and doGenerate calls.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}