# How the Provider-Agnostic AI Agent Runtime Integrates with LLM APIs in Instatic

> Learn how Instatic's provider-agnostic AI agent runtime seamlessly integrates with LLM APIs like OpenAI and Anthropic using a unified driver module approach.

- Repository: [CoreBunch/Instatic](https://github.com/CoreBunch/Instatic)
- Tags: how-to-guide
- Published: 2026-07-29

---

**Instatic's AI subsystem implements a three-layer provider-agnostic runtime under `server/ai/` that abstracts LLM-specific protocols through driver modules while maintaining a unified tool-loop schema, enabling seamless API integration with OpenAI, Anthropic, Ollama, and OpenRouter without hard-coding provider dependencies.**

The CoreBunch/Instatic repository decouples agent execution logic from LLM provider implementation details through a sophisticated runtime architecture. This system orchestrates multi-turn conversations, tool invocations, and streaming responses while remaining completely agnostic to the underlying LLM service.

## Architecture Overview

The provider-agnostic AI agent runtime organizes functionality into three distinct layers:

**Driver Layer** – Provider-specific adapters that translate the generic tool-loop protocol into concrete HTTP calls. Key implementations reside in [`server/ai/drivers/openai.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/openai.ts), [`server/ai/drivers/anthropic.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/anthropic.ts), [`server/ai/drivers/ollama.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/ollama.ts), and [`server/ai/drivers/openrouter.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/openrouter.ts).

**Runtime Layer** – The core orchestration engine that manages tool execution, conversation state, and streaming responses. Located in [`server/ai/runtime/runner.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/runtime/runner.ts), [`server/ai/runtime/transport.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/runtime/transport.ts), and [`server/ai/runtime/persister.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/runtime/persister.ts), this layer has no knowledge of specific LLM providers.

**MCP Bridge Layer** – The external API surface that exposes the runtime to editor clients via the `/_instatic/mcp` endpoint. Implementation files include [`server/ai/mcp/server.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/mcp/server.ts) and [`server/ai/mcp/editorBridge.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/mcp/editorBridge.ts).

## Integration Flow

The runtime achieves provider-agnostic LLM API integration through a standardized six-phase pipeline:

### Tool Loop Definition

All prompts follow a universal schema defined in [`server/ai/drivers/types.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/types.ts). This interface standardizes `Message`, `ToolCall`, and `ToolResult` structures, ensuring that the runtime communicates with every LLM using the same abstract protocol regardless of provider-specific payload formats.

### Driver Selection

During initialization, [`server/ai/boot.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/boot.ts) reads stored credentials from [`server/ai/credentials/store.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/credentials/store.ts). Each credential record contains a `provider` field (e.g., `"openai"`, `"anthropic"`, `"ollama"`). The boot code dynamically maps these provider identifiers to their corresponding driver modules, instantiating the correct adapter without modifying runtime logic.

### HTTP Abstraction

Each driver implements a minimal interface including `chatCompletions` and `toolLoop` functions that wrap provider-specific HTTP calls. Rather than duplicating networking logic, drivers utilize the shared helper in [`server/ai/drivers/http/execTool.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/http/execTool.ts) to execute requests. Adding new LLM support requires only a thin wrapper that supplies the base URL, authentication headers, and payload transformations.

### Streaming Normalization

The runtime handles streaming responses through [`server/ai/drivers/http/sse.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/http/sse.ts), which manages Server-Sent Events from various providers. The [`transport.ts`](https://github.com/CoreBunch/Instatic/blob/main/transport.ts) module normalizes these disparate streams into a uniform async iterator consumed by the rest of the system, abstracting away provider-specific token delivery mechanisms.

### State Persistence

Conversation history, token usage metrics, and tool results are persisted by [`server/ai/runtime/persister.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/runtime/persister.ts). This decouples conversation continuity from the underlying provider, allowing the runtime to resume interactions exactly where they left off even when switching between different LLM services.

### MCP Exposure

External clients interact with the runtime exclusively through MCP endpoints. The [`editorBridge.ts`](https://github.com/CoreBunch/Instatic/blob/main/editorBridge.ts) module forwards incoming requests from the visual editor to `runner.runConversation()` and streams back standardized responses, rendering the specific LLM provider invisible to the caller.

## Implementation Examples

The following code demonstrates how the runtime abstracts provider specifics while maintaining a consistent interface:

```typescript
// Initiating a conversation using the runtime abstraction
import { runConversation } from '@server/ai/runtime';
import { getCredentials } from '@server/ai/credentials/store';

const cred = await getCredentials('my-openai-key'); // provider field determines driver selection
const result = await runConversation({
  credentialId: cred.id,
  messages: [{ role: 'user', content: 'Explain the Instatic architecture.' }],
});
console.log(result.output);

```

Adding support for a new LLM provider requires implementing the driver interface without modifying core runtime code:

```typescript
// Extending the runtime with a custom provider driver
import { createAiDriver } from '@server/ai/drivers';

export const myLlmDriver = createAiDriver({
  baseUrl: 'https://api.my-llm.com/v1',
  async chatCompletions(payload) {
    const res = await fetch(`${this.baseUrl}/chat`, {
      method: 'POST',
      headers: { Authorization: `Bearer ${payload.apiKey}` },
      body: JSON.stringify(payload),
    });
    return await res.json();
  },
});

// Registration in server/ai/boot.ts:
// driverMap['my-llm'] = myLlmDriver;

```

## Key Implementation Files

The provider-agnostic integration relies on these specific modules:

- **[`server/ai/runtime/runner.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/runtime/runner.ts)** – Core orchestrator managing tool cycles and conversation flow.
- **[`server/ai/runtime/transport.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/runtime/transport.ts)** – Normalizes provider-specific streaming (SSE) into uniform async iterators.
- **[`server/ai/drivers/openai.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/openai.ts)** – Reference implementation for OpenAI-compatible APIs.
- **[`server/ai/drivers/anthropic.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/anthropic.ts)** – Adapter for Anthropic's Claude API specifics.
- **[`server/ai/drivers/ollama.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/ollama.ts)** – Driver for self-hosted Ollama LLM deployments.
- **[`server/ai/drivers/openrouter.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/openrouter.ts)** – Integration with OpenRouter's multi-model marketplace.
- **[`server/ai/drivers/http/execTool.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/http/execTool.ts)** – Shared HTTP execution helper used across all drivers.
- **[`server/ai/mcp/editorBridge.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/mcp/editorBridge.ts)** – MCP endpoint forwarding editor requests to the runtime.
- **[`server/ai/credentials/store.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/credentials/store.ts)** – Secure credential storage and provider selection logic.
- **[`server/ai/boot.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/boot.ts)** – Runtime initialization and driver wiring.

## Summary

- **The three-layer architecture** (Drivers, Runtime, MCP Bridge) completely isolates LLM provider specifics from agent execution logic.
- **Dynamic driver selection** in [`server/ai/boot.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/boot.ts) maps credential records to provider-specific implementations without hard-coding.
- **Standardized tool-loop schemas** in [`server/ai/drivers/types.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/types.ts) ensure uniform communication across OpenAI, Anthropic, Ollama, and OpenRouter APIs.
- **Streaming abstraction** via [`transport.ts`](https://github.com/CoreBunch/Instatic/blob/main/transport.ts) normalizes disparate SSE implementations into a single async iterator interface.
- **MCP Bridge exposure** ensures external clients remain unaware of the underlying LLM provider.

## Frequently Asked Questions

### What makes the Instatic runtime "provider-agnostic"?

The runtime achieves provider-agnostic behavior by defining abstract interfaces for tool loops and streaming responses in [`server/ai/drivers/types.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/types.ts) and [`server/ai/runtime/transport.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/runtime/transport.ts). Rather than embedding provider-specific request formats in the core logic, the system delegates all HTTP translation to swappable driver modules located in `server/ai/drivers/`. This allows the same conversation orchestration code in [`server/ai/runtime/runner.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/runtime/runner.ts) to operate against OpenAI, Anthropic, or Ollama without modification.

### How does the runtime handle streaming responses from different LLM APIs?

Provider-specific streaming implementations are normalized through the [`server/ai/drivers/http/sse.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/http/sse.ts) module, which parses Server-Sent Events from various sources. The [`server/ai/runtime/transport.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/runtime/transport.ts) file then converts these disparate streams into a standardized async iterator that the runtime consumes uniformly. This abstraction handles differences in token chunking, termination signals, and metadata formats across providers.

### Where are LLM credentials stored and how does the runtime select the correct driver?

Credentials reside in [`server/ai/credentials/store.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/credentials/store.ts), which maintains records containing `provider` identifiers and API keys. When the runtime initializes via [`server/ai/boot.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/boot.ts), it maps the stored provider string (e.g., `"openai"`, `"anthropic"`) to the corresponding driver module in the driver registry. This dynamic selection occurs at runtime, allowing the same deployment to service requests across multiple LLM backends simultaneously.

### How can I add support for a new LLM provider to the Instatic runtime?

Create a new driver file in `server/ai/drivers/` that implements the `AiDriver` interface, specifically the `chatCompletions` and `toolLoop` methods. Utilize the shared helper in [`server/ai/drivers/http/execTool.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/drivers/http/execTool.ts) for HTTP execution to minimize code duplication. Finally, register the driver in [`server/ai/boot.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/ai/boot.ts) by adding it to the driver map with a unique provider key. The system will automatically route requests with matching credentials to your new implementation.