How the Provider-Agnostic AI Agent Runtime Integrates with LLM APIs in Instatic

Instatic's AI subsystem implements a three-layer provider-agnostic runtime under server/ai/ that abstracts LLM-specific protocols through driver modules while maintaining a unified tool-loop schema, enabling seamless API integration with OpenAI, Anthropic, Ollama, and OpenRouter without hard-coding provider dependencies.

The CoreBunch/Instatic repository decouples agent execution logic from LLM provider implementation details through a sophisticated runtime architecture. This system orchestrates multi-turn conversations, tool invocations, and streaming responses while remaining completely agnostic to the underlying LLM service.

Architecture Overview

The provider-agnostic AI agent runtime organizes functionality into three distinct layers:

Driver Layer – Provider-specific adapters that translate the generic tool-loop protocol into concrete HTTP calls. Key implementations reside in server/ai/drivers/openai.ts, server/ai/drivers/anthropic.ts, server/ai/drivers/ollama.ts, and server/ai/drivers/openrouter.ts.

Runtime Layer – The core orchestration engine that manages tool execution, conversation state, and streaming responses. Located in server/ai/runtime/runner.ts, server/ai/runtime/transport.ts, and server/ai/runtime/persister.ts, this layer has no knowledge of specific LLM providers.

MCP Bridge Layer – The external API surface that exposes the runtime to editor clients via the /_instatic/mcp endpoint. Implementation files include server/ai/mcp/server.ts and server/ai/mcp/editorBridge.ts.

Integration Flow

The runtime achieves provider-agnostic LLM API integration through a standardized six-phase pipeline:

Tool Loop Definition

All prompts follow a universal schema defined in server/ai/drivers/types.ts. This interface standardizes Message, ToolCall, and ToolResult structures, ensuring that the runtime communicates with every LLM using the same abstract protocol regardless of provider-specific payload formats.

Driver Selection

During initialization, server/ai/boot.ts reads stored credentials from server/ai/credentials/store.ts. Each credential record contains a provider field (e.g., "openai", "anthropic", "ollama"). The boot code dynamically maps these provider identifiers to their corresponding driver modules, instantiating the correct adapter without modifying runtime logic.

HTTP Abstraction

Each driver implements a minimal interface including chatCompletions and toolLoop functions that wrap provider-specific HTTP calls. Rather than duplicating networking logic, drivers utilize the shared helper in server/ai/drivers/http/execTool.ts to execute requests. Adding new LLM support requires only a thin wrapper that supplies the base URL, authentication headers, and payload transformations.

Streaming Normalization

The runtime handles streaming responses through server/ai/drivers/http/sse.ts, which manages Server-Sent Events from various providers. The transport.ts module normalizes these disparate streams into a uniform async iterator consumed by the rest of the system, abstracting away provider-specific token delivery mechanisms.

State Persistence

Conversation history, token usage metrics, and tool results are persisted by server/ai/runtime/persister.ts. This decouples conversation continuity from the underlying provider, allowing the runtime to resume interactions exactly where they left off even when switching between different LLM services.

MCP Exposure

External clients interact with the runtime exclusively through MCP endpoints. The editorBridge.ts module forwards incoming requests from the visual editor to runner.runConversation() and streams back standardized responses, rendering the specific LLM provider invisible to the caller.

Implementation Examples

The following code demonstrates how the runtime abstracts provider specifics while maintaining a consistent interface:

// Initiating a conversation using the runtime abstraction
import { runConversation } from '@server/ai/runtime';
import { getCredentials } from '@server/ai/credentials/store';

const cred = await getCredentials('my-openai-key'); // provider field determines driver selection
const result = await runConversation({
  credentialId: cred.id,
  messages: [{ role: 'user', content: 'Explain the Instatic architecture.' }],
});
console.log(result.output);

Adding support for a new LLM provider requires implementing the driver interface without modifying core runtime code:

// Extending the runtime with a custom provider driver
import { createAiDriver } from '@server/ai/drivers';

export const myLlmDriver = createAiDriver({
  baseUrl: 'https://api.my-llm.com/v1',
  async chatCompletions(payload) {
    const res = await fetch(`${this.baseUrl}/chat`, {
      method: 'POST',
      headers: { Authorization: `Bearer ${payload.apiKey}` },
      body: JSON.stringify(payload),
    });
    return await res.json();
  },
});

// Registration in server/ai/boot.ts:
// driverMap['my-llm'] = myLlmDriver;

Key Implementation Files

The provider-agnostic integration relies on these specific modules:

• server/ai/runtime/runner.ts – Core orchestrator managing tool cycles and conversation flow.
• server/ai/runtime/transport.ts – Normalizes provider-specific streaming (SSE) into uniform async iterators.
• server/ai/drivers/openai.ts – Reference implementation for OpenAI-compatible APIs.
• server/ai/drivers/anthropic.ts – Adapter for Anthropic's Claude API specifics.
• server/ai/drivers/ollama.ts – Driver for self-hosted Ollama LLM deployments.
• server/ai/drivers/openrouter.ts – Integration with OpenRouter's multi-model marketplace.
• server/ai/drivers/http/execTool.ts – Shared HTTP execution helper used across all drivers.
• server/ai/mcp/editorBridge.ts – MCP endpoint forwarding editor requests to the runtime.
• server/ai/credentials/store.ts – Secure credential storage and provider selection logic.
• server/ai/boot.ts – Runtime initialization and driver wiring.

Summary

• The three-layer architecture (Drivers, Runtime, MCP Bridge) completely isolates LLM provider specifics from agent execution logic.
• Dynamic driver selection in server/ai/boot.ts maps credential records to provider-specific implementations without hard-coding.
• Standardized tool-loop schemas in server/ai/drivers/types.ts ensure uniform communication across OpenAI, Anthropic, Ollama, and OpenRouter APIs.
• Streaming abstraction via transport.ts normalizes disparate SSE implementations into a single async iterator interface.
• MCP Bridge exposure ensures external clients remain unaware of the underlying LLM provider.

Frequently Asked Questions

What makes the Instatic runtime "provider-agnostic"?

The runtime achieves provider-agnostic behavior by defining abstract interfaces for tool loops and streaming responses in server/ai/drivers/types.ts and server/ai/runtime/transport.ts. Rather than embedding provider-specific request formats in the core logic, the system delegates all HTTP translation to swappable driver modules located in server/ai/drivers/. This allows the same conversation orchestration code in server/ai/runtime/runner.ts to operate against OpenAI, Anthropic, or Ollama without modification.

How does the runtime handle streaming responses from different LLM APIs?

Provider-specific streaming implementations are normalized through the server/ai/drivers/http/sse.ts module, which parses Server-Sent Events from various sources. The server/ai/runtime/transport.ts file then converts these disparate streams into a standardized async iterator that the runtime consumes uniformly. This abstraction handles differences in token chunking, termination signals, and metadata formats across providers.

Where are LLM credentials stored and how does the runtime select the correct driver?

Credentials reside in server/ai/credentials/store.ts, which maintains records containing provider identifiers and API keys. When the runtime initializes via server/ai/boot.ts, it maps the stored provider string (e.g., "openai", "anthropic") to the corresponding driver module in the driver registry. This dynamic selection occurs at runtime, allowing the same deployment to service requests across multiple LLM backends simultaneously.

How can I add support for a new LLM provider to the Instatic runtime?

Create a new driver file in server/ai/drivers/ that implements the AiDriver interface, specifically the chatCompletions and toolLoop methods. Utilize the shared helper in server/ai/drivers/http/execTool.ts for HTTP execution to minimize code duplication. Finally, register the driver in server/ai/boot.ts by adding it to the driver map with a unique provider key. The system will automatically route requests with matching credentials to your new implementation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}