How the Model Transport Layer Supports Different LLM Providers in Prime Agent

Prime Agent's transport layer abstracts LLM communication through a provider-agnostic system that resolves the optimal protocol—SSE, WebSocket, JSONL, or private-framed—at runtime.

The Prime Agent codebase implements a flexible model transport layer that enables seamless switching between LLM providers without changing consumer code. This architecture decouples the streaming protocol from the business logic, allowing the same agent code to work with OpenAI, Anthropic, AWS Bedrock, and custom daemon deployments.

Core Transport Concepts

The transport system in packages/ai/src/ revolves around three foundational constructs that work together to normalize provider differences.

The Transport Type Definition

In packages/ai/src/types.ts, the Transport enum declares all supported transport modes:

• "sse" — Server-Sent Events for HTTP-based streaming
• "websocket" — Bidirectional WebSocket connections
• "jsonl" — Newline-delimited JSON over sockets
• "private-framed" — Proprietary framing for daemon mode
• "auto" — Runtime selection of the best available transport

This type is used throughout the codebase whenever transport configuration is passed between components.

Provider Registration and Capability Declaration

Each LLM provider registers its supported transports in packages/ai/src/api-registry.ts. The registry maps an API identifier to a factory that instantiates transport-specific clients:

registerProvider("openai-chat", {
  transports: ["sse", "websocket"],
  createClient: (opts) => new OpenAIChatClient(opts),
});

registerProvider("anthropic", {
  transports: ["sse", "websocket"],
  createClient: (opts) => new AnthropicClient(opts),
});

registerProvider("bedrock", {
  transports: ["sse"],
  createClient: (opts) => new BedrockClient(opts),
});

This registration pattern allows the system to validate transport choices and reject incompatible configurations early.

Unified Stream Interface

The stream() function in packages/ai/src/stream.ts serves as the single entry point for all LLM requests. It resolves the appropriate transport, instantiates the matching client, and returns a normalized AssistantMessageEventStream that emits standardized event types regardless of the underlying protocol.

Transport Resolution Flow

Understanding how the model transport layer resolves provider-specific protocols requires following the execution path from user configuration to active connection.

Step 1: User Configuration via SettingsManager

User preferences are persisted in packages/coding-agent/src/core/settings-manager.ts. The SettingsManager exposes a getTransport() method that returns the selected mode:

const settingsManager = new SettingsManager();
const transport = settingsManager.getTransport(); // "sse" | "websocket" | "jsonl" | "auto"

When "auto" is selected, the system defers to provider capability detection.

Step 2: Transport Resolution Logic

The stream() function implements the resolution algorithm:

1. Query the registry for the requested API
2. If transport is "auto", select the most efficient transport from provider.transports
3. Validate that the chosen transport is in the provider's capability list
4. Pass transport and provider options to the factory

Step 3: Factory Instantiation

The registry's createClient function receives the resolved transport and returns a protocol-specific implementation. Each client adheres to a common interface:

• startStream(request): Promise<void>
• onMessage(callback): void
• close(): void

Step 4: Protocol-Normalized Event Emission

Despite different underlying protocols, every client emits AssistantMessageEvent objects with these normalized types:

Event Type	Description
text	Model-generated text content
tool_call	Function/tool invocation request
thinking	Chain-of-thought or reasoning output
usage	Token consumption statistics
stop	Stream termination signal

Transport-Specific Client Implementations

The model transport layer delegates actual I/O to specialized clients in packages/ai/src/providers/.

SSE Client (openai-sse.ts)

Uses the browser EventSource API or polyfill for HTTP-based streaming. Suitable for providers that expose standard OpenAI-compatible endpoints over HTTP.

WebSocket Client (openai-ws.ts)

Establishes persistent WebSocket connections for lower latency and bidirectional communication. Required for some Anthropic endpoints and custom deployments.

JSONL Client (daemon-jsonl.ts)

Writes newline-delimited JSON to a Unix domain socket or TCP connection. Used when communicating with the Prime Agent daemon process for local model inference.

Private-Framed Client (private-framed.ts)

Implements a custom binary framing protocol with length-prefixed messages. Optimized for high-throughput local communication where HTTP overhead is unacceptable.

Working with the Transport Layer: Code Examples

Explicit Transport Selection

Force SSE for maximum compatibility across providers:

import { stream, type StreamOptions } from "@prime-agent/ai";

const opts: StreamOptions = {
  model: "gpt-4o",
  api: "openai-chat",
  transport: "sse",  // Explicit protocol choice
  messages: [{ role: "user", content: "Explain quantum entanglement." }],
};

const eventStream = await stream(opts);
for await (const event of eventStream) {
  if (event.type === "text") {
    process.stdout.write(event.content);
  } else if (event.type === "stop") {
    console.log("\n[Stream complete]");
  }
}

Automatic Transport Selection

Let the system optimize based on provider capabilities:

const opts: StreamOptions = {
  model: "anthropic.claude-3-sonnet-20240229-v1:0",
  api: "bedrock",
  transport: "auto",  // Resolves to "sse" (Bedrock's only supported transport)
  messages: [{ role: "user", content: "Write a haiku about sunrise." }],
};

const eventStream = await stream(opts);
// Identical consumption code works regardless of resolved transport
for await (const event of eventStream) {
  console.log(`${event.type}: ${JSON.stringify(event)}`);
}

Interactive Mode Integration

In packages/coding-agent/src/modes/interactive/interactive-mode.ts, the transport flows through the session lifecycle:

class InteractiveMode {
  constructor(private settings: SettingsManager) {}

  async startChat(model: string, api: string) {
    const transport = this.settings.getTransport();
    
    const stream = await stream({
      model,
      api,
      transport,  // User preference or "auto"
      messages: this.conversationHistory,
    });
    
    // Consume identically regardless of provider or protocol
    for await (const ev of stream) {
      this.handleEvent(ev);
    }
  }
}

Adding a New LLM Provider

To extend the model transport layer for a new provider, implement these steps:

1. Define capabilities in api-registry.ts

   registerProvider("custom-llm", {
     transports: ["sse", "jsonl"],
     createClient: (opts) => new CustomLlmClient(opts),
   });

2. Implement the client in packages/ai/src/providers/custom-llm.ts

   • Extend base client interface
   • Handle connection establishment
   • Parse provider-specific response format
   • Emit normalized AssistantMessageEvent objects

3. Register in exports to make available to stream()

Summary

• The model transport layer in Prime Agent decouples streaming protocols from business logic through a registry-based provider system.
• packages/ai/src/types.ts defines the Transport union type with "sse", "websocket", "jsonl", "private-framed", and "auto".
• packages/ai/src/api-registry.ts maps each LLM provider to its supported transports and client factory.
• packages/ai/src/stream.ts provides the unified stream() entry point that resolves transport and returns normalized event streams.
• Transport selection flows from SettingsManager through resolution logic to protocol-specific clients, with "auto" enabling runtime optimization.
• All clients emit identical AssistantMessageEvent types, making provider swapping transparent to consuming code.

Frequently Asked Questions

What happens if I specify a transport the provider doesn't support?

The transport layer throws a validation error during stream() invocation. In packages/ai/src/stream.ts, the resolution logic checks provider.transports.includes(resolvedTransport) before instantiating the client, failing fast with a descriptive message rather than attempting an incompatible connection.

Can I use "auto" transport with any provider?

Yes. When "auto" is specified, the system selects the most efficient transport from the provider's declared capabilities. For OpenAI this typically resolves to "websocket"; for Bedrock it resolves to "sse" as the only available option. The selection prioritizes connection reuse and latency characteristics.

How do I add WebSocket support to a custom provider implementation?

Register "websocket" in your provider's transports array in api-registry.ts, then implement a client class handling the WebSocket lifecycle in packages/ai/src/providers/. Your client must open the connection in startStream(), emit parsed events via the callback, and clean up in close(). Reference openai-ws.ts for the canonical implementation pattern.

Where is the transport setting persisted between sessions?

The SettingsManager in packages/coding-agent/src/core/settings-manager.ts handles persistence. It serializes the transport preference alongside other configuration, restoring the value on agent restart. The TUI exposes this through settings panels, and programmatic access uses settingsManager.setTransport() and getTransport().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}