How Magnitude Achieves Provider-Agnostic AI Model Interaction

Magnitude achieves provider-agnostic AI model interaction through a unified TypeScript interface layer that abstracts vendor-specific implementations behind a standard contract, model catalog, and transport-independent streaming protocol.

The open-source Magnitude framework (magnitudedev/magnitude) eliminates vendor lock-in by decoupling application logic from underlying AI providers. This architecture enables developers to switch between OpenAI, Ollama, or custom local daemons without changing application code. At the heart of this capability lies the @magnitudedev/ai package, which implements a three-pillar abstraction strategy grounded in Effect-TS type safety.

The Unified Provider Contract

All model providers in Magnitude implement a single TypeScript interface defined in packages/ai/src/provider/contract.ts. This Provider interface establishes a universal vocabulary for AI interactions regardless of the backend vendor.

The contract mandates three core capabilities:

• Model Discovery – Providers implement listModels and modelInfo methods to advertise available models and their capabilities.
• Standardized Completion – The call method accepts a provider-agnostic request schema (ChatCompletionRequest) and returns a provider-agnostic response schema (ChatCompletionResponse).
• Type Safety – All schemas use Effect-TS schemas, ensuring compile-time guarantees that persist across provider boundaries.

Because every provider speaks this common language, the rest of the codebase remains ignorant of whether it is communicating with a cloud API or a local GPU daemon.

Model Catalog and Runtime Binding

Magnitude maintains a provider-independent model catalog that resolves abstract model identifiers to concrete implementations. The catalog logic resides in packages/ai/src/provider/catalog.ts, while the daemon-side implementation lives in packages/acn/src/provider-model-catalog.ts.

When a consumer requests a specific model—such as "phi-2-medium"—the system executes the following resolution flow:

1. The catalog consults ProviderModelBindOptions to identify which registered provider offers the requested model.
2. The ProviderRegistryService (defined in packages/providers/src/registry.ts) matches the model requirements against registered providers.
3. The consumer receives a handle to the model without ever learning which vendor hosts it.

This binding mechanism allows runtime flexibility. Applications can request hardware-constrained models (e.g., maxMemoryGiB: 8) and receive the best available match regardless of whether the provider is OpenAI, Ollama, or a custom enterprise endpoint.

Transport-Independent Streaming

Streaming responses pose a significant challenge for provider abstraction because each vendor may use different wire protocols. Magnitude solves this through transport agnosticism implemented in two key files:

• packages/ai/src/transport/sse.ts – Handles the low-level SSE/stream transport, normalizing raw bytes from HTTP endpoints, Unix sockets, or local daemons.
• packages/ai/src/streaming/parser/parser.ts – Parses incoming streams into the unified ChatCompletionChunk type.

These modules ensure that whether data arrives from a cloud SSE endpoint or a local Unix socket, the application receives standardized chunks containing deltas and metadata. The streaming API remains identical across all providers.

SDK Orchestration and the Provider Client

The ProviderClient (located in packages/sdk/src/provider-client.ts) orchestrates the provider-agnostic flow by creating a thin RPC façade over the ACN daemon. This client forwards requests to the appropriate provider implementation while maintaining the abstract interface.

The orchestration flow works as follows:

1. The SDK initializes via createMagnitudeClient.
2. The client queries the ProviderRegistryService to locate a provider matching the requested model.
3. The selected provider implementation (e.g., @magnitudedev/providers/openai or @magnitudedev/providers/ollama) executes the concrete call logic.
4. Results return through the unified schema layers, appearing identical to the consumer regardless of origin.

Because the architecture relies on typed Effect-TS services, adding a new provider requires only implementing the Provider interface and registering it with the registry—no changes are needed elsewhere in the codebase.

Practical Implementation

The following TypeScript demonstrates the provider-agnostic workflow:

// Initialize the SDK entry point
import { createMagnitudeClient } from '@magnitudedev/sdk';
const client = await createMagnitudeClient({});

// Query the catalog for hardware-appropriate models
import { ProviderCatalog } from '@magnitudedev/ai';
const model = await ProviderCatalog.selectBestModel({
  maxMemoryGiB: 8,
});

// Execute chat completion through the abstracted API
import { chatCompletion } from '@magnitudedev/ai';
const response = await chatCompletion({
  model,
  messages: [{ role: 'user', content: 'Explain provider-agnostic AI architecture.' }],
});

// Stream responses uniformly from any provider
await chatCompletion.stream({
  model,
  messages: [{ role: 'user', content: 'Tell me a joke.' }],
  onChunk: chunk => console.log(chunk.delta?.content ?? ''),
});

All imports resolve to identical public APIs whether the underlying provider operates via cloud API or local inference.

Summary

• Unified Interface – The Provider contract in packages/ai/src/provider/contract.ts forces all vendors to implement listModels, modelInfo, and call using standardized request/response schemas.
• Dynamic Resolution – The model catalog in packages/ai/src/provider/catalog.ts binds abstract model requests to concrete providers at runtime without consumer awareness.
• Streaming Abstraction – Transport layers in packages/ai/src/transport/sse.ts and packages/ai/src/streaming/parser/parser.ts normalize all streams into ChatCompletionChunk types.
• Type Safety – Effect-TS schemas and services ensure compile-time correctness across provider boundaries, making the system extensible without core modifications.

Frequently Asked Questions

What interface must providers implement to join the Magnitude ecosystem?

Providers must implement the Provider interface defined in packages/ai/src/provider/contract.ts. This requires implementing listModels and modelInfo for discovery, plus a call method that accepts ChatCompletionRequest and returns ChatCompletionResponse objects. All methods must use Effect-TS schemas for type safety.

How does Magnitude handle model discovery across different vendors?

Magnitude uses the ProviderCatalog class in packages/ai/src/provider/catalog.ts combined with the ProviderRegistryService in packages/providers/src/registry.ts. When an application requests a model, the catalog queries registered providers through ProviderModelBindOptions to find matches based on model name, hardware constraints, and availability.

Can I stream responses from any provider using the same API?

Yes. Magnitude normalizes all streaming protocols through packages/ai/src/transport/sse.ts and packages/ai/src/streaming/parser/parser.ts. These modules decode provider-specific formats into the unified ChatCompletionChunk type, allowing the chatCompletion.stream() method to function identically whether streaming from OpenAI, Ollama, or custom local daemons.

What is required to add a new custom provider to Magnitude?

Adding a new provider requires creating a package that implements the Provider interface from packages/ai/src/provider/contract.ts and registering it with the ProviderRegistryService. The provider must implement model discovery and chat completion logic using the standard schemas. No modifications to the SDK or application code are necessary once the provider is registered.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}