How the AI Provider Abstraction Works in Magnitude: Unifying Heterogeneous Model Providers

Magnitude implements a unified AI provider abstraction that treats every model service—whether OpenAI, Anthropic, or custom HTTP endpoints—as an interchangeable component conforming to a standard TypeScript contract, enabling seamless provider swapping without refactoring consumer code.

The open-source Magnitude framework (magnitudedev/magnitude) achieves vendor neutrality through a layered AI provider abstraction that separates interface definitions from concrete implementations. By enforcing strict contracts across disparate model back-ends, the architecture allows developers to integrate new providers by implementing a single interface in packages/providers/src/ rather than modifying core application logic.

The Provider Contract: Defining the Universal Interface

The foundation of Magnitude’s flexibility lies in the Provider interface defined in packages/ai/src/provider/contract.ts. This contract establishes the mandatory capabilities that any model service must expose:

export interface Provider {
  /** Unique identifier, e.g. “openai”, “anthropic”, “custom” */
  readonly id: ProviderId;

  /** Human‑readable name */
  readonly name: string;

  /** Returns a catalog of models offered by the provider */
  listModels(): Effect.Effect<ReadonlyArray<ModelInfo>>;

  /** Retrieves a single model’s description */
  getModel(id: ModelId): Effect.Effect<ModelInfo>;

  /** Invokes a model (chat, completion, embeddings…) */
  call(options: ProviderCallOptions): Effect.Effect<ProviderResponse>;
}

All providers must implement these methods using Effect.Effect from Effect-TS, which provides functional error handling and composable asynchronous workflows. The rest of Magnitude interacts exclusively with this interface, ensuring that swapping from OpenAI to a custom endpoint requires zero changes to UI components or SDK consumers.

Provider Registry and Client Architecture

The dynamic discovery and management of providers occurs through two complementary services in the packages/providers/src/ directory.

The Registry Service

The ProviderRegistryService in packages/providers/src/registry.ts maintains a singleton map of ProviderId → Provider:

export interface ProviderRegistryService {
  /** Register a provider */
  register(provider: Provider): Effect.Effect<void>;

  /** Resolve a provider by its ID */
  get(id: ProviderId): Effect.Effect<Provider>;
}

During application startup, the ProviderClient (located in packages/providers/src/provider-client.ts) instantiates an Effect-based RPC client that communicates with the Magnitude daemon (acn). It then discovers and registers each concrete provider implementation with the registry, making them available for injection throughout the system.

Concrete Provider Implementations

Magnitude ships with several built-in implementations that demonstrate the abstraction’s flexibility. Each resides in its own subdirectory under packages/providers/src/:

• src/magnitude/provider.ts – Wraps the native Magnitude daemon models, handling authentication and transport specifics for the platform’s internal inference engine.
• src/exa/provider.ts – Implements the EXA web-search provider, translating Magnitude’s standard call() interface into EXA-specific API requests for search-augmented generation.
• src/custom-endpoint/provider.ts – Enables integration of arbitrary HTTP-compatible LLM endpoints by allowing users to specify custom URLs, headers, and request formatting while still conforming to the universal Provider contract.

Each file exports a class or factory function that returns a complete Provider implementation, satisfying the listModels, getModel, and call requirements.

Catalog Aggregation: Unified Model Discovery

Because users can enable multiple providers simultaneously, Magnitude must present a unified view of available models. The CatalogAggregator in packages/providers/src/catalog-aggregator.ts iterates over all registered providers, invokes their listModels() methods, and concatenates the results.

To maintain global uniqueness, the aggregator prefixes each model ID with its provider identifier (e.g., openai/gpt-4o, anthropic/claude-3). This allows UI components and SDK methods to reference specific models unambiguously while the underlying registry handles provider resolution transparently.

Runtime Execution Flow

The AI provider abstraction operates through a five-stage pipeline:

1. Startup Initialization – The ProviderClient establishes the RPC connection to the Magnitude daemon and instantiates concrete provider classes.
2. Registration – Each provider implementation registers itself with the ProviderRegistryService via the register() method.
3. Catalog Query – When the UI renders the model selector, it calls ProviderRegistryService.listAll() (via the aggregator) to retrieve the merged catalog of all available models.
4. Model Invocation – Upon user selection, the system retrieves the correct provider using ProviderRegistryService.get(providerId) and forwards standardized ProviderCallOptions to the provider’s call() method. The provider translates these options into vendor-specific request formats (JSON for OpenAI, multipart for certain custom endpoints).
5. Response Normalization – The provider parses the raw API response into the unified ProviderResponse shape, enabling consistent handling of streaming data, token usage tracking, and error management across all back-ends.

Practical Integration Examples

Listing All Available Models

To retrieve the aggregated catalog of models from all registered providers:

import { ProviderRegistryService } from '@magnitudedev/providers';
import { Effect } from 'effect';

const listAllModels = ProviderRegistryService
  .listAll()
  .pipe(
    Effect.map(models => models.map(m => `${m.providerId}/${m.modelId}`)),
  );

Effect.runPromise(listAllModels).then(console.log);

This operation, defined in packages/providers/src/catalog-aggregator.ts, yields a flat list of globally unique model identifiers suitable for dropdown menus or CLI interfaces.

Invoking a Model

To execute a chat completion through any registered provider:

import { ProviderRegistryService } from '@magnitudedev/providers';
import { ProviderCallOptions } from '@magnitudedev/ai';
import { Effect } from 'effect';

const callOpts: ProviderCallOptions = {
  modelId: 'openai/gpt-4o',
  messages: [{ role: 'user', content: 'Explain quantum entanglement.' }],
  temperature: 0.7,
};

const invoke = ProviderRegistryService
  .get('openai')
  .pipe(
    Effect.flatMap(provider => provider.call(callOpts)),
    Effect.map(resp => resp.choices[0].message.content),
  );

Effect.runPromise(invoke).then(console.log);

The ProviderRegistryService.get() call resolves the correct implementation based on the provider prefix in the model ID, while the call() method handles all transport-specific logic internally.

Registering a Custom Endpoint

Adding a proprietary or self-hosted LLM requires only implementing the contract and registering it:

import { CustomEndpointProvider } from '@magnitudedev/providers/custom-endpoint';
import { ProviderRegistryService } from '@magnitudedev/providers';
import { Effect } from 'effect';

const myProvider = CustomEndpointProvider.make({
  id: 'my-llm',
  name: 'My LLM',
  endpoint: 'https://api.my-llm.com/v1/chat/completions',
  apiKey: process.env.MY_LLM_API_KEY!,
});

Effect.runPromise(ProviderRegistryService.register(myProvider));

Once registered, the custom endpoint appears in the aggregated catalog and responds to standard invocation calls without requiring changes to existing UI or business logic.

Summary

• Universal Contract – The Provider interface in packages/ai/src/provider/contract.ts defines the mandatory listModels, getModel, and call methods that normalize access to any AI service.
• Registry Pattern – The ProviderRegistryService (packages/providers/src/registry.ts) manages provider lifecycle as a singleton, enabling dependency injection throughout the application.
• Runtime Discovery – Concrete implementations for OpenAI, Anthropic, EXA, and custom endpoints live in packages/providers/src/ and auto-register during startup.
• Unified Catalog – The CatalogAggregator (packages/providers/src/catalog-aggregator.ts) merges disparate provider model lists into a single, prefixed namespace.
• Effect-Based Architecture – All provider operations return Effect.Effect types, ensuring composable error handling and cancellation across asynchronous model invocations.

Frequently Asked Questions

What is the Provider interface in Magnitude?

The Provider interface is a TypeScript contract defined in packages/ai/src/provider/contract.ts that requires every AI service to implement three core methods: listModels() for catalog discovery, getModel() for metadata retrieval, and call() for model invocation. This interface uses Effect-TS types to handle asynchronous operations and errors functionally, ensuring that all providers—whether OpenAI, Anthropic, or custom HTTP endpoints—present an identical API to the rest of the Magnitude system.

How does Magnitude handle models from multiple providers simultaneously?

Magnitude uses the CatalogAggregator located in packages/providers/src/catalog-aggregator.ts to query all registered providers via their listModels() methods and concatenate the results. The aggregator prefixes each model ID with its provider identifier (e.g., openai/gpt-4o, anthropic/claude-3), creating a unified namespace that allows the UI and SDK to display and select from heterogeneous model catalogs as if they were a single collection.

Can I add a custom LLM endpoint to Magnitude without modifying core code?

Yes. You can integrate any HTTP-compatible LLM by using the CustomEndpointProvider in packages/providers/src/custom-endpoint/provider.ts. Simply instantiate the provider with your endpoint URL, authentication headers, and configuration, then register it with ProviderRegistryService.register(). Because the custom implementation conforms to the standard Provider interface, it immediately participates in the catalog aggregation and invocation flow without requiring changes to existing UI components or business logic.

What role does Effect play in the provider abstraction?

Effect (from the Effect-TS library) provides the functional programming foundation for Magnitude’s provider layer. Every method in the Provider interface returns an Effect.Effect type rather than raw Promises, enabling sophisticated error handling, request cancellation, resource management, and composable workflows. This ensures that provider implementations can handle failures gracefully—such as API timeouts or rate limiting—while maintaining type safety across the asynchronous boundary between the SDK and model services.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}