# How LLM Providers Are Managed in ai-memory: A Complete Technical Guide

> Discover how ai-memory manages LLM providers with a technical deep dive. Explore the LlmProvider trait, factory pattern, and eight implementations like OpenAI and Gemini.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: deep-dive
- Published: 2026-09-05

---

**LLM providers in ai-memory are abstracted behind the `LlmProvider` trait, with eight concrete implementations (OpenAI, Anthropic, Google, Gemini, Copilot, Local, OpenAI OAuth, and OIDC) instantiated through a factory pattern that reads from Figment configuration.**

The **ai-memory** crate orchestrates multiple Large Language Model backends through a unified, async-compatible interface. At the core of this system lies the `LlmProvider` trait defined in [`crates/ai-memory-llm/src/provider.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-llm/src/provider.rs), which decouples the rest of the codebase from vendor-specific HTTP implementations and authentication mechanisms.

## The LlmProvider Trait: Core Abstraction

The `LlmProvider` trait defines a minimal, provider-agnostic API that all LLM backends must implement. This design enables **type-erased, thread-safe provider storage** via `Arc<dyn LlmProvider>` across the MCP server and hook router.

The trait specifies six essential methods:

- **`name()`** – Returns a human-readable identifier (e.g., `"openai"`, `"anthropic"`)
- **`model()`** – Provides the model name used in HTTP requests
- **`complete()`** – Executes plain-text chat completion
- **`complete_with_operation_id()`** – Completion tied to a logical operation ID (default fallback to `complete`)
- **`complete_structured_raw()`** – JSON-schema-constrained completion returning `serde_json::Value`
- **`complete_structured_raw_with_operation_id()`** – Operation-aware structured completion (default forwards to raw method)

The trait carries **`Send + Sync`** bounds, allowing safe concurrent access from any Tokio task without cloning connection state.

## Supported LLM Provider Implementations

ai-memory ships eight concrete implementations, each isolated to its own source file under `crates/ai-memory-llm/src/`:

| Provider | Source File | Key Implementation Details |
|----------|-------------|---------------------------|
| **OpenAI** | [`openai.rs`](https://github.com/akitaonrails/ai-memory/blob/main/openai.rs) | `reqwest` client with `rustls-tls-native-roots`; operation IDs mapped to OpenAI's `assistant_id` field |
| **Anthropic** | [`anthropic.rs`](https://github.com/akitaonrails/ai-memory/blob/main/anthropic.rs) | Messages API endpoint with streaming support and `completion_id` metadata handling |
| **Google Vertex AI** | [`google.rs`](https://github.com/akitaonrails/ai-memory/blob/main/google.rs) | Service-account authentication; model name mapping to Vertex AI identifiers |
| **Gemini** | [`gemini.rs`](https://github.com/akitaonrails/ai-memory/blob/main/gemini.rs) | Native Gemini request format with specialized schema handling |
| **GitHub Copilot** | [`copilot.rs`](https://github.com/akitaonrails/ai-memory/blob/main/copilot.rs) | Copilot completions endpoint supporting both plain-text and structured outputs |
| **Local (in-process)** | [`local.rs`](https://github.com/akitaonrails/ai-memory/blob/main/local.rs) | Pure-Rust embeddings via `candle` crate for offline/test environments |
| **OpenAI OAuth** | [`openai_oauth.rs`](https://github.com/akitaonrails/ai-memory/blob/main/openai_oauth.rs) | Token refresh logic layered over base OpenAI provider |
| **OIDC** | [`oidc.rs`](https://github.com/akitaonrails/ai-memory/blob/main/oidc.rs) | Generic OpenID Connect token handling for OIDC-capable providers |

Each implementation handles its own authentication, request serialization, error mapping, and response parsing while exposing the same uniform interface.

## Provider Factory Pattern and Configuration

Provider instantiation is centralized through **`LlmProviderFactory`** in [`crates/ai-memory-llm/src/factory.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-llm/src/factory.rs). The factory consumes **Figment** configuration to determine which concrete provider to build, injects appropriate credentials, and returns a boxed `dyn LlmProvider`.

Standard construction pattern:

```rust
let cfg = figment::Figment::new()
    .merge(figment::providers::Toml::file("ai-memory.toml"));

let provider = ai_memory_llm::factory::make_provider(&cfg)?; 
// Returns Box<dyn LlmProvider>

```

Configuration values—API keys, model names, endpoint URLs, OAuth scopes—are sourced from [`ai-memory.toml`](https://github.com/akitaonrails/ai-memory/blob/main/ai-memory.toml) and processed through the core `Config` struct in [`crates/ai-memory-core/src/config.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/config.rs). This centralization ensures credential isolation and environment-specific overrides without code changes.

## Typed Helper Functions for Structured Output

Beyond raw trait methods, ai-memory provides high-level helpers in [`crates/ai-memory-llm/src/provider.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-llm/src/provider.rs) (lines 70-106) that automate JSON schema generation and deserialization:

```rust
use ai_memory_llm::{provider::LlmProvider, complete_structured};
use ai_memory_llm::types::{ChatRequest, ChatResponse};

let request = ChatRequest::new("Explain quantum tunneling");
let response: MyResponse = complete_structured(&*provider, request).await?;

```

These helpers leverage **`schemars`** for compile-time schema derivation, eliminating manual JSON Schema construction while maintaining full type safety. The `complete_structured` and `complete_structured_with_operation_id` functions wrap the raw trait methods, bridging the gap between Rust's type system and LLM JSON outputs.

## Thread Safety and Concurrency Model

The `Send + Sync` trait bounds enable **shared provider ownership** across async boundaries. A single `Arc<dyn LlmProvider>` can be:

- Stored in the MCP server state
- Passed to hook routers
- Cloned into spawned Tokio tasks

This design avoids connection-per-request overhead while maintaining safety guarantees. Providers internally manage their own connection pooling and rate limiting where appropriate.

## Summary

- **Trait abstraction**: `LlmProvider` isolates vendor-specific HTTP logic behind six core methods
- **Eight implementations**: OpenAI, Anthropic, Google, Gemini, Copilot, Local, OpenAI OAuth, and OIDC each live in dedicated source files
- **Factory instantiation**: `LlmProviderFactory` reads Figment configuration to build the correct provider at runtime
- **Type-safe helpers**: `complete_structured` functions automate JSON schema handling via `schemars`
- **Thread-safe design**: `Send + Sync` bounds permit `Arc<dyn LlmProvider>` sharing across Tokio tasks

## Frequently Asked Questions

### How do I add a new LLM provider to ai-memory?

Implement the `LlmProvider` trait for your provider struct in a new file under `crates/ai-memory-llm/src/`, then register it in the factory at [`crates/ai-memory-llm/src/factory.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-llm/src/factory.rs). Your implementation must satisfy `Send + Sync` and handle its own HTTP client configuration and authentication.

### Can I use multiple LLM providers simultaneously in the same ai-memory instance?

Yes. Because providers are trait objects, you can instantiate multiple `Box<dyn LlmProvider>` or `Arc<dyn LlmProvider>` values from the factory and route requests to different providers based on operation type, cost optimization, or fallback logic.

### Where are API credentials stored in ai-memory?

Credentials reside in the Figment-managed configuration file ([`ai-memory.toml`](https://github.com/akitaonrails/ai-memory/blob/main/ai-memory.toml)). The `Config` struct in [`crates/ai-memory-core/src/config.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/config.rs) parses these values and passes them to the factory, keeping secrets out of source code and enabling environment-specific configuration files.

### What is the purpose of operation IDs in LLM completions?

Operation IDs enable **request tracing and idempotency** across distributed systems. The `complete_with_operation_id` and `complete_structured_raw_with_operation_id` methods allow callers to associate a logical operation identifier with a completion request, which providers like OpenAI map to native fields (`assistant_id`) for tracking and potential retry semantics.