# Configuring Multiple Model Providers with Provider-Specific Settings in Flue

> Configure multiple LLM model providers with Flue's registerProvider and configureProvider. Route traffic to local Ollama, custom gateways, or Cloudflare Workers AI without rebuilding.

- Repository: [Astro/flue](https://github.com/withastro/flue)
- Tags: how-to-guide
- Published: 2026-05-11

---

**Flue lets you register unlimited LLM endpoints and patch their transport settings at runtime using `registerProvider` and `configureProvider`, enabling you to route traffic to local Ollama instances, custom gateways, or Cloudflare Workers AI without rebuilding your agents.**

The `withastro/flue` SDK decouples your agent code from specific LLM backends through a flexible provider registry. When you need to switch between OpenAI, Anthropic, and local models—or route traffic through corporate proxies—you can configure multiple model providers with provider‑specific settings using a URL‑prefix based registration system that persists at runtime.

## Understanding Flue's Provider Registry

Flue maintains two critical data structures in [`packages/sdk/src/runtime/providers.ts`](https://github.com/withastro/flue/blob/main/packages/sdk/src/runtime/providers.ts): the **userModels** registry that maps URL prefixes to provider implementations, and the **providerOverrides** map that stores transport‑level patches. According to the source code, when you call `registerProvider(name, registration)`, the system stores your configuration in `userModels` using a last‑write‑wins strategy (lines 89‑94).

The `configureProvider` function then writes to `providerOverrides` (lines 138‑146), allowing you to modify settings like `baseUrl` or `apiKey` without touching the original registration. When resolving models, Flue merges these overrides via `getProviderConfiguration` (lines 71‑75) before passing the final configuration to the runtime harness.

### Registration Object Structure

For HTTP‑based providers, registrations require the `api` field (a pi‑ai API slug like `"openai-completions"`), a `baseUrl` string, and optional `apiKey`, `headers`, and `provider` override fields.

```typescript
{
  api: "openai-completions",
  baseUrl: "https://api.anthropic.com/v1",
  apiKey?: string,
  headers?: Record<string, string>,
  provider?: string
}

```

Cloudflare Workers AI deployments use a separate `CloudflareAIBindingRegistration` interface that accepts a Workers AI binding instead of HTTP credentials.

## Registering Custom Model Providers

To add a new provider, import `registerProvider` from `@flue/sdk/app` and specify the transport details. The registration creates a URL prefix that agents reference using the `"prefix/modelId"` syntax.

The following example from [`examples/hello-world/.flue/app.ts`](https://github.com/withastro/flue/blob/main/examples/hello-world/.flue/app.ts) registers two local OpenAI‑compatible servers:

```typescript
import { registerProvider } from '@flue/sdk/app';

// Register Ollama endpoint
registerProvider('ollama', {
  api: 'openai-completions',
  baseUrl: 'http://localhost:11434/v1',
});

// Register LM Studio endpoint
registerProvider('lmstudio', {
  api: 'openai-completions',
  baseUrl: 'http://localhost:1234/v1',
});

```

Each registration creates a unique namespace. When an agent requests `ollama/llama3.2`, Flue resolves the prefix to the Ollama registration, constructs the appropriate pi‑ai `Model` object, and routes the request to `http://localhost:11434/v1`.

## Patching Provider Settings with configureProvider

The `configureProvider(slug, settings)` function enables runtime modifications to any registered provider, including built‑ins like Anthropic. The system uses the **provider slug**—the resolved `Model.provider` value—to look up overrides and merge them into the final configuration.

```typescript
import { configureProvider } from '@flue/sdk/app';

// Route Anthropic traffic through a corporate gateway
if (process.env.ANTHROPIC_GATEWAY_URL) {
  configureProvider('anthropic', {
    baseUrl: process.env.ANTHROPIC_GATEWAY_URL,
    apiKey: process.env.ANTHROPIC_API_KEY,
    headers: {
      'X-Custom-Auth': process.env.INTERNAL_AUTH_TOKEN
    }
  });
}

```

This patching occurs at runtime without requiring a rebuild. The overrides are stored separately from the base registration, allowing you to modify transport details while preserving the catalog metadata that pi‑ai supplies.

## Referencing Models in Agent Code

Once configured, models become immediately available using the prefix syntax. The runtime resolves these references through `resolveRegisteredModel` → `buildModelFromRegistration`, applying any configured overrides before executing the API call.

```typescript
// Uses the patched Anthropic gateway
await ctx.session.prompt({
  model: 'anthropic/claude-3-5-sonnet-20241022',
});

// Routes to local Ollama instance
await ctx.session.prompt({
  model: 'ollama/llama3.2',
});

// Routes to LM Studio
await ctx.session.prompt({
  model: 'lmstudio/mistral',
});

```

## Implementing Custom API Handlers

For providers not supported by pi‑ai natively, use `registerApiProvider` to supply your own streaming implementations, then alias them with `registerProvider`. The API registration must implement `stream` and `streamSimple` methods that conform to the pi‑ai transport interface.

```typescript
import { registerApiProvider, registerProvider } from '@flue/sdk/app';
import { myCustomStream, myCustomStreamSimple } from './my-api-implementation';

// 1. Register the low-level API implementation
registerApiProvider({
  api: 'my-custom-api',
  stream: myCustomStream,
  streamSimple: myCustomStreamSimple,
});

// 2. Create a friendly URL prefix
registerProvider('custom', {
  api: 'my-custom-api',
  baseUrl: 'https://api.custom-service.com/v1',
  apiKey: process.env.CUSTOM_API_KEY,
});

```

Now agents can request `custom/my-model` and Flue will route the call through your custom stream handlers instead of the default HTTP transport.

## Summary

- **`registerProvider`** creates URL‑prefix mappings in `userModels` that link short names to HTTP endpoints or Cloudflare Workers AI bindings.
- **`configureProvider`** patches transport settings (`baseUrl`, `apiKey`, `headers`) at runtime via the `providerOverrides` map without losing catalog metadata.
- **Provider slugs** are the resolved `Model.provider` values used for override lookups, distinct from the URL prefixes used for registration.
- **Custom APIs** require `registerApiProvider` to define stream implementations before aliasing them with `registerProvider`.
- All configuration happens in [`packages/sdk/src/runtime/providers.ts`](https://github.com/withastro/flue/blob/main/packages/sdk/src/runtime/providers.ts) and is exposed through [`packages/sdk/src/app.ts`](https://github.com/withastro/flue/blob/main/packages/sdk/src/app.ts) for application code.

## Frequently Asked Questions

### What is the difference between `registerProvider` and `configureProvider` in Flue?

`registerProvider` establishes the initial URL‑prefix mapping and specifies the pi‑ai API type (e.g., `"openai-completions"`) and endpoint location, storing data in the `userModels` registry. `configureProvider` modifies existing registrations (including built‑ins) by writing to `providerOverrides`, allowing you to change transport details like authentication headers or base URLs at runtime without redefining the provider's core metadata.

### How do I route a built-in provider like Anthropic through a corporate proxy?

Use `configureProvider('anthropic', { baseUrl: 'https://gateway.company.com', ... })` after importing the function from `@flue/sdk/app`. The system looks up overrides via `getProviderConfiguration` before each request, ensuring your gateway receives the traffic while the agent code continues using the standard `anthropic/model-id` syntax.

### Can I register multiple local OpenAI-compatible servers simultaneously?

Yes. Register each local endpoint with a unique prefix in your [`app.ts`](https://github.com/withastro/flue/blob/main/app.ts) file. For example, `registerProvider('ollama', { baseUrl: 'http://localhost:11434/v1', ... })` and `registerProvider('lmstudio', { baseUrl: 'http://localhost:1234/v1', ... })` allow agents to reference `ollama/llama3.2` and `lmstudio/mistral` concurrently, with each request routed to the appropriate localhost port.

### Where does Flue store provider configurations at runtime?

Flue maintains provider data in two maps defined in [`packages/sdk/src/runtime/providers.ts`](https://github.com/withastro/flue/blob/main/packages/sdk/src/runtime/providers.ts): `userModels` stores the base registrations from `registerProvider`, while `providerOverrides` stores patches from `configureProvider`. When resolving a model, the runtime merges these configurations via `buildModelFromRegistration` before executing the API call.