# How to Set Up Craft Agents with Multiple LLM Providers

> Learn how to set up Craft Agents with multiple LLM providers. Easily connect OpenAI, Anthropic, Ollama, and custom endpoints for flexible AI agent development.

- Repository: [Craft Ai Agents/craft-agents-oss](https://github.com/craft-ai-agents/craft-agents-oss)
- Tags: how-to-guide
- Published: 2026-07-04

---

**Craft Agents supports unlimited LLM providers through a provider-centric configuration system stored in [`packages/shared/src/config/llm-connections.ts`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/packages/shared/src/config/llm-connections.ts), allowing you to mix Anthropic, OpenAI, Ollama, and custom endpoints with automatic runtime resolution.**

The craft-ai-agents/craft-agents-oss repository provides a flexible architecture for managing multiple LLM connections simultaneously. Whether you need to route different sessions to Claude, GPT-4, or local Ollama instances, the configuration system handles distinct authentication methods, endpoints, and model defaults through a unified JSON-based setup. This guide walks through the core types, storage mechanisms, and UI flows required to configure and switch between providers at runtime.

## Understanding the LLM Connections Architecture

The entire multi-provider system centers on the **LLM Connections** subsystem located in [`packages/shared/src/config/llm-connections.ts`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/packages/shared/src/config/llm-connections.ts). This file defines the type safety, validation logic, and resolution algorithms that determine which provider handles each request.

### Core Types and Configuration

Three enums define the provider landscape:

- **`LlmProviderType`** (lines 52-56): Determines which SDK to invoke. Valid values are `anthropic` (direct Anthropic SDK), `pi` (unified Predictive Intelligence API), or `pi_compat` (OpenAI-compatible endpoints).
- **`LlmAuthType`** (lines 82-90): Selects credential handling strategies including `api_key`, `oauth`, `api_key_with_endpoint`, and `none`.
- **`LlmConnection`** (lines 135-170): The serializable record stored in [`config.json`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/config.json) containing `slug`, `name`, `providerType`, `authType`, `baseUrl`, `models` array, and `defaultModel`.

The configuration also supports **model specialization** through `getMiniModel` and `getSummarizationModel` (lines 61-86), which select appropriate "small" models for specific agent tasks, and `setModelSupportsImages` (lines 514-531) to toggle vision capabilities per model.

### Connection Resolution Hierarchy

When a session starts, the engine calls `resolveEffectiveConnectionSlug` (lines 684-693) to determine the active provider. The function evaluates the following precedence chain:

1. **Explicit session lock** – the `llmConnection` field on the session object
2. **Workspace default** – the `defaultLlmConnection` setting in workspace configuration
3. **Global default** – the connection flagged `isDefault: true` in the config
4. **First available** – fallback to the first connection in the list

If the chosen connection no longer exists, `isSessionConnectionUnavailable` (lines 704-711) marks the session as unavailable, prompting the UI to request a new provider selection.

## Configuring Multiple LLM Providers

### Adding a New Provider Connection

Each provider requires a JSON-serializable `LlmConnection` object. For example, to add a custom OpenAI-compatible endpoint:

```json
{
  "slug": "custom-openai",
  "name": "Custom OpenAI Endpoint",
  "providerType": "pi_compat",
  "authType": "api_key",
  "baseUrl": "https://api.custom-endpoint.com/v1",
  "models": ["gpt-4", "gpt-3.5-turbo"],
  "defaultModel": "gpt-4",
  "createdAt": 1720123456789
}

```

The Electron app persists this via the `saveLlmConnection` RPC handler defined in [`server-core/src/handlers/rpc/llm-connections.ts`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/server-core/src/handlers/rpc/llm-connections.ts). Under the hood, the system writes to [`config.json`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/config.json) through `saveConfig` (lines 14-28 in [`packages/shared/src/config/storage.ts`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/packages/shared/src/config/storage.ts)).

### Authentication and Credential Storage

Credentials are stored separately from the connection configuration in an encrypted credential store. The system generates storage keys using `getLlmCredentialKey` (lines 61-66 in [`llm-connections.ts`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/llm-connections.ts)). For API-key based authentication, the UI retrieves the actual key via `credentialManager.getLlmApiKey(connectionSlug)` before making requests.

### Setting Default Models and Capabilities

For `pi_compat` connections (like Ollama), you must explicitly define the `models` array since the system cannot auto-discover available models. You can also configure image support using `setModelSupportsImages` (lines 514-531), which updates the model metadata to indicate vision capabilities.

## Runtime Connection Resolution

When initializing a chat session, the backend executes:

```typescript
const effectiveSlug = resolveEffectiveConnectionSlug(
  session.llmConnection,
  workspace.defaults.defaultLlmConnection,
  connectionsWithStatus
);

```

This function (lines 684-693) returns the first non-null value from the precedence list. The resulting `effectiveSlug` determines which SDK and endpoint the `call_llm` tool uses for that session. If you delete a provider that an active session relies on, the `isSessionConnectionUnavailable` checker (lines 704-711) traps the error and surfaces it to the UI.

## Practical Implementation Examples

### Example 1: Programmatic Setup for Claude, OpenAI, and Ollama

Use the storage helpers to bootstrap multiple providers at startup:

```typescript
// scripts/bootstrap-providers.ts
import { addLlmConnection, setDefaultLlmConnection } from '@config/storage';

async function configureProviders() {
  // Claude via direct Anthropic SDK
  await addLlmConnection({
    slug: 'anthropic',
    name: 'Claude (Anthropic)',
    providerType: 'anthropic',
    authType: 'api_key',
    defaultModel: 'claude-3-5-sonnet-20241022',
    createdAt: Date.now(),
  });

  // OpenAI via Pi unified API
  await addLlmConnection({
    slug: 'openai',
    name: 'OpenAI (ChatGPT)',
    providerType: 'pi',
    authType: 'api_key',
    defaultModel: 'gpt-4o',
    createdAt: Date.now(),
  });

  // Local Ollama (no auth, custom endpoint)
  await addLlmConnection({
    slug: 'ollama',
    name: 'Ollama (Local)',
    providerType: 'pi_compat',
    authType: 'none',
    baseUrl: 'http://127.0.0.1:11434/v1',
    models: ['phi3', 'llama2', 'mistral'],
    defaultModel: 'phi3',
    createdAt: Date.now(),
  });

  // Set Claude as global default
  await setDefaultLlmConnection('anthropic');
}

configureProviders().catch(console.error);

```

### Example 2: React Component for Adding Connections

The settings UI in [`AiSettingsPage.tsx`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/AiSettingsPage.tsx) implements connection creation like this:

```tsx
// Component pattern from AiSettingsPage.tsx
import { useState } from 'react';

function AddConnectionForm() {
  const [name, setName] = useState('');
  const [provider, setProvider] = useState<'anthropic' | 'pi' | 'pi_compat'>('pi');
  const [authType, setAuthType] = useState<'api_key' | 'none'>('api_key');

  const handleSave = async () => {
    const connection = {
      slug: name.toLowerCase().replace(/\s+/g, '-'),
      name,
      providerType: provider,
      authType: authType,
      baseUrl: provider === 'pi_compat' ? 'http://localhost:11434/v1' : undefined,
      createdAt: Date.now(),
    };
    
    await window.electronAPI.saveLlmConnection(connection);
    await window.electronAPI.refreshLlmConnections();
  };

  return (
    <form onSubmit={handleSave}>
      {/* Form fields for name, provider selection, etc. */}
      <button type="submit">Add Provider</button>
    </form>
  );
}

```

### Example 3: Backend Resolution for Agent Sessions

When an agent needs to determine which LLM to use:

```typescript
import { resolveEffectiveConnectionSlug } from '@config/llm-connections';
import { getLlmConnection, getLlmConnections } from '@config/storage';

async function getSessionLlm(session) {
  const connections = await getLlmConnections();
  const effectiveSlug = resolveEffectiveConnectionSlug(
    session.llmConnection,
    session.workspaceDefaultLlmConnection,
    connections.map(c => ({ slug: c.slug, isDefault: c.isDefault }))
  );
  
  return getLlmConnection(effectiveSlug);
}

```

## Managing Providers in the UI

The **Settings → AI → Connections** page ([`AiSettingsPage.tsx`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/AiSettingsPage.tsx)) provides the visual interface for managing multiple providers. The "Default LLM connection" dropdown (lines 403-410) allows setting workspace-specific defaults, while the star button (line 908) triggers `setDefaultLlmConnection` to set the global default.

The main app bootstrap in [`App.tsx`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/App.tsx) (lines 754-758) subscribes to `onLlmConnectionsChanged` events, ensuring that changes propagate instantly to all open chat sessions without requiring a restart.

## Testing Your Configuration

Before deploying agents, verify your setup:

1. **List connections**: Call `window.electronAPI.listLlmConnectionsWithStatus()` to retrieve all configured providers with their `slug`, `providerType`, and `isDefault` status.
2. **Test connectivity**: Invoke `window.electronAPI.testLlmConnection(slug)`, which executes a cheap API call (like `GET /v1/models` for OpenAI-compatible endpoints) via the RPC handler in [`server-core/src/handlers/rpc/llm-connections.ts`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/server-core/src/handlers/rpc/llm-connections.ts) (line 68).
3. **Verify credentials**: Ensure `credentialManager.getLlmApiKey(slug)` returns valid credentials for the chosen authentication type.

## Summary

- **Core configuration** lives in [`packages/shared/src/config/llm-connections.ts`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/packages/shared/src/config/llm-connections.ts), defining `LlmProviderType`, `LlmAuthType`, and `LlmConnection` interfaces.
- **Resolution logic** follows a strict hierarchy through `resolveEffectiveConnectionSlug` (lines 684-693): session lock → workspace default → global default → first available.
- **Persistence** uses `saveLlmConnection` and `addLlmConnection` to write to [`config.json`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/config.json), while credentials are stored separately via `getLlmCredentialKey`.
- **UI synchronization** happens through `onLlmConnectionsChanged` events (lines 754-758 in [`App.tsx`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/App.tsx)), ensuring real-time updates across all windows.
- **Custom endpoints** require `pi_compat` provider type with explicit `baseUrl` and `models` array, plus `setModelSupportsImages` for vision capabilities.

## Frequently Asked Questions

### Can I use local LLMs like Ollama alongside cloud providers?

Yes. Configure Ollama as a `pi_compat` provider with `authType: "none"` and `baseUrl: "http://127.0.0.1:11434/v1"`. You must explicitly define the `models` array (e.g., `["phi3", "llama2"]`) since local endpoints cannot auto-populate model lists. This setup runs alongside Anthropic or OpenAI connections in the same workspace.

### How does Craft Agents handle authentication for multiple providers?

Each connection stores its authentication type (`api_key`, `oauth`, or `none`) in the `LlmConnection` record. The actual credentials are stored separately in an encrypted credential store using keys generated by `getLlmCredentialKey` (lines 61-66). When a session starts, the system retrieves the appropriate key based on the resolved connection slug.

### What happens if a configured provider becomes unavailable?

The `isSessionConnectionUnavailable` function (lines 704-711) checks if the session's locked connection still exists in the configuration. If the provider was deleted or the credentials expired, the function returns `true`, and the UI prompts the user to select a new provider from the dropdown. The session retains its other state (history, files) but pauses LLM calls until a valid connection is selected.

### How do I set different default models for different workspaces?

Set the `defaultLlmConnection` field in the workspace settings JSON. The resolution chain in `resolveEffectiveConnectionSlug` checks this workspace-specific value before falling back to the global default. In the UI, the dropdown in [`AiSettingsPage.tsx`](https://github.com/craft-ai-agents/craft-agents-oss/blob/main/AiSettingsPage.tsx) (line 403) allows per-workspace overrides, while the global default is managed via the `setDefaultLlmConnection` RPC (line 908).