How to Set Up Craft Agents with Multiple LLM Providers

Craft Agents supports unlimited LLM providers through a provider-centric configuration system stored in packages/shared/src/config/llm-connections.ts, allowing you to mix Anthropic, OpenAI, Ollama, and custom endpoints with automatic runtime resolution.

The craft-ai-agents/craft-agents-oss repository provides a flexible architecture for managing multiple LLM connections simultaneously. Whether you need to route different sessions to Claude, GPT-4, or local Ollama instances, the configuration system handles distinct authentication methods, endpoints, and model defaults through a unified JSON-based setup. This guide walks through the core types, storage mechanisms, and UI flows required to configure and switch between providers at runtime.

Understanding the LLM Connections Architecture

The entire multi-provider system centers on the LLM Connections subsystem located in packages/shared/src/config/llm-connections.ts. This file defines the type safety, validation logic, and resolution algorithms that determine which provider handles each request.

Core Types and Configuration

Three enums define the provider landscape:

• LlmProviderType (lines 52-56): Determines which SDK to invoke. Valid values are anthropic (direct Anthropic SDK), pi (unified Predictive Intelligence API), or pi_compat (OpenAI-compatible endpoints).
• LlmAuthType (lines 82-90): Selects credential handling strategies including api_key, oauth, api_key_with_endpoint, and none.
• LlmConnection (lines 135-170): The serializable record stored in config.json containing slug, name, providerType, authType, baseUrl, models array, and defaultModel.

The configuration also supports model specialization through getMiniModel and getSummarizationModel (lines 61-86), which select appropriate "small" models for specific agent tasks, and setModelSupportsImages (lines 514-531) to toggle vision capabilities per model.

Connection Resolution Hierarchy

When a session starts, the engine calls resolveEffectiveConnectionSlug (lines 684-693) to determine the active provider. The function evaluates the following precedence chain:

1. Explicit session lock – the llmConnection field on the session object
2. Workspace default – the defaultLlmConnection setting in workspace configuration
3. Global default – the connection flagged isDefault: true in the config
4. First available – fallback to the first connection in the list

If the chosen connection no longer exists, isSessionConnectionUnavailable (lines 704-711) marks the session as unavailable, prompting the UI to request a new provider selection.

Configuring Multiple LLM Providers

Adding a New Provider Connection

Each provider requires a JSON-serializable LlmConnection object. For example, to add a custom OpenAI-compatible endpoint:

{
  "slug": "custom-openai",
  "name": "Custom OpenAI Endpoint",
  "providerType": "pi_compat",
  "authType": "api_key",
  "baseUrl": "https://api.custom-endpoint.com/v1",
  "models": ["gpt-4", "gpt-3.5-turbo"],
  "defaultModel": "gpt-4",
  "createdAt": 1720123456789
}

The Electron app persists this via the saveLlmConnection RPC handler defined in server-core/src/handlers/rpc/llm-connections.ts. Under the hood, the system writes to config.json through saveConfig (lines 14-28 in packages/shared/src/config/storage.ts).

Authentication and Credential Storage

Credentials are stored separately from the connection configuration in an encrypted credential store. The system generates storage keys using getLlmCredentialKey (lines 61-66 in llm-connections.ts). For API-key based authentication, the UI retrieves the actual key via credentialManager.getLlmApiKey(connectionSlug) before making requests.

Setting Default Models and Capabilities

For pi_compat connections (like Ollama), you must explicitly define the models array since the system cannot auto-discover available models. You can also configure image support using setModelSupportsImages (lines 514-531), which updates the model metadata to indicate vision capabilities.

Runtime Connection Resolution

When initializing a chat session, the backend executes:

const effectiveSlug = resolveEffectiveConnectionSlug(
  session.llmConnection,
  workspace.defaults.defaultLlmConnection,
  connectionsWithStatus
);

This function (lines 684-693) returns the first non-null value from the precedence list. The resulting effectiveSlug determines which SDK and endpoint the call_llm tool uses for that session. If you delete a provider that an active session relies on, the isSessionConnectionUnavailable checker (lines 704-711) traps the error and surfaces it to the UI.

Practical Implementation Examples

Example 1: Programmatic Setup for Claude, OpenAI, and Ollama

Use the storage helpers to bootstrap multiple providers at startup:

// scripts/bootstrap-providers.ts
import { addLlmConnection, setDefaultLlmConnection } from '@config/storage';

async function configureProviders() {
  // Claude via direct Anthropic SDK
  await addLlmConnection({
    slug: 'anthropic',
    name: 'Claude (Anthropic)',
    providerType: 'anthropic',
    authType: 'api_key',
    defaultModel: 'claude-3-5-sonnet-20241022',
    createdAt: Date.now(),
  });

  // OpenAI via Pi unified API
  await addLlmConnection({
    slug: 'openai',
    name: 'OpenAI (ChatGPT)',
    providerType: 'pi',
    authType: 'api_key',
    defaultModel: 'gpt-4o',
    createdAt: Date.now(),
  });

  // Local Ollama (no auth, custom endpoint)
  await addLlmConnection({
    slug: 'ollama',
    name: 'Ollama (Local)',
    providerType: 'pi_compat',
    authType: 'none',
    baseUrl: 'http://127.0.0.1:11434/v1',
    models: ['phi3', 'llama2', 'mistral'],
    defaultModel: 'phi3',
    createdAt: Date.now(),
  });

  // Set Claude as global default
  await setDefaultLlmConnection('anthropic');
}

configureProviders().catch(console.error);

Example 2: React Component for Adding Connections

The settings UI in AiSettingsPage.tsx implements connection creation like this:

// Component pattern from AiSettingsPage.tsx
import { useState } from 'react';

function AddConnectionForm() {
  const [name, setName] = useState('');
  const [provider, setProvider] = useState<'anthropic' | 'pi' | 'pi_compat'>('pi');
  const [authType, setAuthType] = useState<'api_key' | 'none'>('api_key');

  const handleSave = async () => {
    const connection = {
      slug: name.toLowerCase().replace(/\s+/g, '-'),
      name,
      providerType: provider,
      authType: authType,
      baseUrl: provider === 'pi_compat' ? 'http://localhost:11434/v1' : undefined,
      createdAt: Date.now(),
    };
    
    await window.electronAPI.saveLlmConnection(connection);
    await window.electronAPI.refreshLlmConnections();
  };

  return (
    <form onSubmit={handleSave}>
      {/* Form fields for name, provider selection, etc. */}
      <button type="submit">Add Provider</button>
    </form>
  );
}

Example 3: Backend Resolution for Agent Sessions

When an agent needs to determine which LLM to use:

import { resolveEffectiveConnectionSlug } from '@config/llm-connections';
import { getLlmConnection, getLlmConnections } from '@config/storage';

async function getSessionLlm(session) {
  const connections = await getLlmConnections();
  const effectiveSlug = resolveEffectiveConnectionSlug(
    session.llmConnection,
    session.workspaceDefaultLlmConnection,
    connections.map(c => ({ slug: c.slug, isDefault: c.isDefault }))
  );
  
  return getLlmConnection(effectiveSlug);
}

Managing Providers in the UI

The Settings → AI → Connections page (AiSettingsPage.tsx) provides the visual interface for managing multiple providers. The "Default LLM connection" dropdown (lines 403-410) allows setting workspace-specific defaults, while the star button (line 908) triggers setDefaultLlmConnection to set the global default.

The main app bootstrap in App.tsx (lines 754-758) subscribes to onLlmConnectionsChanged events, ensuring that changes propagate instantly to all open chat sessions without requiring a restart.

Testing Your Configuration

Before deploying agents, verify your setup:

1. List connections: Call window.electronAPI.listLlmConnectionsWithStatus() to retrieve all configured providers with their slug, providerType, and isDefault status.
2. Test connectivity: Invoke window.electronAPI.testLlmConnection(slug), which executes a cheap API call (like GET /v1/models for OpenAI-compatible endpoints) via the RPC handler in server-core/src/handlers/rpc/llm-connections.ts (line 68).
3. Verify credentials: Ensure credentialManager.getLlmApiKey(slug) returns valid credentials for the chosen authentication type.

Summary

• Core configuration lives in packages/shared/src/config/llm-connections.ts, defining LlmProviderType, LlmAuthType, and LlmConnection interfaces.
• Resolution logic follows a strict hierarchy through resolveEffectiveConnectionSlug (lines 684-693): session lock → workspace default → global default → first available.
• Persistence uses saveLlmConnection and addLlmConnection to write to config.json, while credentials are stored separately via getLlmCredentialKey.
• UI synchronization happens through onLlmConnectionsChanged events (lines 754-758 in App.tsx), ensuring real-time updates across all windows.
• Custom endpoints require pi_compat provider type with explicit baseUrl and models array, plus setModelSupportsImages for vision capabilities.

Frequently Asked Questions

Can I use local LLMs like Ollama alongside cloud providers?

Yes. Configure Ollama as a pi_compat provider with authType: "none" and baseUrl: "http://127.0.0.1:11434/v1". You must explicitly define the models array (e.g., ["phi3", "llama2"]) since local endpoints cannot auto-populate model lists. This setup runs alongside Anthropic or OpenAI connections in the same workspace.

How does Craft Agents handle authentication for multiple providers?

Each connection stores its authentication type (api_key, oauth, or none) in the LlmConnection record. The actual credentials are stored separately in an encrypted credential store using keys generated by getLlmCredentialKey (lines 61-66). When a session starts, the system retrieves the appropriate key based on the resolved connection slug.

What happens if a configured provider becomes unavailable?

The isSessionConnectionUnavailable function (lines 704-711) checks if the session's locked connection still exists in the configuration. If the provider was deleted or the credentials expired, the function returns true, and the UI prompts the user to select a new provider from the dropdown. The session retains its other state (history, files) but pauses LLM calls until a valid connection is selected.

How do I set different default models for different workspaces?

Set the defaultLlmConnection field in the workspace settings JSON. The resolution chain in resolveEffectiveConnectionSlug checks this workspace-specific value before falling back to the global default. In the UI, the dropdown in AiSettingsPage.tsx (line 403) allows per-workspace overrides, while the global default is managed via the setDefaultLlmConnection RPC (line 908).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}