How to Configure LLM Providers with Third-Party Endpoints (OpenRouter, Ollama) in Craft Agents OSS

Craft Agents OSS supports third-party LLM endpoints like OpenRouter and Ollama through the LLM Connection abstraction in packages/shared/src/config/llm-connections.ts, using providerType: "pi" with a custom baseUrl for OpenRouter and providerType: "pi_compat" with authType: "none" for local Ollama instances.

Craft Agents OSS abstracts every language model backend behind a configurable LLM Connection object. This architecture allows you to route agent requests to third-party endpoints such as OpenRouter or local Ollama servers without modifying core application code, as implemented in craft-ai-agents/craft-agents-oss.

Understanding the LLM Connection Architecture

Every LLM provider configuration centers on the LlmConnection interface defined in packages/shared/src/config/llm-connections.ts. This structure decouples provider selection, authentication, and endpoint configuration into four critical fields:

• providerType — Determines which SDK implementation to use. Options include anthropic (Claude SDK), pi (Pi unified AI SDK), and pi_compat (custom OpenAI-compatible endpoints).
• piAuthProvider — When using providerType: "pi", this specifies the underlying provider name (e.g., openai, anthropic, openrouter) as mapped in provider-metadata.ts.
• baseUrl — Optional custom endpoint URL. Required for pi_compat connections (e.g., Ollama) and optional for pi connections when routing through gateways like OpenRouter.
• authType — Credential strategy (api_key, api_key_with_endpoint, none). The runtime uses this field in resolveAuthEnvVars to determine which environment variables to inject.

Configuring OpenRouter as a Custom Endpoint

OpenRouter implements the OpenAI-compatible API, allowing the Pi SDK to communicate with it using the standard pi provider flow with endpoint overrides.

Set providerType to "pi" and piAuthProvider to "openrouter" (registered in packages/shared/src/config/provider-metadata.ts). Configure the baseUrl to point to https://openrouter.ai/api/v1 and set authType to "api_key". When the runtime initializes, resolveAuthEnvVars injects the stored API key into the appropriate environment variable for the Pi SDK to authenticate against OpenRouter.

{
  "slug": "openrouter",
  "name": "OpenRouter (ChatGPT-compatible)",
  "providerType": "pi",
  "piAuthProvider": "openrouter",
  "baseUrl": "https://openrouter.ai/api/v1",
  "authType": "api_key",
  "models": [],
  "defaultModel": "gpt-4o",
  "createdAt": 1720156800000
}

The Pi SDK dynamically populates the models array at runtime via the _piModelResolver registered in the application startup sequence.

Configuring Ollama for Local Development

Ollama runs a local HTTP server implementing the OpenAI-compatible completions API without requiring authentication. For this scenario, use providerType: "pi_compat" to bypass the Pi SDK's credential handling entirely.

Set authType to "none" and specify the local endpoint in baseUrl (typically http://localhost:11434 or http://127.0.0.1:11434). Because Ollama cannot enumerate available models via API, you must manually populate the models array with the specific models you have pulled locally.

{
  "slug": "ollama-local",
  "name": "Ollama (local)",
  "providerType": "pi_compat",
  "baseUrl": "http://127.0.0.1:11434",
  "authType": "none",
  "models": [
    { "id": "ollama/mistral", "name": "Mistral", "shortName": "Mistral" },
    { "id": "ollama/llama2", "name": "Llama 2", "shortName": "Llama2" }
  ],
  "defaultModel": "ollama/mistral",
  "createdAt": 1720156800000
}

The UI detects authType: "none" and hides the API key input field, streamlining the local development experience.

Runtime Authentication and Model Resolution

The connection logic differentiates runtime behavior based on provider type. For pi connections like OpenRouter, the system relies on the Pi SDK resolver (_piModelResolver) to fetch model lists dynamically. For pi_compat connections like Ollama, the system uses the manually specified models array because local endpoints cannot provide model catalogs.

When initializing a connection, the runtime calls resolveAuthEnvVars from packages/shared/src/config/llm-connections.ts to prepare the execution environment:

import { resolveAuthEnvVars } from '@/shared/config/llm-connections';

async function prepareEnv(connection, slug, credMgr) {
  const { envVars, success, warning } = await resolveAuthEnvVars(
    connection,
    slug,
    credMgr,
    async (s) => ({ accessToken: await getOAuthToken(s) })
  );
  if (!success) console.warn(warning);
  return envVars;
}

For OpenRouter, this resolves the API key and optionally sets ANTHROPIC_BASE_URL if a custom endpoint is present. For Ollama, the function returns empty credentials since authType is "none".

Summary

• LLM Connections in packages/shared/src/config/llm-connections.ts abstract all provider configurations through providerType, piAuthProvider, and baseUrl fields.
• OpenRouter uses providerType: "pi" with piAuthProvider: "openrouter" and standard API key authentication injected via resolveAuthEnvVars.
• Ollama requires providerType: "pi_compat", authType: "none", and manual model definitions because it operates as a local, keyless endpoint.
• The Pi SDK handles dynamic model discovery for cloud providers, while pi_compat connections rely on static configuration for self-hosted instances.

Frequently Asked Questions

What is the difference between providerType "pi" and "pi_compat"?

The pi provider type uses the Pi unified AI SDK, which manages authentication and model resolution for supported cloud providers like OpenAI and Anthropic. The pi_compat type bypasses the Pi SDK's credential handling entirely, enabling connections to self-hosted or local endpoints like Ollama that implement OpenAI-compatible APIs but do not require standard authentication flows.

How does authentication work for OpenRouter connections?

OpenRouter connections use authType: "api_key" and the Pi SDK handles authentication internally. When the connection initializes, resolveAuthEnvVars retrieves the stored API key from the credential manager and injects it into the environment. The Pi SDK then maps this key to the appropriate header when making requests to the OpenRouter gateway specified in baseUrl.

Why do I need to manually specify models for Ollama?

Ollama exposes a local HTTP server that does not provide a model enumeration endpoint. Because the Pi SDK cannot query a local Ollama instance to discover available models, the models array in the connection configuration must be populated manually with the specific model IDs you have pulled locally (e.g., ollama/mistral).

Which file handles the provider metadata mapping?

Provider metadata such as display names and dashboard URLs are mapped in packages/shared/src/config/provider-metadata.ts. This file defines the mapping for piAuthProvider values like "openrouter" and "ollama", ensuring the UI displays correct branding and links when these third-party endpoints are configured.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}