Using Custom Endpoints for Model Resolution in Craft Agents

Craft Agents supports custom endpoints for model resolution by checking a dedicated 'custom-endpoint' provider registry before standard providers, ensuring your self-hosted or third-party LLMs take precedence when specified.

The lukilabs/craft-agents-oss repository provides a flexible model resolution system that allows developers to integrate custom endpoints for model resolution alongside standard providers. This capability enables connections to OpenAI-compatible APIs such as Ollama, OpenRouter, or self-hosted services while ensuring these custom models are prioritized during the resolution process.

How Custom Endpoint Resolution Works in Craft Agents

The resolution logic centers on the resolvePiModel function located in packages/pi-agent-server/src/model-resolution.ts (lines 10-14). When the Pi SDK needs to resolve a model identifier, the system first checks the custom-endpoint registry before falling back to standard providers.

Specifically, when the preferCustomEndpoint flag is set to true, the resolver immediately queries the 'custom-endpoint' provider (lines 24-28). If a matching model is found, it is returned immediately; otherwise, the resolver proceeds with normal provider lookup. This guarantees that models registered for a custom endpoint are selected even when a model with the same ID exists under another provider.

Implementing Custom Endpoints for Model Resolution

Setting up custom endpoints for model resolution requires three distinct steps: building synthetic model definitions, registering them with the custom provider, and resolving with priority flags.

Building Synthetic Model Definitions

Because remote custom services cannot be queried for capabilities at runtime, Craft Agents uses synthetic model definitions constructed by buildCustomEndpointModelDef in packages/pi-agent-server/src/custom-endpoint-models.ts (lines 13-34). This function generates model objects with sensible defaults, including a 131 KB context window and optional image support flags.

Registering Models with the Custom Endpoint Provider

Once built, synthetic models must be registered under the 'custom-endpoint' provider name. The runtime registration occurs in packages/pi-agent-server/src/index.ts, which handles connections using base URLs and API keys persisted in packages/shared/src/config/storage.ts. The registry stores these synthetic models alongside standard provider offerings.

Resolving Models with Custom Endpoint Priority

To ensure your custom endpoint models are selected, invoke resolvePiModel with preferCustomEndpoint: true. This parameter triggers the priority check against the custom-endpoint registry before any standard provider lookup begins.

Complete Code Example for Custom Endpoint Integration

The following example demonstrates connecting to an Ollama instance using the custom endpoint resolution system:

import { resolvePiModel } from
  'craft-agents-oss/packages/pi-agent-server/src/model-resolution';
import { buildCustomEndpointModelDef } from
  'craft-agents-oss/packages/pi-agent-server/src/custom-endpoint-models';
import type { ModelRegistry } from '@mariozechner/pi-coding-agent';

// 1️⃣ Build a synthetic model for a custom endpoint (e.g., Ollama)
const customModel = buildCustomEndpointModelDef('gemma-2b', {
  supportsImages: true,
});

// 2️⃣ Register the model with the Pi SDK registry
const registry: ModelRegistry = getPiModelRegistry(); // assume existing registry
registry.register('custom-endpoint', customModel);

// 3️⃣ Resolve a model ID, preferring the custom endpoint
const model = resolvePiModel(
  registry,
  'gemma-2b',          // model ID supplied by the user
  undefined,           // optional auth provider (none for custom)
  true,                // preferCustomEndpoint
);

if (model) {
  console.log('Resolved model:', model.id);
} else {
  console.error('Model not found');
}

This implementation imports the resolver and builder from their respective source files in packages/pi-agent-server/src/. The code constructs a synthetic model definition for gemma-2b, registers it under the custom-endpoint provider, and resolves the model with priority flag enabled to ensure selection over any standard provider variants.

Key Source Files for Custom Endpoint Resolution

Understanding the implementation requires familiarity with these specific files in the lukilabs/craft-agents-oss repository:

File	Role
packages/pi-agent-server/src/model-resolution.ts	Implements resolvePiModel, the entry point for model lookup with custom-endpoint precedence.
packages/pi-agent-server/src/custom-endpoint-models.ts	Provides buildCustomEndpointModelDef to create synthetic model objects for custom endpoints.
packages/pi-agent-server/src/index.ts	Registers custom-endpoint models at runtime when connections with custom base URLs are established.
packages/shared/src/config/storage.ts	Persists connection details (including baseUrl and apiKey) that trigger custom-endpoint handling.

These components collectively enable the custom endpoints for model resolution functionality, allowing seamless integration of third-party and self-hosted LLM services.

Summary

• Custom endpoints for model resolution in Craft Agents allow integration of OpenAI-compatible APIs (Ollama, OpenRouter, etc.) alongside standard providers.
• The resolvePiModel function in packages/pi-agent-server/src/model-resolution.ts checks the 'custom-endpoint' provider first when preferCustomEndpoint is enabled.
• Synthetic model definitions are constructed using buildCustomEndpointModelDef in packages/pi-agent-server/src/custom-endpoint-models.ts to provide default capabilities for remote services.
• Connection details are persisted in packages/shared/src/config/storage.ts and runtime registration occurs in packages/pi-agent-server/src/index.ts.

Frequently Asked Questions

What is the purpose of the preferCustomEndpoint flag in Craft Agents?

The preferCustomEndpoint flag ensures that model resolution prioritizes the 'custom-endpoint' provider registry before checking standard providers. When set to true, the resolvePiModel function immediately queries custom registered models in packages/pi-agent-server/src/model-resolution.ts (lines 24-28), ensuring your self-hosted or third-party models are selected even if identical IDs exist in standard providers.

How does Craft Agents handle model capabilities for custom endpoints?

Since remote custom services cannot be queried for capabilities at runtime, Craft Agents uses the buildCustomEndpointModelDef function in packages/pi-agent-server/src/custom-endpoint-models.ts (lines 13-34) to generate synthetic model definitions. These provide sensible defaults including a 131 KB context window and configurable image support flags, ensuring the model can be used immediately without manual capability specification.

Can multiple custom endpoints be registered simultaneously?

Yes, the model registry supports multiple synthetic models under the single 'custom-endpoint' provider name. Each custom endpoint connection generates its own model definitions via buildCustomEndpointModelDef, and all are registered in the same provider namespace in packages/pi-agent-server/src/index.ts. The resolvePiModel function searches across all registered custom models when the preference flag is enabled.

Where are custom endpoint connection details stored in Craft Agents?

Connection details including the baseUrl and optional apiKey for custom endpoints are persisted in packages/shared/src/config/storage.ts. This storage layer maintains the configuration data that triggers custom-endpoint handling in the runtime, ensuring that synthetic model registration and resolution operate with the correct connection parameters when the Pi SDK initializes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}