How to Integrate Custom AI Models into HolaOS: A Complete Developer Guide

Yes, HolaOS supports custom AI model integration through its model‑agnostic harness architecture, allowing you to register any provider and model without modifying core runtime code.

The HolaOS platform is designed around a declarative model registry system that decouples AI capabilities from specific vendors. Whether you need to connect a private OpenAI‑compatible endpoint, an Ollama instance, or a proprietary model gateway, the runtime resolves models dynamically through typed configuration fields and a centralized routing layer.

Understanding the HolaOS Model Architecture

HolaOS uses three core abstractions for AI model integration:

• Provider: A connection configuration (base URL, authentication, timeouts) identified by a provider_id
• Model: A concrete model descriptor registered under a provider, identified by a model_id
• Harness: The runtime component that consumes models through the routing layer

The ModelRegistry class in runtime/harness-host/src/pi.ts maintains these mappings in memory, while the routing layer in runtime/harnesses/src/model-routing.ts handles normalization and lookup at request time.

This separation means you can add, remove, or swap models without redeploying the core platform—only the registry configuration changes.

Registering a Custom Model Provider

Before adding individual models, you must register the provider that hosts them. This typically occurs during server initialization.

// runtime/harness-host/src/pi.ts
import { ModelRegistry } from "./model-registry";
import { buildPiProviderConfig } from "./provider-config";

const modelRegistry = ModelRegistry.create();

// Register a custom proxy or direct endpoint
modelRegistry.registerProvider(
  "my_custom_proxy",
  buildPiProviderConfig({
    baseUrl: "https://my-model-gateway.example.com",
    authToken: process.env.MY_PROXY_TOKEN,
  })
);

The buildPiProviderConfig helper constructs a normalized configuration object that the runtime uses for connection pooling and retry logic. Supported provider types include openai, ollama_direct, and holaboss_model_proxy.

Adding Models to the Registry

Once the provider exists, register specific models under it:

// Continued from above
modelRegistry.registerModel("my_custom_proxy", {
  id: "my-gpt-5-custom",
  name: "My Custom GPT-5",
  maxTokens: 8192,
  // Additional metadata for runtime optimizations
});

The id field becomes your model_id throughout the system. The name field appears in UI selectors and logs. Optional fields like maxTokens guide the runtime's request batching and context window management.

Consuming Custom Models in Harness Requests

With the model registered, reference it in any harness payload:

// runtime/harnesses/src/pi.ts
import type { HarnessModelRoutingRequest } from "./model-routing";

const request: HarnessModelRoutingRequest = {
  provider_id: "my_custom_proxy",
  model_id: "my-gpt-5-custom",
  model_client: "openai", // Routing hint for protocol adaptation
  // Prompt, temperature, and other inference parameters follow
};

await startPiTool(request);

The startPiTool function delegates to resolvePiModel in the registry, which validates that the provider_id + model_id pair exists and injects the resolved endpoint into the request pipeline.

Exposing Model Selection in Plugin Templates

Plugins can surface model choices to end users through typed configuration variables. The BaseFieldType system in runtime/state-store/src/store.ts includes a dedicated model_id type for this purpose.

// runtime/api-server/src/plugin-templates.ts
const MY_PLUGIN_TEMPLATE = defineTemplate({
  id: "my_plugin",
  version: "1",
  name: "My Plugin",
  pluginVariables: [
    { 
      key: "model_id", 
      type: "string", 
      default: "my-gpt-5-custom" 
    }
  ],
  instantiateWorkflows: ({ typedConfig, plugin }) => [
    {
      workflowId: `${plugin.pluginId}__run`,
      name: "Run with custom model",
      nodes: [
        {
          id: "model",
          type: "pi",
          payload: { model_id: typedConfig.model_id },
        },
        // Additional workflow nodes
      ],
      edges: [{ from: "model", to: "next" }],
    },
  ],
});

The typedConfig.model_id value flows through the state store's type system, ensuring compile-time validation and runtime persistence of the user's selection.

Model ID Normalization and Routing

The routing layer in runtime/harnesses/src/model-routing.ts performs two critical operations:

1. normalizeHarnessModelId: Sanitizes incoming model_id values to handle aliases, versioning suffixes, and legacy formats
2. Registry lookup: Resolves the normalized ID against the in-memory ModelRegistry

This indirection enables zero-downtime model migrations—you can update the registry mapping while keeping existing workflow definitions unchanged.

Key Source Files for Custom Model Integration

Component	File Path	Purpose
Model registry	runtime/harness-host/src/pi.ts	Provider and model registration; resolvePiModel implementation
Routing logic	runtime/harnesses/src/model-routing.ts	normalizeHarnessModelId and request routing
State field definitions	runtime/state-store/src/store.ts	BaseFieldType enum including model_id
Template SDK	runtime/api-server/src/plugin-template-sdk.ts	defineTemplate and plugin variable system
SDK documentation	docs/plugin-sdk.md	Complete plugin development reference

Summary

• HolaOS uses a declarative registry—no core code changes required for new models
• Three-step integration: register provider → register model → reference in payloads
• Typed configuration via BaseFieldType.model_id enables plugin-level model selection
• Routing layer abstraction handles normalization and endpoint injection transparently
• Source locations: runtime/harness-host/src/pi.ts for registry, runtime/harnesses/src/model-routing.ts for routing

Frequently Asked Questions

What model protocols does HolaOS support?

HolaOS supports any OpenAI‑compatible API, Ollama endpoints, and custom protocols through the model_client routing hint. The buildPiProviderConfig function in runtime/harness-host/src/pi.ts adapts connection parameters accordingly.

Can I switch models without restarting the runtime?

Yes. The ModelRegistry is designed for dynamic updates, though provider registration typically occurs at startup. Hot-reloading of model configurations depends on your deployment's registry refresh strategy.

How do I validate that my custom model is reachable?

The harness system includes health check integrations through resolvePiModel. Check runtime logs for registry lookup results and endpoint resolution status when invoking startPiTool.

Where should I store provider credentials?

Use environment variables injected into buildPiProviderConfig, as shown in the registration example. Never commit tokens to the model registry definition—reference them through process.env at initialization time.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}