How to Add a Custom LLM Provider to LMForge: Complete Implementation Guide

Adding a custom LLM provider to LMForge requires registering the provider in providers.yaml, creating a provider package under api/internal/core/language_model/providers/<provider_name>/, and implementing a LangChain-based model class that subclasses BaseLanguageModel, all without modifying core platform code.

LMForge is an end-to-end LLMOps platform designed for multi-model agents that supports dynamic provider registration through a YAML-driven registry. If you need to integrate a proprietary, self-hosted, or third-party language model beyond the built-in options, the platform's extensible architecture enables you to add custom LLM providers by following specific file structure conventions. This guide demonstrates the exact implementation based on the haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents source code.

Understanding the LMForge Provider Architecture

LMForge loads language model providers dynamically using a YAML-driven registry pattern centered around the LanguageModelManager class. According to the source code in api/internal/core/language_model/language_model_manager.py, the manager reads the global providers.yaml file and instantiates Provider objects (defined in api/internal/core/language_model/entities/provider_entity.py) for each entry.

The Provider class validates each provider by calling dynamic_import("internal.core.language_model.providers.{provider_name}.{model_type}", Symbol) to lazily load model classes. This means you can add a custom LLM provider to LMForge by simply conforming to the expected directory layout and YAML schema—no changes to the core API layer are necessary.

Key components in the architecture include:

• providers.yaml – The global registry at api/internal/core/language_model/providers/providers.yaml listing all available providers with their metadata and supported model types.
• LanguageModelManager – Reads the registry and builds a provider_map containing ProviderEntity objects.
• positions.yaml – Defines the display order and identifiers for models within a specific provider directory.
• <model>.yaml – Contains individual model metadata including parameters, descriptions, and configuration options.
• <model_type>.py – The concrete implementation file (e.g., chat.py) that must contain a class matching the capitalized model type name (e.g., Chat).

Step-by-Step Guide to Adding a Custom LLM Provider

Step 1: Register the Provider in providers.yaml

First, add your provider entry to the global registry. The name field becomes the directory name and unique identifier used throughout the system.

# api/internal/core/language_model/providers/providers.yaml

- name: mycustom
  label: My Custom LLM
  description: My own hosted LLM offering with OpenAI-compatible API.
  icon: icon.png
  background: "#F0F0F0"
  supported_model_types:
    - chat

Commit this change before creating the provider package. The supported_model_types list declares which model implementation files (e.g., chat.py) the manager should expect to find.

Step 2: Create the Provider Package Structure

Create a new directory matching the provider name under the providers folder. The structure must include __init__.py, implementation files, and YAML metadata:

api/internal/core/language_model/providers/
└─ mycustom/
   ├─ __init__.py
   ├─ chat.py
   ├─ positions.yaml
   ├─ mymodel.yaml
   └─ _asset/
      └─ icon.png

Step 3: Implement the Model Class

Create the implementation file named after the model type (e.g., chat.py for chat models). The class name must match the capitalized model type exactly—chat.py requires a class named Chat.

# api/internal/core/language_model/providers/mycustom/chat.py

from langchain_community.chat_models import ChatOpenAI
from internal.core.language_model.entities.model_entity import BaseLanguageModel

class Chat(ChatOpenAI, BaseLanguageModel):
    """Custom chat model implementation for MyCustom provider."""
    pass

The LanguageModelManager imports this as internal.core.language_model.providers.mycustom.chat.Chat using the dynamic import mechanism defined in Provider.validate_provider() (lines 29-44 of provider_entity.py).

Step 4: Define Model Ordering with positions.yaml

Create positions.yaml to list the models belonging to this provider in display order. Each entry references a corresponding <model>.yaml file.

# api/internal/core/language_model/providers/mycustom/positions.yaml

- mymodel

Step 5: Configure Model Metadata

Create individual YAML files for each model, defining parameters, defaults, and descriptions. These files construct ModelEntity objects during provider validation (lines 58-82 of provider_entity.py).

# api/internal/core/language_model/providers/mycustom/mymodel.yaml

name: mymodel
label: MyCustom-Chat-v1
description: A 7B-parameter chat model hosted at https://my.custom.endpoint
type: chat
parameters:
  - name: temperature
    type: float
    default: 0.7
    description: Controls randomness of the output.
  - name: max_tokens
    type: int
    default: 1024
    description: Maximum number of generated tokens.

For reusable parameter sets across multiple models, reference a template using use_template: <template_key> as demonstrated in the built-in providers.

Step 6: Add Provider Assets (Optional)

Place an icon file in the _asset/ subdirectory and reference it in providers.yaml via the icon field. The LMForge UI renders this image automatically when displaying the provider in model selection interfaces.

Step 7: Restart LMForge

When the platform boots, LanguageModelManager reads the updated providers.yaml, discovers the mycustom directory, validates the Chat class import, constructs the ModelEntity for mymodel, and registers the provider. The new models become immediately available through the API endpoint:

GET /v1/language-models?provider_name=mycustom

Testing Your Custom LLM Provider

Verify your implementation using the following Python script to list providers, enumerate models, and invoke completions:

import requests

BASE = "http://localhost:8000/v1"

# List all providers - should include "mycustom"

print(requests.get(f"{BASE}/language-models/providers").json())

# List models for the new provider

print(requests.get(f"{BASE}/language-models?provider_name=mycustom").json())

# Invoke the custom chat model

payload = {
    "provider_name": "mycustom",
    "model_name": "mymodel",
    "model_type": "chat",
    "messages": [{"role": "user", "content": "Hello, world!"}],
    "parameters": {"temperature": 0.6}
}
resp = requests.post(f"{BASE}/chat/completions", json=payload)
print(resp.json())

Replace the BASE URL with your deployment host and port. Successful responses confirm that LanguageModelManager.get_provider("mycustom") and Provider.get_model_class("chat") are functioning correctly.

Summary

• Zero core modifications are required to add a custom LLM provider to LMForge—only YAML configuration and a thin LangChain wrapper class.
• The dynamic import system (dynamic_import) in ProviderEntity automatically discovers model classes when you follow the naming convention: <provider>/<model_type>.py containing a class named with the capitalized model type.
• Metadata-driven configuration via providers.yaml, positions.yaml, and <model>.yaml files decouples provider definitions from implementation code.
• The platform constructs provider and model entities (ProviderEntity, ModelEntity) at startup, making new providers immediately available via LanguageModelManager.get_provider(name) and the REST API.

Frequently Asked Questions

What file naming conventions must I follow for LMForge to recognize my model?

You must name the implementation file after the model type (e.g., chat.py for chat models, completion.py for completion models) and ensure the class inside matches the capitalized type exactly (Chat, Completion). The provider directory name must match the name field in providers.yaml. The LanguageModelManager uses these conventions to build the import path internal.core.language_model.providers.{provider_name}.{model_type}.

Can I add multiple model types (chat, completion, embedding) to a single custom provider?

Yes. List all supported types in the supported_model_types array within your providers.yaml entry, then create corresponding implementation files (chat.py, completion.py, embedding.py) in your provider directory. Each file must contain a class named with the capitalized model type. The Provider class maintains a model_class_map that stores references to each implementation for runtime retrieval via Provider.get_model_class(model_type).

How does LMForge handle authentication for custom LLM providers?

Authentication credentials are typically handled within your model implementation class by subclassing LangChain base classes (e.g., ChatOpenAI) and passing API keys or tokens through environment variables or the parameters section of your <model>.yaml configuration. The platform does not enforce a specific authentication pattern in the registry layer, allowing you to implement standard LangChain credential mechanisms in your chat.py or other model files.

Is it possible to modify an existing provider without editing the core codebase?

No modifications to existing built-in providers (OpenAI, Anthropic, etc.) are necessary to add your own. The providers.yaml registry supports multiple concurrent providers, and the LanguageModelManager maintains separate ProviderEntity instances for each entry in its provider_map. You only need to create new files in a separate directory under api/internal/core/language_model/providers/ without touching existing provider implementations or the manager logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}