# How to Add a Custom LLM Provider to LMForge: Complete Implementation Guide > Learn to add a custom LLM provider to LMForge by editing providers.yaml, creating a provider package, and implementing a LangChain model subclass, extending LMForge's capabilities. - Repository: [Haohao/lmforge-end-to-end-llmops-platform-for-multi-model-agents](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents) - Tags: how-to-guide - Published: 2026-03-03 --- **Adding a custom LLM provider to LMForge requires registering the provider in [`providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/providers.yaml), creating a provider package under `api/internal/core/language_model/providers//`, and implementing a LangChain-based model class that subclasses `BaseLanguageModel`, all without modifying core platform code.** LMForge is an end-to-end LLMOps platform designed for multi-model agents that supports dynamic provider registration through a YAML-driven registry. If you need to integrate a proprietary, self-hosted, or third-party language model beyond the built-in options, the platform's extensible architecture enables you to add custom LLM providers by following specific file structure conventions. This guide demonstrates the exact implementation based on the `haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents` source code. ## Understanding the LMForge Provider Architecture LMForge loads language model providers dynamically using a **YAML-driven registry pattern** centered around the `LanguageModelManager` class. According to the source code in [`api/internal/core/language_model/language_model_manager.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/core/language_model/language_model_manager.py), the manager reads the global [`providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/providers.yaml) file and instantiates `Provider` objects (defined in [`api/internal/core/language_model/entities/provider_entity.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/core/language_model/entities/provider_entity.py)) for each entry. The `Provider` class validates each provider by calling `dynamic_import("internal.core.language_model.providers.{provider_name}.{model_type}", Symbol)` to lazily load model classes. This means you can add a custom LLM provider to LMForge by simply conforming to the expected directory layout and YAML schema—no changes to the core API layer are necessary. Key components in the architecture include: - **[`providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/providers.yaml)** – The global registry at [`api/internal/core/language_model/providers/providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/api/internal/core/language_model/providers/providers.yaml) listing all available providers with their metadata and supported model types. - **`LanguageModelManager`** – Reads the registry and builds a `provider_map` containing `ProviderEntity` objects. - **[`positions.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/positions.yaml)** – Defines the display order and identifiers for models within a specific provider directory. - **`.yaml`** – Contains individual model metadata including parameters, descriptions, and configuration options. - **`.py`** – The concrete implementation file (e.g., [`chat.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/chat.py)) that must contain a class matching the capitalized model type name (e.g., `Chat`). ## Step-by-Step Guide to Adding a Custom LLM Provider ### Step 1: Register the Provider in providers.yaml First, add your provider entry to the global registry. The `name` field becomes the directory name and unique identifier used throughout the system. ```yaml # api/internal/core/language_model/providers/providers.yaml - name: mycustom label: My Custom LLM description: My own hosted LLM offering with OpenAI-compatible API. icon: icon.png background: "#F0F0F0" supported_model_types: - chat ``` Commit this change before creating the provider package. The `supported_model_types` list declares which model implementation files (e.g., [`chat.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/chat.py)) the manager should expect to find. ### Step 2: Create the Provider Package Structure Create a new directory matching the provider `name` under the providers folder. The structure must include [`__init__.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/__init__.py), implementation files, and YAML metadata: ``` api/internal/core/language_model/providers/ └─ mycustom/ ├─ __init__.py ├─ chat.py ├─ positions.yaml ├─ mymodel.yaml └─ _asset/ └─ icon.png ``` ### Step 3: Implement the Model Class Create the implementation file named after the model type (e.g., [`chat.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/chat.py) for chat models). The class name must match the capitalized model type exactly—[`chat.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/chat.py) requires a class named `Chat`. ```python # api/internal/core/language_model/providers/mycustom/chat.py from langchain_community.chat_models import ChatOpenAI from internal.core.language_model.entities.model_entity import BaseLanguageModel class Chat(ChatOpenAI, BaseLanguageModel): """Custom chat model implementation for MyCustom provider.""" pass ``` The `LanguageModelManager` imports this as `internal.core.language_model.providers.mycustom.chat.Chat` using the dynamic import mechanism defined in `Provider.validate_provider()` (lines 29-44 of [`provider_entity.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/provider_entity.py)). ### Step 4: Define Model Ordering with positions.yaml Create [`positions.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/positions.yaml) to list the models belonging to this provider in display order. Each entry references a corresponding `.yaml` file. ```yaml # api/internal/core/language_model/providers/mycustom/positions.yaml - mymodel ``` ### Step 5: Configure Model Metadata Create individual YAML files for each model, defining parameters, defaults, and descriptions. These files construct `ModelEntity` objects during provider validation (lines 58-82 of [`provider_entity.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/provider_entity.py)). ```yaml # api/internal/core/language_model/providers/mycustom/mymodel.yaml name: mymodel label: MyCustom-Chat-v1 description: A 7B-parameter chat model hosted at https://my.custom.endpoint type: chat parameters: - name: temperature type: float default: 0.7 description: Controls randomness of the output. - name: max_tokens type: int default: 1024 description: Maximum number of generated tokens. ``` For reusable parameter sets across multiple models, reference a template using `use_template: ` as demonstrated in the built-in providers. ### Step 6: Add Provider Assets (Optional) Place an icon file in the `_asset/` subdirectory and reference it in [`providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/providers.yaml) via the `icon` field. The LMForge UI renders this image automatically when displaying the provider in model selection interfaces. ### Step 7: Restart LMForge When the platform boots, `LanguageModelManager` reads the updated [`providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/providers.yaml), discovers the `mycustom` directory, validates the `Chat` class import, constructs the `ModelEntity` for `mymodel`, and registers the provider. The new models become immediately available through the API endpoint: ``` GET /v1/language-models?provider_name=mycustom ``` ## Testing Your Custom LLM Provider Verify your implementation using the following Python script to list providers, enumerate models, and invoke completions: ```python import requests BASE = "http://localhost:8000/v1" # List all providers - should include "mycustom" print(requests.get(f"{BASE}/language-models/providers").json()) # List models for the new provider print(requests.get(f"{BASE}/language-models?provider_name=mycustom").json()) # Invoke the custom chat model payload = { "provider_name": "mycustom", "model_name": "mymodel", "model_type": "chat", "messages": [{"role": "user", "content": "Hello, world!"}], "parameters": {"temperature": 0.6} } resp = requests.post(f"{BASE}/chat/completions", json=payload) print(resp.json()) ``` Replace the `BASE` URL with your deployment host and port. Successful responses confirm that `LanguageModelManager.get_provider("mycustom")` and `Provider.get_model_class("chat")` are functioning correctly. ## Summary - **Zero core modifications** are required to add a custom LLM provider to LMForge—only YAML configuration and a thin LangChain wrapper class. - The **dynamic import system** (`dynamic_import`) in `ProviderEntity` automatically discovers model classes when you follow the naming convention: `/.py` containing a class named with the capitalized model type. - **Metadata-driven configuration** via [`providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/providers.yaml), [`positions.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/positions.yaml), and `.yaml` files decouples provider definitions from implementation code. - The platform constructs **provider and model entities** (`ProviderEntity`, `ModelEntity`) at startup, making new providers immediately available via `LanguageModelManager.get_provider(name)` and the REST API. ## Frequently Asked Questions ### What file naming conventions must I follow for LMForge to recognize my model? You must name the implementation file after the model type (e.g., [`chat.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/chat.py) for chat models, [`completion.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/completion.py) for completion models) and ensure the class inside matches the capitalized type exactly (`Chat`, `Completion`). The provider directory name must match the `name` field in [`providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/providers.yaml). The `LanguageModelManager` uses these conventions to build the import path `internal.core.language_model.providers.{provider_name}.{model_type}`. ### Can I add multiple model types (chat, completion, embedding) to a single custom provider? Yes. List all supported types in the `supported_model_types` array within your [`providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/providers.yaml) entry, then create corresponding implementation files ([`chat.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/chat.py), [`completion.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/completion.py), [`embedding.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/embedding.py)) in your provider directory. Each file must contain a class named with the capitalized model type. The `Provider` class maintains a `model_class_map` that stores references to each implementation for runtime retrieval via `Provider.get_model_class(model_type)`. ### How does LMForge handle authentication for custom LLM providers? Authentication credentials are typically handled within your model implementation class by subclassing LangChain base classes (e.g., `ChatOpenAI`) and passing API keys or tokens through environment variables or the `parameters` section of your `.yaml` configuration. The platform does not enforce a specific authentication pattern in the registry layer, allowing you to implement standard LangChain credential mechanisms in your [`chat.py`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/chat.py) or other model files. ### Is it possible to modify an existing provider without editing the core codebase? No modifications to existing built-in providers (OpenAI, Anthropic, etc.) are necessary to add your own. The [`providers.yaml`](https://github.com/haohao-end/lmforge-end-to-end-llmops-platform-for-multi-model-agents/blob/main/providers.yaml) registry supports multiple concurrent providers, and the `LanguageModelManager` maintains separate `ProviderEntity` instances for each entry in its `provider_map`. You only need to create new files in a separate directory under `api/internal/core/language_model/providers/` without touching existing provider implementations or the manager logic.