How to Add a New LLM Provider to ChatMCP: A Complete Integration Guide

To add a new LLM provider to ChatMCP, extend BaseLLMClient to implement the provider's HTTP API, register the client in LLMFactory, and map the provider ID in LLMFactoryHelper.providerMap.

ChatMCP is an open-source Flutter application (available at daodao97/chatmcp) that unifies interactions with multiple language model services through a common client interface. Whether you're integrating a commercial API like DeepSeek or a self-hosted Ollama-compatible endpoint, the architecture uses a factory pattern that lets you plug in new providers without touching the chat UI or settings logic.

Implement a Concrete Client Class

Every LLM provider in ChatMCP inherits from BaseLLMClient, located in lib/llm/base_llm_client.dart. This abstract class defines three required members you must implement:

• chatCompletion: Handles one-shot requests returning Future<LLMResponse>
• chatStreamCompletion: Returns Stream<LLMResponse> for server-sent events (SSE)
• models: Returns Future<List<String>> of available model identifiers

Create the Provider File

Create a new Dart file at lib/llm/mynew_client.dart (replace "mynew" with your provider name). Use lib/llm/openai_client.dart as a reference implementation for OpenAI-compatible APIs:

import 'package:http/http.dart' as http;
import 'base_llm_client.dart';
import 'dart:convert';
import 'model.dart';
import 'package:logging/logging.dart';

class MyNewClient extends BaseLLMClient {
  final String apiKey;
  final String baseUrl;
  final Map<String, String> _headers;

  MyNewClient({required this.apiKey, String? baseUrl})
      : baseUrl = (baseUrl == null || baseUrl.isEmpty) 
            ? 'https://api.mynew.ai/v1' 
            : baseUrl,
        _headers = {
          'Content-Type': 'application/json; charset=utf-8', 
          'Authorization': 'Bearer $apiKey'
        };

  @override
  Future<LLMResponse> chatCompletion(CompletionRequest request) async {
    final httpClient = BaseLLMClient.createHttpClient();
    final body = {
      'model': request.model,
      'messages': chatMessageToOpenAIMessage(request.messages),
      // Add provider-specific fields here
    };
    addModelSettingsToBody(body, request.modelSetting);
    
    final response = await httpClient.post(
      Uri.parse("$baseUrl/chat/completions"), 
      headers: _headers, 
      body: jsonEncode(body)
    );
    
    // Parse response into LLMResponse (adapt based on provider's JSON schema)
    final data = jsonDecode(utf8.decode(response.bodyBytes));
    return LLMResponse(
      content: data['choices'][0]['message']['content'],
      model: request.model,
    );
  }

  @override
  Stream<LLMResponse> chatStreamCompletion(CompletionRequest request) async* {
    final httpClient = BaseLLMClient.createHttpClient();
    final body = {
      'model': request.model,
      'messages': chatMessageToOpenAIMessage(request.messages),
      'stream': true,
    };
    addModelSettingsToBody(body, request.modelSetting);
    
    final requestUri = Uri.parse("$baseUrl/chat/completions");
    final requestObj = http.Request('POST', requestUri)
      ..headers.addAll(_headers)
      ..body = jsonEncode(body);
      
    final response = await httpClient.send(requestObj);
    
    await for (final chunk in response.stream
        .transform(utf8.decoder)
        .transform(const LineSplitter())) {
      if (chunk.startsWith('data: ')) {
        final jsonData = chunk.substring(6);
        if (jsonData.trim() == '[DONE]') break;
        
        final data = jsonDecode(jsonData);
        yield LLMResponse(
          content: data['choices'][0]['delta']['content'] ?? '',
          model: request.model,
        );
      }
    }
  }

  @override
  Future<List<String>> models() async {
    final httpClient = BaseLLMClient.createHttpClient();
    final response = await httpClient.get(
      Uri.parse("$baseUrl/models"), 
      headers: _headers
    );
    
    final data = jsonDecode(utf8.decode(response.bodyBytes));
    return (data['data'] as List)
        .map((m) => m['id'] as String)
        .toList();
  }
}

Leverage Base Class Utilities

When implementing your client, utilize these utilities from BaseLLMClient:

• Use BaseLLMClient.createHttpClient() to instantiate the HTTP client; this respects the global proxy configuration defined in the app's network settings.
• Call chatMessageToOpenAIMessage for providers using OpenAI-compatible message schemas, or implement your own conversion for custom formats.
• Invoke addModelSettingsToBody to automatically inject temperature, top-p, and other inference parameters from the user's ModelSetting configuration.

Register the Client in the Factory

The factory system in lib/llm/llm_factory.dart instantiates clients based on provider identifiers. You must update four locations in this file:

1. Extend the Provider Enum

Add your provider to the LLMProvider enum around line 15:

enum LLMProvider { 
  openai, 
  claude, 
  ollama, 
  deepseek, 
  gemini, 
  foundry, 
  claudeCode, 
  copilot, 
  mynew 
}

2. Import Your Client

Add the import statement at the top of lib/llm/llm_factory.dart:

import 'mynew_client.dart';

3. Add Factory Case

Insert a case in the LLMFactory.create switch block (around line 20) to instantiate your client:

static BaseLLMClient create(LLMProvider provider, {
  required String apiKey, 
  required String baseUrl
}) {
  switch (provider) {
    // ... existing cases ...
    case LLMProvider.mynew:
      return MyNewClient(apiKey: apiKey, baseUrl: baseUrl);
  }
}

4. Map the Provider ID

Add an entry to LLMFactoryHelper.providerMap (lines 47-56) so the UI can resolve provider strings from model configurations:

static final Map<String, LLMProvider> providerMap = {
  "openai": LLMProvider.openai,
  "claude": LLMProvider.claude,
  // ... existing mappings ...
  "mynew": LLMProvider.mynew,
};

Once registered, LLMFactoryHelper.createFromModel automatically looks up the provider ID from a model's providerId field, falls back to the enum name, and calls LLMFactory.create to instantiate your client.

Configure the UI and Settings (Optional)

To allow users to configure API keys and endpoints for your new provider:

• Add configuration fields to LLMProviderSetting in lib/provider/settings_provider.dart. The class already stores apiKey, apiEndpoint, and providerId fields that you can reuse.
• The settings page at lib/page/setting/llm_setting.dart dynamically iterates over ProviderManager.settingsProvider.apiSettings. Adding a configuration entry with providerId: "mynew" automatically renders the provider in the dropdown without modifying the UI code.

Verify the Integration

Test your implementation with these steps:

1. Run the application: Execute flutter run to launch ChatMCP.
2. Configure the provider: Navigate to Settings > LLM Settings, select your new provider from the dropdown, and enter the API key and endpoint URL.
3. Test chat completion: Start a new conversation and select a model associated with your provider. The model list populates from your client's models() method.
4. Verify streaming: Send a message and confirm that token-by-token streaming works via chatStreamCompletion.

If the chat interface loads models and generates responses using your client, the integration is complete.

Summary

• Extend BaseLLMClient in lib/llm/base_llm_client.dart to implement provider-specific HTTP logic for chatCompletion, chatStreamCompletion, and models.
• Register in LLMFactory by adding an enum value to LLMProvider, importing your client file, and adding a case to the factory's create method.
• Map the provider ID in LLMFactoryHelper.providerMap so the system can resolve "mynew" strings to your LLMProvider.mynew enum.
• Reuse base utilities like createHttpClient(), chatMessageToOpenAIMessage(), and addModelSettingsToBody() to maintain consistency with global proxy settings and message formatting.
• Configure settings by adding a provider entry in SettingsProvider; the UI at llm_setting.dart will detect it automatically.

Frequently Asked Questions

Do I need to modify the chat UI code to add a new LLM provider?

No. ChatMCP's architecture separates provider logic from the UI. Once you register the client in LLMFactory and map the provider ID in LLMFactoryHelper, the chat interface in lib/page/chat/ automatically uses LLMFactoryHelper.createFromModel to instantiate the correct client based on the selected model's providerId field.

What if my LLM provider uses a non-OpenAI message format?

Implement your own conversion logic instead of using chatMessageToOpenAIMessage. In your client's chatCompletion method, transform the request.messages list (which contains ChatMessage objects) into your provider's required JSON structure. You can reference lib/llm/claude_client.dart for an example of a provider with custom message formatting and authentication headers.

How do I handle custom authentication schemes beyond Bearer tokens?

Override the header construction in your client's constructor or request methods. While the example uses Authorization: Bearer $apiKey, you can modify _headers to include API keys as query parameters, custom header names (like x-api-key), or multi-header authentication schemes required by your specific provider.

Can I add support for a local model without an HTTP API?

Yes, though this requires additional implementation. BaseLLMClient expects HTTP-based implementations, but you can adapt the pattern for local processes by overriding chatCompletion to communicate via stdin/stdout or a local socket, while still returning LLMResponse objects. Ensure your models() method returns a static list if the local endpoint doesn't support model enumeration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}