How to Configure and Integrate a Custom AI Model with Chat2DB’s AI Assistant

You can configure and integrate a custom AI model with Chat2DB by creating a model configuration via the /ai/model-config endpoint with your provider, API key, and optional base URL, then referencing that configuration ID in subsequent chat requests.

Chat2DB is an open-source database management tool from OtterMind that ships with a pluggable AI assistant architecture. According to the Chat2DB source code, the integration relies on a clear separation between configuration persistence, runtime model resolution, and client instantiation, allowing you to connect any OpenAI-compatible, Anthropic Claude, or Google Gemini LLM—including self-hosted endpoints.

Supported AI Providers and Architecture Overview

Chat2DB’s AI integration is built around three core layers that handle how to configure and integrate a custom AI model with Chat2DB:

1. API / Controller Layer – Exposes REST endpoints for CRUD operations on model configs and chat messaging. The AiChatController class handles requests to save, test, list, and delete configurations.

2. Service Layer – AiModelConfigServiceImpl persists configurations using AES-256-GCM encryption and resolves which model to use at runtime based on the authenticated user.

3. Factory Layer – AiModelFactory constructs the concrete Spring-AI ChatClient implementation based on the AiProviderEnum (OPENAI, CLAUDE, or GEMINI) stored in the configuration.

The system supports three built-in providers defined in AiProviderEnum:

• OPENAI – Standard OpenAI APIs and Azure OpenAI-compatible endpoints.
• CLAUDE – Anthropic’s Claude models.
• GEMINI – Google Vertex AI Gemini models.

For custom or self-hosted models, use the OPENAI provider and point the baseUrl to any API implementing the OpenAI chat-completion schema.

Step‑by‑Step: Configure a Custom AI Model

1. Select Your Provider and Gather Credentials

Before sending requests, determine your provider enum value and obtain the necessary credentials. For OpenAI-compatible endpoints, you need an API key and optionally a custom baseUrl if you are not using the official OpenAI API.

2. Create the Model Configuration via REST API

Send a POST request to /ai/model-config (handled by AiChatController.saveModelConfig). The request body must match the ModelConfigSaveRequest DTO defined in chat2db-community-web/src/main/java/ai/chat2db/community/web/api/model/request/ai/ModelConfigSaveRequest.java (lines 13‑34):

{
  "name": "My Custom LLM",
  "provider": "OPENAI",
  "model": "gpt-4o-mini",
  "apiKey": "sk-xxxxxxxxxxxxxxxxxxxx",
  "baseUrl": "https://my-selfhosted-openai.com",
  "temperature": 0.7,
  "maxTokens": 2048,
  "enabled": true,
  "defaultConfig": false
}

The apiKey is encrypted at rest using the per-installation AES-256-GCM encryption key generated by script/security/init-community-encryption-key.sh. The baseUrl field is optional; omit it to use the provider’s default endpoint.

3. Test the Configuration

Validate connectivity before using the model in production by sending a request to POST /ai/model-config/test (controller method testModelConfig). This endpoint uses AiModelFactory to instantiate a temporary client and performs a lightweight health check against the LLM endpoint, verifying that the API key and network path are functional.

4. Use the Model in Chat Requests

When sending a chat request to POST /ai/chat, include the modelConfigId returned from the save operation, or omit it to use the default configuration. The ChatRequest DTO (lines 48‑55 in ChatRequest.java) accepts the following structure:

{
  "input": "Explain the difference between INNER JOIN and LEFT JOIN.",
  "modelConfigId": "12345",
  "sessionId": "my-session-01",
  "enableTools": true
}

The AiChatStreamAdapter (line 179) retrieves the runtime model via IAiModelConfigService and delegates to AiModelFactory.create to build the appropriate AiChatClient wrapper around the Spring-AI ChatModel.

Advanced Integration Scenarios

Self‑Hosted and Proxy Endpoints

To integrate a self-hosted LLM (such as Ollama, vLLM, or Text Generation Inference), set provider to OPENAI and configure the baseUrl to point to your local or proxy server. Ensure the endpoint implements the /v1/chat/completions schema for full compatibility with AiModelFactory.

Runtime Parameter Overrides

You can override stored configuration values on a per-request basis by including temperature, maxTokens, or baseUrl directly in the ChatRequest JSON. The factory implementation prefers runtime values from the persisted config but falls back to request-level parameters when present, allowing temporary adjustments without modifying the saved configuration.

Implementation Details: How Chat2DB Resolves Models

When a chat request arrives, the system follows this resolution path:

1. Controller – AiChatController (lines 152‑166) receives the ChatRequest and extracts modelConfigId.
2. Service – AiModelConfigServiceImpl decrypts the stored API key and returns a runtime model object.
3. Factory – AiModelFactory.create (lines 54‑70) switches on the provider enum to instantiate the correct client:

AiProviderEnum provider = AiProviderEnum.from(runtimeModel.getProvider());
switch (provider) {
    case OPENAI -> return openAiClient(runtimeModel, retryTemplate);
    case CLAUDE -> return claudeClient(runtimeModel, retryTemplate);
    case GEMINI -> return geminiClient(runtimeModel, retryTemplate);
    default -> throw new IllegalArgumentException("Unsupported provider");
}

This architecture ensures that adding support for a new provider requires changes only in the factory layer, while configuration management remains provider-agnostic.

Managing Model Configurations

After you configure and integrate a custom AI model with Chat2DB, you can manage configurations through the following endpoints:

• List – GET /ai/model-config/list returns all saved configs for the current user via modelConfigList().
• Delete – POST /ai/model-config/delete removes a configuration and securely erases the encrypted API key via deleteModelConfig().

All operations are scoped to the local OS user, as Chat2DB operates as a single-user, local-first application.

Summary

• Define your model using ModelConfigSaveRequest with a provider enum, model name, encrypted API key, and optional custom baseUrl.
• Persist the configuration through AiModelConfigServiceImpl, which handles AES-256-GCM encryption for secrets.
• Test connectivity via /ai/model-config/test before deploying to production chat workflows.
• Chat by passing the modelConfigId in ChatRequest; the system resolves the config through AiModelConfigService and builds the client via AiModelFactory.
• Manage configurations through standard REST endpoints for listing and deletion.

Frequently Asked Questions

Can I use a local LLM like Ollama with Chat2DB?

Yes. Set the provider to OPENAI in your ModelConfigSaveRequest and point the baseUrl to your local Ollama server (e.g., http://localhost:11434/v1). Ollama exposes an OpenAI-compatible API that AiModelFactory can consume using the standard OpenAI client builder.

How does Chat2DB secure my API keys?

API keys are encrypted at rest using AES-256-GCM with a key generated during installation (see script/security/init-community-encryption-key.sh). The AiModelConfigServiceImpl decrypts the key only when instantiating the client for an active chat session, and keys are never logged or exposed in API responses.

What happens if I don’t specify a modelConfigId in the chat request?

If modelConfigId is omitted from the ChatRequest, the system falls back to the configuration marked as defaultConfig: true for the current user. If no default exists, the request will fail with a validation error indicating that a model configuration is required.

Can I switch between different AI models during the same session?

Yes. The sessionId in ChatRequest maintains conversation context independently of the model used. You can send subsequent requests with different modelConfigId values to route queries to different providers (e.g., Claude for analysis, GPT-4 for SQL generation) while retaining the same sessionId if your client implementation supports it.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}