How to Configure Custom OpenAI-Compatible Endpoint Support in Meetily

Meetily’s summarization engine can connect to any server implementing the OpenAI chat-completion API through its configurable Custom OpenAI provider, allowing you to set a base URL, authentication key, model name, and optional generation parameters without touching core Rust code.

The Zackriya-Solutions/meetily codebase exposes a fully pluggable backend for self-hosted or third-party LLM services such as Ollama, LocalAI, Azure OpenAI, or vLLM. By leveraging the CustomOpenAIConfig struct and a set of dedicated Tauri commands, you can point Meetily at a custom endpoint, persist the settings to SQLite, verify connectivity, and immediately start generating meeting summaries.

What Is Custom OpenAI-Compatible Endpoint Support in Meetily?

Meetily’s custom OpenAI-compatible endpoint support is a first-class provider path inside the Rust-based summarization pipeline. Instead of hardcoding vendor-specific logic, the backend relies on a JSON-serializable configuration object and generic HTTP client code that targets the standard /chat/completions route.

The CustomOpenAIConfig Struct

The configuration contract is defined in src-tauri/src/summary/mod.rs (lines 12‑31) as CustomOpenAIConfig:

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CustomOpenAIConfig {
    /// Base URL of the OpenAI‑compatible API endpoint (e.g. "http://localhost:8000/v1")
    pub endpoint: String,
    /// API key for authentication (optional)
    #[serde(rename = "apiKey")]
    pub api_key: Option<String>,
    /// Model identifier to use (e.g. "gpt‑4", "llama‑3‑70b")
    pub model: String,
    /// Maximum tokens for completion (optional)
    #[serde(rename = "maxTokens")]
    pub max_tokens: Option<i32>,
    /// Temperature (0.0‑2.0, optional)
    pub temperature: Option<f32>,
    /// Top‑P (0.0‑1.0, optional)
    #[serde(rename = "topP")]
    pub top_p: Option<f32>,
}

The struct stores everything needed to reach the server and control text generation. The api_key is optional, making it suitable for local, unauthenticated endpoints, while max_tokens, temperature, and top_p let you fine-tune completions on a per-configuration basis.

How to Save and Retrieve Your Custom Endpoint Configuration

Meetily persists the configuration through Tauri commands that serialize the struct and upsert it into the local SQLite settings table.

Saving the Configuration

The api_save_custom_openai_config command in src-tauri/src/api/api.rs (lines 1172‑1220) validates incoming fields and stores the result:

import { invoke } from '@tauri-apps/api/tauri';

await invoke('api_save_custom_openai_config', {
  endpoint: 'http://localhost:8000/v1',
  api_key: 'my-secret-key',   // optional
  model: 'gpt-4o-mini',
  max_tokens: 1024,           // optional
  temperature: 0.7,           // optional
  top_p: 0.9                  // optional
});

On success, the command returns a JSON payload confirming the operation:

{ "status": "success", "message": "Custom OpenAI configuration saved successfully" }

Retrieving the Stored Configuration

To read back the saved settings, the frontend calls api_get_custom_openai_config as defined in src-tauri/src/api/api.rs (lines 1244‑1265):

const cfg = await invoke<CustomOpenAIConfig>('api_get_custom_openai_config');
console.log('Current custom OpenAI config:', cfg);

If no configuration exists, the command resolves to None (or null on the TypeScript side), letting the UI fall back to a default provider or prompt the user for setup.

Testing Connectivity to a Custom OpenAI-Compatible Endpoint

Before running expensive summarization jobs, Meetily provides a lightweight health check via api_test_custom_openai_connection in src-tauri/src/api/api.rs (lines 1270‑1350). This command sends a minimal chat request—prompt "Hi" with max_tokens: 5—to {endpoint}/chat/completions and validates that the response contains a choices array with a valid message.content field.

const result = await invoke('api_test_custom_openai_connection', {
  endpoint: 'http://localhost:8000/v1',
  api_key: 'my-secret-key',
  model: 'gpt-4o-mini'
});

console.log(result);
// → { status: 'success', message: 'Connection successful …' }

Because the test uses only five tokens, it offers a fast, low-cost way to confirm that the base URL, authentication, and model identifier are correct.

Using a Custom Endpoint for Meeting Summarization

Once persisted, the Custom OpenAI provider is selected at runtime by passing the provider identifier "custom-openai" to the summarization command.

Provider Parsing and URL Construction

Inside src-tauri/src/summary/llm_client.rs (lines 66‑89), the LLM client maps the string "custom-openai" to the LLMProvider::CustomOpenAI enum variant. When this variant is active, lines 173‑180 construct the request URL by trimming any trailing slash from the stored endpoint and appending /chat/completions:

LLMProvider::CustomOpenAI => {
    let endpoint = custom_openai_endpoint.ok_or_else(|| "Custom OpenAI endpoint not configured".to_string())?;
    (format!("{}/chat/completions", endpoint.trim_end_matches('/')), HeaderMap::new())
}

Conditional Generation Parameters

Notably, src-tauri/src/summary/llm_client.rs only injects max_tokens, temperature, and top_p into the outgoing ChatRequest body when the Custom OpenAI provider is selected. This keeps request payloads lean for other built-in providers while giving you full control over your self-hosted endpoint.

To trigger a summary with the custom backend, invoke the summary command from the frontend:

await invoke('api_get_summary', {
  provider: 'custom-openai',
  // additional arguments such as meeting_id, transcription_text, etc.
});

Summary

• CustomOpenAIConfig in src-tauri/src/summary/mod.rs defines the JSON contract for base URLs, API keys, model names, and optional generation knobs.
• api_save_custom_openai_config persists the struct to SQLite, while api_get_custom_openai_config reads it back.
• api_test_custom_openai_connection performs a minimal, low-token health check against /chat/completions before real workloads run.
• The LLM client in src-tauri/src/summary/llm_client.rs maps the "custom-openai" provider string to a dynamically constructed URL and conditionally injects max_tokens, temperature, and top_p.
• This architecture lets Zackriya-Solutions/meetily users plug in Ollama, LocalAI, Azure OpenAI, or any OpenAI-compatible service without recompiling the Rust core.

Frequently Asked Questions

Which self-hosted or third-party services work with Meetily's custom endpoint?

Any service that exposes the standard OpenAI /chat/completions JSON schema is compatible. Popular options include Ollama, LocalAI, vLLM, and Azure OpenAI, provided they accept the same request and response shapes.

How does Meetily store the API key for a custom endpoint?

The optional api_key field inside CustomOpenAIConfig is serialized and persisted alongside the other settings via SettingsRepository into the local SQLite settings table. The api_save_custom_openai_config command handles this upsert transparently.

Why does the connection test use only a minimal prompt?

The api_test_custom_openai_connection command sends the single-word prompt "Hi" with max_tokens capped at 5. This design verifies network reachability, authentication, and JSON structure while consuming negligible compute and token budget.

Are generation parameters applied to every LLM provider in Meetily?

No. According to the request builder in src-tauri/src/summary/llm_client.rs, max_tokens, temperature, and top_p are appended to the ChatRequest body only when the active provider is CustomOpenAI. Other providers use their own default or server-side behavior.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}