How Meetily Implements Custom OpenAI-Compatible Endpoint Support for Private AI Infrastructure

Meetily routes LLM calls to self-hosted OpenAI-compatible endpoints through a runtime-configurable provider that requires zero code changes and keeps credentials stored locally in SQLite.

Meetily is an open-source meeting assistant built on Tauri and Rust. Its custom OpenAI-compatible endpoint support enables organizations with private AI infrastructure—such as Azure OpenAI, self-hosted Ollama servers, or internal proxies—to route transcription summaries to internal LLM services without modifying the codebase.

Configuration Flow: From UI to SQLite Storage

Organizations configure their private endpoints through Meetily’s settings interface. The frontend captures the base URL and optional API key, then persists these values via the Tauri command layer.

The settings repository at src-tauri/src/database/commands.rs handles storage and retrieval of these credentials in a local SQLite database. This ensures sensitive endpoint URLs and keys never leave the local machine and remain available across application restarts.

Provider Selection and Enum Routing

When a user requests a meeting summary, the frontend passes the provider identifier "custom-openai" to the Rust backend. In src-tauri/src/summary/llm_client.rs, this string resolves to the LLMProvider::CustomOpenAI variant:

// src-tauri/src/summary/llm_client.rs
pub enum LLMProvider {
    OpenAI,
    CustomOpenAI,
    // ... other providers
}

The generate_summary function receives this enum and branches to the custom endpoint logic. This runtime selection means organizations can switch between public OpenAI and private endpoints without recompiling the application.

Endpoint Resolution and URL Construction

Inside generate_summary, the code extracts the user-provided base URL and constructs the final request endpoint by appending the standard OpenAI chat completions path:

// Inside generate_summary (src-tauri/src/summary/llm_client.rs)
LLMProvider::CustomOpenAI => {
    let endpoint = custom_openai_endpoint
        .ok_or_else(|| "Custom OpenAI endpoint not configured".to_string())?;
    (
        format!("{}/chat/completions", endpoint.trim_end_matches('/')),
        header::HeaderMap::new(),
    )
}

The function strips trailing slashes from the base URL to prevent malformed paths, then appends /chat/completions. This matches the OpenAI API contract, ensuring compatibility with any server implementing the same protocol.

Request Handling and Protocol Compatibility

Meetily reuses the generic OpenAI request structures defined in src-tauri/src/summary/llm_client.rs. The ChatRequest and ChatMessage structs serialize to JSON that conforms exactly to the official OpenAI schema:

{
  "model": "gpt-4-32k",
  "messages": [
    { "role": "system", "content": "You are a meeting summarizer." },
    { "role": "user", "content": "…meeting transcript…" }
  ],
  "max_tokens": 1024,
  "temperature": 0.7,
  "top_p": 0.9
}

The reqwest::Client maintained by the application sends the POST request with a 5-minute timeout. If the private endpoint requires authentication, the frontend injects the Authorization: Bearer <KEY> header into the client's default headers before invoking the Tauri command. The CustomOpenAI branch in generate_summary returns an empty HeaderMap, allowing upstream configuration to control authentication headers.

Response parsing uses the ChatResponse → Choice → MessageContent struct hierarchy, identical to the public OpenAI provider, ensuring consistent text extraction regardless of the backend.

Frontend Integration and Tauri Commands

The orchestration layer in src-tauri/src/summary/processor.rs assembles the final prompt and passes the optional custom_openai_endpoint parameter through the service layer (src-tauri/src/summary/service.rs) to the generation function.

Frontend TypeScript code invokes the custom endpoint like this:

import { invoke } from '@tauri-apps/api/tauri';

await invoke('generate_summary', {
  provider: 'custom-openai',
  modelName: 'gpt-4-32k',
  systemPrompt: 'You are a meeting summarizer.',
  userPrompt: meetingTranscript,
  customOpenaiEndpoint: 'https://my-private-ai.example.com/v1',
  apiKey: 'sk-my-private-api-key',
});

The Tauri command exposed in src-tauri/src/api/commands.rs handles the bridge between frontend requests and the Rust backend logic.

Security and Isolation Benefits

Organizations running private AI infrastructure benefit from several architectural decisions in Meetily's implementation:

• Local credential storage: API keys and endpoint URLs reside only in the local SQLite database, never in environment variables or source code.
• Runtime configuration: The provider selection happens at runtime via the LLMProvider enum, eliminating the need for conditional compilation or code forks.
• Network isolation: All HTTP requests originate from the desktop application's Rust core, keeping traffic within the organization's network perimeter when using internal endpoints.
• Cancellation support: The same cancellation token mechanism used for public providers applies to custom endpoints, allowing users to abort long-running requests against slow private servers.

Summary

• Zero-code integration: Organizations select custom-openai at runtime to route requests to private endpoints without modifying src-tauri/src/summary/llm_client.rs.
• Standard protocol: Meetily expects the OpenAI /chat/completions JSON contract, compatible with Azure OpenAI, Ollama, and other OpenAI-compatible servers.
• Secure storage: Endpoint URLs and keys persist in local SQLite via src-tauri/src/database/commands.rs, keeping credentials off public networks.
• ** consistent behavior**: Custom endpoints support the same 5-minute timeouts, cancellation tokens, and response parsing as the built-in OpenAI provider.

Frequently Asked Questions

How does Meetily handle authentication for private endpoints?

Meetily stores the optional API key in the local SQLite database through src-tauri/src/database/commands.rs. When invoking generate_summary, the frontend injects the Authorization: Bearer <KEY> header into the reqwest::Client before the request executes. The CustomOpenAI variant in src-tauri/src/summary/llm_client.rs intentionally returns an empty HeaderMap, allowing upstream configuration to control authentication without hardcoding credentials in the Rust source.

Can I use Azure OpenAI Service with Meetily's custom endpoint feature?

Yes. Azure OpenAI Service implements the OpenAI-compatible /chat/completions endpoint, making it compatible with Meetily's implementation. Configure the base URL to your Azure OpenAI endpoint (e.g., https://{your-resource}.openai.azure.com/openai/deployments/{deployment-id}) in the UI, and the application will append /chat/completions to complete the request path. Ensure your API key has the necessary permissions for the chat completions API.

What timeout values apply to custom endpoint requests?

Meetily applies a consistent 5-minute request timeout to all LLM providers, including custom OpenAI-compatible endpoints. This value is enforced by the reqwest::Client configuration in src-tauri/src/summary/llm_client.rs. The same cancellation token mechanism used for built-in providers also applies, allowing users to abort pending requests if the private infrastructure responds slowly.

Does Meetily support self-hosted models like Ollama or Llama.cpp?

Yes. Any server exposing an OpenAI-compatible HTTP interface—including Ollama with its OpenAI compatibility layer, Llama.cpp's server mode, or vLLM—works with Meetily's CustomOpenAI provider. The only requirement is that the server implements the /chat/completions endpoint with the standard request/response JSON schema defined by OpenAI. Configure the base URL to your local server (e.g., http://localhost:11434/v1 for Ollama) in the application settings.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}