Architecture of Meetily's Multi-Provider LLM Integration (Ollama, Claude, Groq, OpenRouter)

Meetily implements a provider-agnostic LLM client architecture centered on an LLMProvider enum and a unified generate_summary function that routes requests to OpenAI-compatible endpoints, Claude's Messages API, or a local sidecar, while automatically handling authentication headers, request payloads, and chunked summarization for context-limited local models.

Meetily's open-source meeting assistant supports multiple Large Language Model backends through a modular Rust architecture that abstracts provider differences behind a single interface. This design allows users to switch between cloud APIs like Anthropic Claude, Groq, and OpenRouter, local Ollama instances, or a built-in AI sidecar without modifying application logic, as implemented in the Zackriya-Solutions/meetily repository.

Core Architecture Overview

The integration lives in frontend/src-tauri/src/summary/llm_client.rs, which exposes a provider-agnostic interface to the rest of the application. The architecture decouples the summarization workflow from transport specifics by using:

• LLMProvider enum – Enumerates all supported backends (OpenAI, Claude, Groq, Ollama, OpenRouter, BuiltInAI, CustomOpenAI).
• generate_summary function – The universal entry point that constructs provider-specific HTTP headers, request payloads, and endpoint URLs.
• processor.rs – Orchestrates single-pass vs. multi-chunk strategies based on provider capabilities.

Provider Enumeration and Request Routing

The LLMProvider enum in llm_client.rs defines each backend and drives conditional logic within generate_summary. The function adapts authentication and payload formats automatically:

Provider	Endpoint Pattern	Authentication Method	Request Payload
OpenAI	https://api.openai.com/v1/chat/completions	Authorization: Bearer <API‑KEY>	ChatRequest (generic)
Claude	https://api.anthropic.com/v1/messages	x‑api‑key header + anthropic‑version	ClaudeRequest (custom)
Groq	https://api.groq.com/openai/v1/chat/completions	Authorization: Bearer <API‑KEY>	ChatRequest (OpenAI-compatible)
OpenRouter	https://openrouter.ai/api/v1/chat/completions	Authorization: Bearer <API‑KEY>	ChatRequest (OpenAI-compatible)
Ollama	http://localhost:11434/v1/chat/completions (configurable)	None required	ChatRequest (generic)
BuiltInAI	Local sidecar binary	None required	Direct function call (generate_with_builtin)
CustomOpenAI	User-provided base URL	Authorization: Bearer <API‑KEY>	ChatRequest (generic)

The generate_summary function implements a 5-minute timeout with optional cancellation token support and transforms provider-specific responses into plain text strings.

Provider-Specific Modules

Each backend has a dedicated module handling model discovery and provider-specific utilities:

Claude (Anthropic)

frontend/src-tauri/src/anthropic/anthropic.rs defines the model catalog (e.g., claude-3-opus-20240229) and provides utilities for Claude-specific features. Unlike OpenAI-compatible APIs, Claude requires the API key in the x-api-key header and mandates an anthropic-version header.

Groq

frontend/src-tauri/src/groq/groq.rs fetches and caches Groq model metadata. Groq uses an OpenAI-compatible API surface, so it reuses the generic ChatRequest payload, but the module handles API-key-less fallbacks and model list resolution.

OpenRouter

frontend/src-tauri/src/openrouter/openrouter.rs resolves model lists from the OpenRouter service. It follows the same OpenAI-compatible request pattern as Groq but uses the provider name for logging and metadata tracking.

Ollama

frontend/src-tauri/src/ollama/ollama.rs discovers locally-hosted Ollama models, validates the endpoint (defaulting to http://localhost:11434), and provides CLI fallback mechanisms when the HTTP API is unavailable.

Built-in AI Sidecar

The BuiltInAI variant bypasses HTTP entirely. When selected, generate_summary invokes generate_with_builtin directly, communicating with a local sidecar binary that requires no API tokens or network connectivity.

Chunking Strategy: Cloud vs. Local Processing

The frontend/src-tauri/src/summary/processor.rs module decides whether to use single-pass or multi-chunk summarization based on the selected provider's context window capabilities:

• Cloud providers (OpenAI, Claude, Groq, OpenRouter, CustomOpenAI): The entire transcript is sent in one request because these services support large context windows.
• Local providers (Ollama, BuiltInAI): The transcript is split into overlapping chunks using a token threshold (typically token_threshold - 300 with 100-token overlap). Each chunk is summarized via separate generate_summary calls, and the partial results are recombined into a final markdown report with "\n---\n" separators.

This chunking logic ensures local models with limited context windows can still process long meetings without truncation.

Implementation Examples

Calling the Generic Summarizer from Rust

use crate::summary::llm_client::{generate_summary, LLMProvider};
use reqwest::Client;
use tokio_util::sync::CancellationToken;

// Example: Summarise with Claude
let client = Client::new();
let provider = LLMProvider::Claude;
let model = "claude-3-opus-20240229";
let api_key = "YOUR_ANTHROPIC_API_KEY";
let system_prompt = "You are a concise meeting summariser.";
let user_prompt = "Transcript text …";

let result = generate_summary(
    &client,
    &provider,
    model,
    api_key,
    system_prompt,
    user_prompt,
    None,               // no custom Ollama endpoint
    None,               // no custom OpenAI endpoint
    None, None, None,   // optional max_tokens, temperature, top_p
    None,               // no app_data_dir needed for Claude
    None,               // no cancellation token
).await;

Invoking from the Frontend via Tauri

import { invoke } from '@tauri-apps/api/tauri';

async function summarizeWithOllama(transcript: string) {
  const summary = await invoke<string>('generate_summary', {
    provider: 'ollama',
    modelName: 'llama3:8b',
    apiKey: '',                     // not required for Ollama
    systemPrompt: 'Summarise the meeting.',
    userPrompt: transcript,
    ollamaEndpoint: 'http://localhost:11434',
  });
  console.log(summary);
}

Internal Chunked Summarization Logic

// Inside processor.rs – split transcript, call generate_summary per chunk
let chunks = chunk_text(transcript, token_threshold - 300, 100);
let mut partials = Vec::new();
for chunk in chunks {
    let chunk_summary = generate_summary(...).await?;
    partials.push(chunk_summary);
}
let combined = partials.join("\n---\n");
let final_summary = generate_summary(..., &combined, ...).await?;

Key Source Files

The following files constitute Meetily's LLM integration layer:

• frontend/src-tauri/src/summary/llm_client.rs – Central provider enum, request construction, and response parsing logic.
• frontend/src-tauri/src/summary/processor.rs – Orchestrates single-pass vs. multi-chunk summarization and result combination.
• frontend/src-tauri/src/anthropic/anthropic.rs – Claude model catalog and Anthropic-specific utilities.
• frontend/src-tauri/src/groq/groq.rs – Groq model fetching, caching, and fallback mechanisms.
• frontend/src-tauri/src/openrouter/openrouter.rs – OpenRouter model discovery and metadata handling.
• frontend/src-tauri/src/ollama/ollama.rs – Local Ollama model discovery, endpoint validation, and CLI integration.
• frontend/src-tauri/src/summary/mod.rs – Public API exports exposing generate_summary and LLMProvider to the Tauri command layer.

Summary

• Unified Interface: The generate_summary function in llm_client.rs abstracts differences between seven distinct LLM providers behind a single async function signature.
• Authentication Abstraction: The architecture automatically switches between Bearer tokens (OpenAI, Groq, OpenRouter), custom headers (Claude's x-api-key), and no authentication (Ollama, BuiltInAI).
• Payload Adaptation: Claude uses a custom ClaudeRequest struct, while all other providers consume the generic ChatRequest compatible with OpenAI's schema.
• Intelligent Chunking: The processor.rs module automatically splits long transcripts for local providers (Ollama, BuiltInAI) while sending full context to cloud APIs.
• Extensibility: The CustomOpenAI variant allows users to plug in any self-hosted OpenAI-compatible server without code changes.

Frequently Asked Questions

How does Meetily handle different authentication schemes across LLM providers?

Meetily's generate_summary function inspects the LLMProvider enum variant to construct the appropriate HTTP headers. For Claude, it sends the x-api-key and anthropic-version headers; for OpenAI, Groq, OpenRouter, and CustomOpenAI, it uses the standard Authorization: Bearer <token> header; and for Ollama and BuiltInAI, it omits authentication entirely.

What is the difference between single-pass and chunked summarization in Meetily?

Single-pass summarization sends the entire meeting transcript to cloud providers (OpenAI, Claude, Groq, OpenRouter) in one HTTP request, leveraging their large context windows. Chunked summarization, used for Ollama and BuiltInAI, splits the transcript into smaller segments using a sliding window approach, summarizes each chunk individually, then combines the partial summaries into a final coherent report.

Can I use a custom OpenAI-compatible API endpoint with Meetily?

Yes. The LLMProvider::CustomOpenAI variant allows you to specify any base URL (such as a self-hosted LiteLLM or vLLM server) through the custom_openai_endpoint parameter in generate_summary. This option uses the standard ChatRequest payload and Bearer token authentication, making it compatible with any OpenAI API-compliant service.

How does the built-in AI sidecar differ from cloud providers in Meetily's architecture?

The BuiltInAI provider bypasses HTTP networking entirely. Instead of constructing web requests, generate_summary calls generate_with_builtin directly to communicate with a local sidecar binary. This approach requires no API keys, functions offline, and avoids the latency associated with cloud round-trips, though it typically requires chunked processing for longer transcripts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}