# Architecture of Meetily's Multi-Provider LLM Integration (Ollama, Claude, Groq, OpenRouter) > Discover Meetily's multi-provider LLM architecture integrating Ollama, Claude, Groq, and OpenRouter. Learn how it handles authentication and chunked summarization for diverse LLM models. - Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily) - Tags: architecture - Published: 2026-07-26 --- **Meetily implements a provider-agnostic LLM client architecture centered on an `LLMProvider` enum and a unified `generate_summary` function that routes requests to OpenAI-compatible endpoints, Claude's Messages API, or a local sidecar, while automatically handling authentication headers, request payloads, and chunked summarization for context-limited local models.** Meetily's open-source meeting assistant supports multiple Large Language Model backends through a modular Rust architecture that abstracts provider differences behind a single interface. This design allows users to switch between cloud APIs like **Anthropic Claude**, **Groq**, and **OpenRouter**, local **Ollama** instances, or a built-in AI sidecar without modifying application logic, as implemented in the [Zackriya-Solutions/meetily](https://github.com/Zackriya-Solutions/meetily) repository. ## Core Architecture Overview The integration lives in **[`frontend/src-tauri/src/summary/llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/llm_client.rs)**, which exposes a provider-agnostic interface to the rest of the application. The architecture decouples the summarization workflow from transport specifics by using: - **`LLMProvider` enum** – Enumerates all supported backends (`OpenAI`, `Claude`, `Groq`, `Ollama`, `OpenRouter`, `BuiltInAI`, `CustomOpenAI`). - **`generate_summary` function** – The universal entry point that constructs provider-specific HTTP headers, request payloads, and endpoint URLs. - **[`processor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/processor.rs)** – Orchestrates single-pass vs. multi-chunk strategies based on provider capabilities. ## Provider Enumeration and Request Routing The `LLMProvider` enum in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs) defines each backend and drives conditional logic within `generate_summary`. The function adapts authentication and payload formats automatically: | Provider | Endpoint Pattern | Authentication Method | Request Payload | |----------|------------------|----------------------|-----------------| | **OpenAI** | `https://api.openai.com/v1/chat/completions` | `Authorization: Bearer ` | `ChatRequest` (generic) | | **Claude** | `https://api.anthropic.com/v1/messages` | `x‑api‑key` header + `anthropic‑version` | `ClaudeRequest` (custom) | | **Groq** | `https://api.groq.com/openai/v1/chat/completions` | `Authorization: Bearer ` | `ChatRequest` (OpenAI-compatible) | | **OpenRouter** | `https://openrouter.ai/api/v1/chat/completions` | `Authorization: Bearer ` | `ChatRequest` (OpenAI-compatible) | | **Ollama** | `http://localhost:11434/v1/chat/completions` (configurable) | None required | `ChatRequest` (generic) | | **BuiltInAI** | Local sidecar binary | None required | Direct function call (`generate_with_builtin`) | | **CustomOpenAI** | User-provided base URL | `Authorization: Bearer ` | `ChatRequest` (generic) | The `generate_summary` function implements a 5-minute timeout with optional cancellation token support and transforms provider-specific responses into plain text strings. ## Provider-Specific Modules Each backend has a dedicated module handling model discovery and provider-specific utilities: ### Claude (Anthropic) **[`frontend/src-tauri/src/anthropic/anthropic.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/anthropic/anthropic.rs)** defines the model catalog (e.g., `claude-3-opus-20240229`) and provides utilities for Claude-specific features. Unlike OpenAI-compatible APIs, Claude requires the API key in the `x-api-key` header and mandates an `anthropic-version` header. ### Groq **[`frontend/src-tauri/src/groq/groq.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/groq/groq.rs)** fetches and caches Groq model metadata. Groq uses an OpenAI-compatible API surface, so it reuses the generic `ChatRequest` payload, but the module handles API-key-less fallbacks and model list resolution. ### OpenRouter **[`frontend/src-tauri/src/openrouter/openrouter.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/openrouter/openrouter.rs)** resolves model lists from the OpenRouter service. It follows the same OpenAI-compatible request pattern as Groq but uses the provider name for logging and metadata tracking. ### Ollama **[`frontend/src-tauri/src/ollama/ollama.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/ollama/ollama.rs)** discovers locally-hosted Ollama models, validates the endpoint (defaulting to `http://localhost:11434`), and provides CLI fallback mechanisms when the HTTP API is unavailable. ### Built-in AI Sidecar The **BuiltInAI** variant bypasses HTTP entirely. When selected, `generate_summary` invokes `generate_with_builtin` directly, communicating with a local sidecar binary that requires no API tokens or network connectivity. ## Chunking Strategy: Cloud vs. Local Processing The **[`frontend/src-tauri/src/summary/processor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/processor.rs)** module decides whether to use single-pass or multi-chunk summarization based on the selected provider's context window capabilities: - **Cloud providers** (OpenAI, Claude, Groq, OpenRouter, CustomOpenAI): The entire transcript is sent in one request because these services support large context windows. - **Local providers** (Ollama, BuiltInAI): The transcript is split into overlapping chunks using a token threshold (typically `token_threshold - 300` with 100-token overlap). Each chunk is summarized via separate `generate_summary` calls, and the partial results are recombined into a final markdown report with `"\n---\n"` separators. This chunking logic ensures local models with limited context windows can still process long meetings without truncation. ## Implementation Examples ### Calling the Generic Summarizer from Rust ```rust use crate::summary::llm_client::{generate_summary, LLMProvider}; use reqwest::Client; use tokio_util::sync::CancellationToken; // Example: Summarise with Claude let client = Client::new(); let provider = LLMProvider::Claude; let model = "claude-3-opus-20240229"; let api_key = "YOUR_ANTHROPIC_API_KEY"; let system_prompt = "You are a concise meeting summariser."; let user_prompt = "Transcript text …"; let result = generate_summary( &client, &provider, model, api_key, system_prompt, user_prompt, None, // no custom Ollama endpoint None, // no custom OpenAI endpoint None, None, None, // optional max_tokens, temperature, top_p None, // no app_data_dir needed for Claude None, // no cancellation token ).await; ``` ### Invoking from the Frontend via Tauri ```typescript import { invoke } from '@tauri-apps/api/tauri'; async function summarizeWithOllama(transcript: string) { const summary = await invoke('generate_summary', { provider: 'ollama', modelName: 'llama3:8b', apiKey: '', // not required for Ollama systemPrompt: 'Summarise the meeting.', userPrompt: transcript, ollamaEndpoint: 'http://localhost:11434', }); console.log(summary); } ``` ### Internal Chunked Summarization Logic ```rust // Inside processor.rs – split transcript, call generate_summary per chunk let chunks = chunk_text(transcript, token_threshold - 300, 100); let mut partials = Vec::new(); for chunk in chunks { let chunk_summary = generate_summary(...).await?; partials.push(chunk_summary); } let combined = partials.join("\n---\n"); let final_summary = generate_summary(..., &combined, ...).await?; ``` ## Key Source Files The following files constitute Meetily's LLM integration layer: - **[`frontend/src-tauri/src/summary/llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/llm_client.rs)** – Central provider enum, request construction, and response parsing logic. - **[`frontend/src-tauri/src/summary/processor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/processor.rs)** – Orchestrates single-pass vs. multi-chunk summarization and result combination. - **[`frontend/src-tauri/src/anthropic/anthropic.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/anthropic/anthropic.rs)** – Claude model catalog and Anthropic-specific utilities. - **[`frontend/src-tauri/src/groq/groq.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/groq/groq.rs)** – Groq model fetching, caching, and fallback mechanisms. - **[`frontend/src-tauri/src/openrouter/openrouter.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/openrouter/openrouter.rs)** – OpenRouter model discovery and metadata handling. - **[`frontend/src-tauri/src/ollama/ollama.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/ollama/ollama.rs)** – Local Ollama model discovery, endpoint validation, and CLI integration. - **[`frontend/src-tauri/src/summary/mod.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/mod.rs)** – Public API exports exposing `generate_summary` and `LLMProvider` to the Tauri command layer. ## Summary - **Unified Interface**: The `generate_summary` function in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs) abstracts differences between seven distinct LLM providers behind a single async function signature. - **Authentication Abstraction**: The architecture automatically switches between Bearer tokens (OpenAI, Groq, OpenRouter), custom headers (Claude's `x-api-key`), and no authentication (Ollama, BuiltInAI). - **Payload Adaptation**: Claude uses a custom `ClaudeRequest` struct, while all other providers consume the generic `ChatRequest` compatible with OpenAI's schema. - **Intelligent Chunking**: The [`processor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/processor.rs) module automatically splits long transcripts for local providers (Ollama, BuiltInAI) while sending full context to cloud APIs. - **Extensibility**: The `CustomOpenAI` variant allows users to plug in any self-hosted OpenAI-compatible server without code changes. ## Frequently Asked Questions ### How does Meetily handle different authentication schemes across LLM providers? Meetily's `generate_summary` function inspects the `LLMProvider` enum variant to construct the appropriate HTTP headers. For **Claude**, it sends the `x-api-key` and `anthropic-version` headers; for **OpenAI**, **Groq**, **OpenRouter**, and **CustomOpenAI**, it uses the standard `Authorization: Bearer ` header; and for **Ollama** and **BuiltInAI**, it omits authentication entirely. ### What is the difference between single-pass and chunked summarization in Meetily? Single-pass summarization sends the entire meeting transcript to cloud providers (OpenAI, Claude, Groq, OpenRouter) in one HTTP request, leveraging their large context windows. Chunked summarization, used for **Ollama** and **BuiltInAI**, splits the transcript into smaller segments using a sliding window approach, summarizes each chunk individually, then combines the partial summaries into a final coherent report. ### Can I use a custom OpenAI-compatible API endpoint with Meetily? Yes. The `LLMProvider::CustomOpenAI` variant allows you to specify any base URL (such as a self-hosted **LiteLLM** or **vLLM** server) through the `custom_openai_endpoint` parameter in `generate_summary`. This option uses the standard `ChatRequest` payload and Bearer token authentication, making it compatible with any OpenAI API-compliant service. ### How does the built-in AI sidecar differ from cloud providers in Meetily's architecture? The **BuiltInAI** provider bypasses HTTP networking entirely. Instead of constructing web requests, `generate_summary` calls `generate_with_builtin` directly to communicate with a local sidecar binary. This approach requires no API keys, functions offline, and avoids the latency associated with cloud round-trips, though it typically requires chunked processing for longer transcripts.