How Meetily's Summary Generation System Supports Multiple LLM Providers (Ollama, Claude, Groq, OpenRouter)

Meetily's summary generation system uses a provider-agnostic abstraction layer in frontend/src-tauri/src/summary/llm_client.rs that unifies cloud-based APIs (OpenAI, Claude, Groq, OpenRouter) and local models (Ollama, Built-in AI) behind a single enum-based interface with provider-specific endpoint selection and request formatting.

Meetily implements a flexible architecture for AI-powered meeting summaries that allows users to choose between cloud LLM services and local self-hosted models without changing the application interface. The system centers on a Rust-based abstraction layer that handles the complexity of different authentication schemes, API endpoints, and request formats. This design enables seamless integration of providers ranging from OpenRouter and Groq to local Ollama instances and built-in sidecar models.

The Core Abstraction: LLMProvider Enum

At the heart of Meetily's multi-provider support lies the LLMProvider enum defined in frontend/src-tauri/src/summary/llm_client.rs. This enumeration covers all supported backends through a unified type system:

pub enum LLMProvider {
    OpenAI,
    Claude,
    Groq,
    Ollama,
    OpenRouter,
    BuiltInAI,
    CustomOpenAI,
}

The system accepts provider identifiers as strings from the UI and parses them case-insensitively using the from_str method. This allows front-end components to pass simple identifiers like "ollama" or "claude" while the backend receives a concrete, type-safe enum value for processing.

Provider-Specific Endpoint and Authentication Handling

The generate_summary function in llm_client.rs selects the correct HTTP endpoint and authentication scheme through a single match block. This architecture handles the distinct requirements of each provider without exposing complexity to upstream callers:

• OpenAI, Groq, and OpenRouter use standard https://…/v1/chat/completions endpoints with Bearer token authentication.
• Claude targets https://api.anthropic.com/v1/messages and requires custom headers including x-api-key and anthropic-version.
• Ollama constructs a local endpoint at http://localhost:11434/v1/chat/completions by default and operates without API key authentication.
• CustomOpenAI supports user-provided endpoints, enabling any OpenAI-compatible server implementation.

Request Construction and Response Parsing

Meetily handles provider-specific payload structures through dedicated request types. For all non-Claude providers, the system uses a generic ChatRequest structure compatible with OpenAI's API format. For Claude, it emits a dedicated ClaudeRequest structure that conforms to Anthropic's message format.

Response parsing follows a similar pattern. The system differentiates between ChatResponse (used by OpenAI-compatible providers) and ClaudeChatResponse (specific to Anthropic), normalizing the output into a consistent format for the summary pipeline. Both cloud and local requests support cancellation via tokio::CancellationToken, allowing users to abort long-running summary generation.

Service Layer Integration

The high-level orchestration occurs in frontend/src-tauri/src/summary/service.rs, which bridges user settings with the underlying LLM client:

1. Retrieves the selected provider from meeting settings.
2. Pulls the appropriate API key (or skips authentication for Ollama and Built-in AI).
3. Fetches provider-specific configuration such as custom Ollama endpoints or custom OpenAI base URLs.
4. Calculates a token_threshold based on the model's context size—dynamic for Ollama, static for Built-in AI, and virtually unlimited for cloud providers.

This layer ensures that provider selection, credential management, and model configuration remain decoupled from the core generation logic.

Built-in AI Sidecar Support

When LLMProvider::BuiltInAI is selected, the system delegates to a local sidecar process running a Llama-compatible model, eliminating network requests entirely. The sidecar client in frontend/src-tauri/src/summary/summary_engine/client.rs handles:

• Model caching to avoid repeated filesystem overhead when loading models.
• Process communication via JSON-encoded Request::Generate messages sent to the sidecar process.
• Cancellation and graceful shutdown through the same token-based mechanism used for cloud providers.

This architecture allows users to generate summaries completely offline while maintaining API compatibility with the rest of the pipeline.

Dynamic Token Thresholds and Chunking

The frontend/src-tauri/src/summary/processor.rs module adapts transcript chunking based on provider-specific context limits. The system calculates appropriate token thresholds dynamically:

• Ollama: Uses dynamic threshold detection based on the specific local model loaded.
• Built-in AI: Applies static context sizes defined in the sidecar configuration.
• Cloud providers: Assumes virtually unlimited context (within practical meeting transcript lengths).

This intelligent chunking ensures optimal use of available context windows across different deployment scenarios.

Code Examples

Selecting a Provider from the React Frontend

import { invoke } from '@tauri-apps/api/tauri';

async function generateSummary(
  meetingId: string,
  provider: string,          // "ollama" | "claude" | "groq" | "openrouter"
  modelName: string,
  systemPrompt: string,
  userPrompt: string,
) {
  const result = await invoke<string>('start_summary_generation', {
    meetingId,
    provider,
    modelName,
    systemPrompt,
    userPrompt,
  });
  return result; // generated summary text
}

The Tauri command start_summary_generation delegates to the Rust implementation in llm_client.rs.

Direct Rust Integration

use reqwest::Client;
use tokio_util::sync::CancellationToken;
use crate::summary::llm_client::{generate_summary, LLMProvider};

let client = Client::new();
let provider = LLMProvider::from_str("ollama").unwrap(); // could be "claude", "groq", etc.
let token = CancellationToken::new();

let summary = generate_summary(
    &client,
    &provider,
    "llama3:8b",
    "",                         // no API key for Ollama
    "Summarize the meeting.",
    "Full transcript text …",
    Some("http://localhost:11434"), // optional custom Ollama endpoint
    None,                           // no custom OpenAI endpoint
    None, None, None,
    None,
    Some(&token),
)
.await?;
println!("Summary: {}", summary);

Using the Built-in AI Sidecar

use crate::summary::summary_engine::client::generate_with_builtin;
use tokio_util::sync::CancellationToken;
use std::path::PathBuf;

let app_data_dir = PathBuf::from("/Users/me/Library/Application Support/Meetily");
let token = CancellationToken::new();

let result = generate_with_builtin(
    &app_data_dir,
    "gemma3:1b",
    "Summarize the meeting.",
    "Full transcript …",
    Some(&token),
).await?;
println!("Built‑in summary: {}", result);

Summary

• Meetily uses a unified LLMProvider enum in llm_client.rs to abstract differences between cloud and local LLM providers.
• The system handles provider-specific endpoints, authentication headers (Bearer tokens vs. x-api-key), and request formats automatically.
• Ollama integration requires no API keys and supports custom local endpoints, while Claude requires specific headers and message formatting.
• The Built-in AI sidecar enables fully offline summary generation through a separate process managed by summary_engine/client.rs.
• Dynamic token thresholds in processor.rs optimize chunking strategies based on whether the provider is cloud-based or local.

Frequently Asked Questions

How does Meetily handle API authentication differently between Claude and OpenAI-compatible providers?

According to the source code in frontend/src-tauri/src/summary/llm_client.rs, OpenAI-compatible providers (including Groq and OpenRouter) use standard Bearer token authentication in the Authorization header. Claude requires custom headers: x-api-key for the API key and anthropic-version to specify the API version. The match block in the endpoint selection logic automatically applies the correct authentication scheme based on the active LLMProvider variant.

Can I use a custom OpenAI-compatible endpoint with Meetily?

Yes. The LLMProvider::CustomOpenAI variant supports any OpenAI-compatible server. When this provider is selected, the system accepts a user-provided base URL through the optional custom_openai_endpoint parameter in generate_summary. This enables integration with private API gateways, proxy servers, or alternative OpenAI-compatible implementations without modifying the core codebase.

What happens when I select Ollama as the provider but haven't started the local server?

The system will attempt to connect to http://localhost:11434/v1/chat/completions (or your custom configured endpoint) and return a connection error if the Ollama server is unavailable. Since Ollama requires no API key authentication, the connection attempt happens immediately during the request phase. The generate_summary function accepts a CancellationToken, allowing users to abort the request if the server is unresponsive.

How does the Built-in AI sidecar differ from cloud-based summary generation?

The Built-in AI provider (LLMProvider::BuiltInAI) delegates to a local sidecar process defined in frontend/src-tauri/src/summary/summary_engine/client.rs rather than making HTTP requests to external APIs. This sidecar runs Llama-compatible models locally, caches model paths to avoid reload overhead, and communicates via inter-process JSON messages. Unlike cloud providers, this approach requires no network connectivity, incurs no API costs, and uses the token_threshold calculation specific to the local model's static context size.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}