How Ollama, Claude, Groq, and OpenRouter Differ for Summary Generation in Meetily

Ollama operates locally without authentication and supports larger context windows (≈8000 tokens), while Claude, Groq, and OpenRouter require API keys, utilize cloud endpoints with 4000-token thresholds, and differ in request payload structures and model discovery mechanisms.

Meetily's meeting summarization engine abstracts multiple large language model backends through a unified Rust client architecture. The system supports Ollama, Claude, Groq, and OpenRouter as distinct providers for summary generation in Meetily, each with unique authentication requirements, request formats, and performance characteristics optimized for different deployment scenarios.

Core Architecture and Provider Abstraction

The summarization pipeline centers on the LLMProvider enum defined in src-tauri/src/summary/llm_client.rs at line 68:

pub enum LLMProvider {
    OpenAI, Claude, Groq, OpenRouter,
    Ollama, BuiltInAI, CustomOpenAI,
}

During a summary request, the service layer (src-tauri/src/summary/service.rs) resolves the selected provider from the UI configuration, builds the appropriate HTTP payload, injects authentication headers when required, and applies token thresholds that determine whether the transcript must be chunked before sending.

Deployment Models and Authentication

Ollama (Local Deployment)

Ollama runs as a local HTTP server (default endpoint http://127.0.0.1:11434) with no network hop required. The client disables the Authorization header entirely for this provider. This configuration is preferred for offline, privacy-first operation when a local GPU is available.

Cloud Providers (Claude, Groq, and OpenRouter)

All three cloud providers require API keys stored in the application's ConfigContext:

• Claude: Requires an Anthropic API key; uses the endpoint https://api.anthropic.com/v1
• Groq: Requires a Groq API key; uses the OpenAI-compatible endpoint https://api.groq.com/openai/v1
• OpenRouter: Requires an OpenRouter API key; uses https://openrouter.ai/api/v1 with additional headers (X-Title)

The UI handles missing credentials differently for each: Ollama errors are detected via isOllamaNotInstalledError in frontend/src/lib/utils.ts, while cloud providers trigger toast notifications for missing API keys within ModelSettingsModal.tsx.

Request Format and Payload Structure

The llm_client.rs file branches request construction based on the provider enum at line 220:

OpenAI-Compatible Providers (Groq, OpenRouter, BuiltInAI):

let request_body = serde_json::json!({
    "model": model,
    "messages": messages,
    "max_tokens": max_tokens_val,
    // ...
});

Claude-Specific Format: Claude uses distinct structs (ClaudeRequest / ClaudeChatResponse defined at lines 46-62) and includes the anthropic-version header inside the request body rather than the standard Authorization bearer token. The client specifically excludes the Authorization header for Claude at lines 203-204:

if provider != &LLMProvider::Claude {
    request_builder = request_builder.header("Authorization", format!("Bearer {}", api_key));
}

Ollama-Specific Format: Ollama uses its own JSON schema with model, messages, and stream parameters, handled by the local command interface in src-tauri/src/ollama/ollama.rs.

Token Thresholds and Context Handling

The service applies different token limits based on provider capabilities (line 394 in service.rs):

• Ollama: Uses a threshold of approximately 8000 tokens because the local server can handle larger contexts without the latency penalties of cloud APIs
• Claude, Groq, OpenRouter: Use a threshold of approximately 4000 tokens to stay within API limits and manage cloud latency

When transcripts exceed these thresholds, the service automatically chunks the content before sending parallel requests.

Model Discovery and UI Integration

The frontend handles model lists differently for each provider in frontend/src/components/ModelSettingsModal.tsx:

• Ollama: Dynamically queries the local installation via invoke('get_ollama_models'), which executes the command defined in src-tauri/src/ollama/ollama.rs
• Claude: Fetches available models from Anthropic's model endpoint via loadClaudeModels (around line 544)
• Groq and OpenRouter: Display hard-coded model lists maintained within the ModelSettingsModal.tsx component itself

Practical Implementation Examples

Selecting a Provider Programmatically

import { invoke } from '@tauri-apps/api/tauri';

async function generateSummary(transcript: string, provider: string) {
  // provider: 'ollama' | 'claude' | 'groq' | 'openrouter'
  const result = await invoke<string>('generate_summary', {
    transcript,
    modelProvider: provider,
  });
  return result;
}

Validating Ollama Before Generation

async function checkOllamaModelsAndGenerate(endpoint: string) {
  const models = await invoke<OllamaModel[]>('get_ollama_models', { endpoint });
  if (models.length === 0) {
    toast.error('No Ollama models found', {
      description: 'Download a model from Settings first.',
    });
    return;
  }
  // Proceed with generation
}

Provider Selection in React Components

<Select
  value={modelConfig.provider}
  onChange={e => updateConfig({ provider: e.target.value })}>
  <SelectItem value="ollama">Ollama (Local)</SelectItem>
  <SelectItem value="claude">Claude (Anthropic)</SelectItem>
  <SelectItem value="groq">Groq</SelectItem>
  <SelectItem value="openrouter">OpenRouter</SelectItem>
</Select>

Summary

• Ollama requires no API key, runs locally at 127.0.0.1:11434, uses Ollama-specific JSON payloads, supports ≈8000 tokens before chunking, and discovers models via the local get_ollama_models command
• Claude requires an Anthropic API key, uses Claude-specific request structs (ClaudeRequest), excludes standard Authorization headers in favor of anthropic-version, and fetches models from Anthropic's API
• Groq requires a Groq API key, uses standard OpenAI-compatible JSON schemas, applies a 4000-token threshold, and displays a hard-coded model list
• OpenRouter requires an OpenRouter API key, uses OpenAI-compatible schemas with additional headers (X-Title), applies a 4000-token threshold, and displays a hard-coded model list optimized for marketplace routing

Frequently Asked Questions

Which provider offers the best privacy for sensitive meeting transcripts?

Ollama provides the strongest privacy guarantees because it processes all data locally on your machine without transmitting transcripts to external servers. As implemented in src-tauri/src/ollama/ollama.rs, the Ollama provider communicates only with your local HTTP server at 127.0.0.1:11434, ensuring no API keys or meeting content leave your device.

Why does Meetily use different token thresholds for Ollama versus cloud providers?

The service configures a higher token threshold (≈8000) for Ollama because local inference eliminates network latency and API rate limits, allowing efficient processing of larger context windows in a single request. Cloud providers (Claude, Groq, OpenRouter) use a lower threshold (≈4000) to minimize API costs, reduce latency, and respect provider-specific context limits as configured in src-tauri/src/summary/service.rs at line 394.

How does Meetily handle authentication differently between Claude and other cloud providers?

Claude uses a unique authentication mechanism where the client omits the standard Authorization header entirely and instead passes the API key within a custom anthropic-version header inside the ClaudeRequest struct. In contrast, Groq and OpenRouter receive the standard Authorization: Bearer {api_key} header as shown in the conditional logic at lines 203-204 of llm_client.rs.

Can I use OpenRouter to access Claude models within Meetily?

Yes, OpenRouter acts as an aggregation marketplace that can route requests to Anthropic's Claude models among others. When using OpenRouter, Meetily sends standard OpenAI-compatible JSON payloads with the additional X-Title header to identify the application, allowing you to access Claude models through OpenRouter's unified API rather than directly through Anthropic's native endpoint.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}