# How Meetily's Summary Generation System Supports Multiple LLM Providers (Ollama, Claude, Groq, OpenRouter)

> Meetily's summary generation system supports multiple LLM providers like Ollama, Claude, Groq, and OpenRouter via a provider-agnostic abstraction layer. Integrate local and cloud models seamlessly.

- Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily)
- Tags: internals
- Published: 2026-07-06

---

**Meetily's summary generation system uses a provider-agnostic abstraction layer in [`frontend/src-tauri/src/summary/llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/llm_client.rs) that unifies cloud-based APIs (OpenAI, Claude, Groq, OpenRouter) and local models (Ollama, Built-in AI) behind a single enum-based interface with provider-specific endpoint selection and request formatting.**

Meetily implements a flexible architecture for AI-powered meeting summaries that allows users to choose between cloud LLM services and local self-hosted models without changing the application interface. The system centers on a Rust-based abstraction layer that handles the complexity of different authentication schemes, API endpoints, and request formats. This design enables seamless integration of providers ranging from OpenRouter and Groq to local Ollama instances and built-in sidecar models.

## The Core Abstraction: LLMProvider Enum

At the heart of Meetily's multi-provider support lies the `LLMProvider` enum defined in [`frontend/src-tauri/src/summary/llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/llm_client.rs). This enumeration covers all supported backends through a unified type system:

```rust
pub enum LLMProvider {
    OpenAI,
    Claude,
    Groq,
    Ollama,
    OpenRouter,
    BuiltInAI,
    CustomOpenAI,
}

```

The system accepts provider identifiers as strings from the UI and parses them case-insensitively using the `from_str` method. This allows front-end components to pass simple identifiers like `"ollama"` or `"claude"` while the backend receives a concrete, type-safe enum value for processing.

## Provider-Specific Endpoint and Authentication Handling

The `generate_summary` function in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs) selects the correct HTTP endpoint and authentication scheme through a single match block. This architecture handles the distinct requirements of each provider without exposing complexity to upstream callers:

- **OpenAI, Groq, and OpenRouter** use standard `https://…/v1/chat/completions` endpoints with Bearer token authentication.
- **Claude** targets `https://api.anthropic.com/v1/messages` and requires custom headers including `x-api-key` and `anthropic-version`.
- **Ollama** constructs a local endpoint at `http://localhost:11434/v1/chat/completions` by default and operates without API key authentication.
- **CustomOpenAI** supports user-provided endpoints, enabling any OpenAI-compatible server implementation.

## Request Construction and Response Parsing

Meetily handles provider-specific payload structures through dedicated request types. For all non-Claude providers, the system uses a generic `ChatRequest` structure compatible with OpenAI's API format. For Claude, it emits a dedicated `ClaudeRequest` structure that conforms to Anthropic's message format.

Response parsing follows a similar pattern. The system differentiates between `ChatResponse` (used by OpenAI-compatible providers) and `ClaudeChatResponse` (specific to Anthropic), normalizing the output into a consistent format for the summary pipeline. Both cloud and local requests support cancellation via `tokio::CancellationToken`, allowing users to abort long-running summary generation.

## Service Layer Integration

The high-level orchestration occurs in [`frontend/src-tauri/src/summary/service.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/service.rs), which bridges user settings with the underlying LLM client:

1. Retrieves the selected provider from meeting settings.
2. Pulls the appropriate API key (or skips authentication for Ollama and Built-in AI).
3. Fetches provider-specific configuration such as custom Ollama endpoints or custom OpenAI base URLs.
4. Calculates a `token_threshold` based on the model's context size—dynamic for Ollama, static for Built-in AI, and virtually unlimited for cloud providers.

This layer ensures that provider selection, credential management, and model configuration remain decoupled from the core generation logic.

## Built-in AI Sidecar Support

When `LLMProvider::BuiltInAI` is selected, the system delegates to a local sidecar process running a Llama-compatible model, eliminating network requests entirely. The sidecar client in [`frontend/src-tauri/src/summary/summary_engine/client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/summary_engine/client.rs) handles:

- **Model caching** to avoid repeated filesystem overhead when loading models.
- **Process communication** via JSON-encoded `Request::Generate` messages sent to the sidecar process.
- **Cancellation and graceful shutdown** through the same token-based mechanism used for cloud providers.

This architecture allows users to generate summaries completely offline while maintaining API compatibility with the rest of the pipeline.

## Dynamic Token Thresholds and Chunking

The [`frontend/src-tauri/src/summary/processor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/processor.rs) module adapts transcript chunking based on provider-specific context limits. The system calculates appropriate token thresholds dynamically:

- **Ollama**: Uses dynamic threshold detection based on the specific local model loaded.
- **Built-in AI**: Applies static context sizes defined in the sidecar configuration.
- **Cloud providers**: Assumes virtually unlimited context (within practical meeting transcript lengths).

This intelligent chunking ensures optimal use of available context windows across different deployment scenarios.

## Code Examples

### Selecting a Provider from the React Frontend

```typescript
import { invoke } from '@tauri-apps/api/tauri';

async function generateSummary(
  meetingId: string,
  provider: string,          // "ollama" | "claude" | "groq" | "openrouter"
  modelName: string,
  systemPrompt: string,
  userPrompt: string,
) {
  const result = await invoke<string>('start_summary_generation', {
    meetingId,
    provider,
    modelName,
    systemPrompt,
    userPrompt,
  });
  return result; // generated summary text
}

```

The Tauri command `start_summary_generation` delegates to the Rust implementation in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs).

### Direct Rust Integration

```rust
use reqwest::Client;
use tokio_util::sync::CancellationToken;
use crate::summary::llm_client::{generate_summary, LLMProvider};

let client = Client::new();
let provider = LLMProvider::from_str("ollama").unwrap(); // could be "claude", "groq", etc.
let token = CancellationToken::new();

let summary = generate_summary(
    &client,
    &provider,
    "llama3:8b",
    "",                         // no API key for Ollama
    "Summarize the meeting.",
    "Full transcript text …",
    Some("http://localhost:11434"), // optional custom Ollama endpoint
    None,                           // no custom OpenAI endpoint
    None, None, None,
    None,
    Some(&token),
)
.await?;
println!("Summary: {}", summary);

```

### Using the Built-in AI Sidecar

```rust
use crate::summary::summary_engine::client::generate_with_builtin;
use tokio_util::sync::CancellationToken;
use std::path::PathBuf;

let app_data_dir = PathBuf::from("/Users/me/Library/Application Support/Meetily");
let token = CancellationToken::new();

let result = generate_with_builtin(
    &app_data_dir,
    "gemma3:1b",
    "Summarize the meeting.",
    "Full transcript …",
    Some(&token),
).await?;
println!("Built‑in summary: {}", result);

```

## Summary

- Meetily uses a unified `LLMProvider` enum in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs) to abstract differences between cloud and local LLM providers.
- The system handles provider-specific endpoints, authentication headers (Bearer tokens vs. `x-api-key`), and request formats automatically.
- **Ollama** integration requires no API keys and supports custom local endpoints, while **Claude** requires specific headers and message formatting.
- The **Built-in AI** sidecar enables fully offline summary generation through a separate process managed by [`summary_engine/client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/summary_engine/client.rs).
- Dynamic token thresholds in [`processor.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/processor.rs) optimize chunking strategies based on whether the provider is cloud-based or local.

## Frequently Asked Questions

### How does Meetily handle API authentication differently between Claude and OpenAI-compatible providers?

According to the source code in [`frontend/src-tauri/src/summary/llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/llm_client.rs), OpenAI-compatible providers (including Groq and OpenRouter) use standard Bearer token authentication in the `Authorization` header. Claude requires custom headers: `x-api-key` for the API key and `anthropic-version` to specify the API version. The match block in the endpoint selection logic automatically applies the correct authentication scheme based on the active `LLMProvider` variant.

### Can I use a custom OpenAI-compatible endpoint with Meetily?

Yes. The `LLMProvider::CustomOpenAI` variant supports any OpenAI-compatible server. When this provider is selected, the system accepts a user-provided base URL through the optional `custom_openai_endpoint` parameter in `generate_summary`. This enables integration with private API gateways, proxy servers, or alternative OpenAI-compatible implementations without modifying the core codebase.

### What happens when I select Ollama as the provider but haven't started the local server?

The system will attempt to connect to `http://localhost:11434/v1/chat/completions` (or your custom configured endpoint) and return a connection error if the Ollama server is unavailable. Since Ollama requires no API key authentication, the connection attempt happens immediately during the request phase. The `generate_summary` function accepts a `CancellationToken`, allowing users to abort the request if the server is unresponsive.

### How does the Built-in AI sidecar differ from cloud-based summary generation?

The Built-in AI provider (`LLMProvider::BuiltInAI`) delegates to a local sidecar process defined in [`frontend/src-tauri/src/summary/summary_engine/client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/summary_engine/client.rs) rather than making HTTP requests to external APIs. This sidecar runs Llama-compatible models locally, caches model paths to avoid reload overhead, and communicates via inter-process JSON messages. Unlike cloud providers, this approach requires no network connectivity, incurs no API costs, and uses the `token_threshold` calculation specific to the local model's static context size.