# How the Meetily Summary Generation Engine Integrates with Ollama, Claude, Groq, and OpenRouter

> Discover how the Meetily summary generation engine seamlessly integrates with Ollama, Claude, Groq, and OpenRouter using a unified function for dynamic LLM provider support.

- Repository: [Zackriya Solutions/meetily](https://github.com/Zackriya-Solutions/meetily)
- Tags: deep-dive
- Published: 2026-07-19

---

**The Meetily summary generation engine integrates with LLM providers through a single `generate_summary` function in [`frontend/src-tauri/src/summary/llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/llm_client.rs), dynamically switching request formats, authentication headers, and endpoints to support Ollama, Claude, Groq, OpenRouter, and a built-in local AI sidecar.**

The Zackriya-Solutions/meetily repository implements this Rust-based dispatch layer inside the `frontend/src-tauri/src/summary` subsystem. Whether the front end selects a local Ollama instance or a remote Claude API key, the engine manages serialization, HTTP dispatch, and response parsing without UI changes. This article walks through the exact integration mechanisms—from enum dispatch to sidecar fallback—based on the current source tree.

## Provider Selection and Runtime Dispatch

At runtime, the front end passes a provider string such as `"ollama"` or `"claude"`. The engine converts this string into a typed variant via `LLMProvider::from_str` defined in [`frontend/src-tauri/src/summary/llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/llm_client.rs).

This enum drives every downstream decision, including which request struct to build, which headers to attach, and which URL to target. Because the provider is resolved before any network call, adding a new backend requires changes only inside the summary subsystem.

## Request Builder and Provider Types

The engine maintains two request families: one for OpenAI-compatible providers and one for Anthropic Claude.

### OpenAI-Compatible Providers

**Ollama**, **Groq**, **OpenRouter**, and **CustomOpenAI** all use the `ChatRequest` struct. This JSON payload carries `model`, `messages`, and optional generation parameters. The definition lives in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs) around lines 18-28 and is reused across all four backends by serializing the same struct to different endpoints.

### Claude

**Claude** deviates from the OpenAI shape. The engine constructs a `ClaudeRequest` struct that includes a top-level `system` field and a fixed `max_tokens` value. This format aligns with Anthropic’s `/v1/messages` endpoint rather than the standard chat-completions path.

## Authentication and Endpoint Routing

Each provider carries its own URL and header scheme, hard-coded or configured in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs).

- **OpenAI** targets `https://api.openai.com/v1/chat/completions`
- **Groq** targets `https://api.groq.com/openai/v1/chat/completions`
- **OpenRouter** targets `https://openrouter.ai/api/v1/chat/completions`
- **Ollama** defaults to `http://localhost:11434/v1/chat/completions` but accepts a custom override
- **CustomOpenAI** uses a user-supplied base URL for self-hosted OpenAI-compatible servers

For authentication, the engine injects `Authorization: Bearer <api_key>` for every remote provider except Claude. Claude instead requires `x-api-key` and `anthropic-version` headers. Ollama typically omits the header entirely when running locally without API key validation.

## HTTP Transport with Cancellation

Every outbound request is wrapped in an async `tokio::select!` block. The engine accepts a `CancellationToken` from the caller, allowing the summary generation to abort immediately if the user dismisses the dialog or closes the window.

This pattern appears in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs) between lines 66-82. The timeout and cancellation logic is provider-agnostic, meaning Groq and OpenRouter requests receive the same lifecycle management as a local Ollama call.

## Response Parsing

After the HTTP client returns bytes, the engine deserializes the payload based on the active provider.

For Claude, the JSON maps to `ClaudeChatResponse`, and the extractor returns the first text block in the content array. For all other providers, the engine uses the generic `ChatResponse` shape and extracts `choices[0].message.content`. These two parsers live in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs) around lines 100-124.

## Built-In AI Sidecar

When the provider variant is `BuiltInAI`, the engine bypasses HTTP entirely. It communicates with a local sidecar process running a Llama-style model.

This path is orchestrated by [`frontend/src-tauri/src/summary/summary_engine/client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/summary_engine/client.rs). The sidecar client manages process lifecycle and exchanges messages over local IPC, offering fully offline summarization without an external API key.

## Supporting Modules for Model Discovery

Before a summary request fires, the UI often queries available models. Meetily maintains provider-specific helpers for this.

### Ollama Model Management

The file [`frontend/src-tauri/src/ollama/ollama.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/ollama/ollama.rs) exposes `get_ollama_models`, `pull_ollama_model`, `delete_ollama_model`, and `get_ollama_model_context`. These commands populate the dropdown and ensure the correct model weights are present locally before the engine attempts inference.

### Groq Model Caching

[`frontend/src-tauri/src/groq/groq.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/groq/groq.rs) provides `get_groq_models`, which lists chat-capable Groq models and caches the result for five minutes. The Tauri command layer re-exports this to the front end.

### OpenRouter Catalog

[`frontend/src-tauri/src/openrouter/openrouter.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/openrouter/openrouter.rs) implements `get_openrouter_models` via a blocking HTTP call to the OpenRouter API. The returned catalog is streamed straight to the UI for selection.

## Practical Integration Examples

Below are Rust snippets that mirror the patterns found in the Meetily source.

### Calling the Generic Generator

```rust
use reqwest::Client;
use tauri::async_runtime::spawn;
use crate::summary::llm_client::{generate_summary, LLMProvider};
use tokio_util::sync::CancellationToken;

#[tauri::command]
async fn summarize_meeting(
    provider: String,
    model: String,
    system_prompt: String,
    user_prompt: String,
) -> Result<String, String> {
    let client = Client::new();
    let provider = LLMProvider::from_str(&provider)?;
    let cancel_token = CancellationToken::new();

    generate_summary(
        &client,
        &provider,
        &model,
        "",                  // API key (empty for local Ollama)
        &system_prompt,
        &user_prompt,
        None,                // optional Ollama endpoint
        None,                // optional custom OpenAI endpoint
        None, None, None,    // optional token/temperature/top_p
        None,                // app_data_dir
        Some(&cancel_token),
    )
    .await
}

```

### Ensuring an Ollama Model Exists

```rust
use crate::ollama::ollama::{get_ollama_models, pull_ollama_model};
use tauri::AppHandle;

pub async fn ensure_ollama_model(
    app: &AppHandle,
    model_name: &str,
) -> Result<(), String> {
    let models = get_ollama_models(None).await?;
    if !models.iter().any(|m| m.name == model_name) {
        pull_ollama_model(app.clone(), model_name.to_string(), None).await?;
    }
    Ok(())
}

```

### Listing Groq Models in a Tauri Command

```rust
use crate::groq::groq::get_groq_models;

#[tauri::command]
async fn list_groq_models(
    api_key: Option<String>,
) -> Result<Vec<crate::groq::groq::GroqModel>, String> {
    get_groq_models(api_key).await
}

```

## Summary

- The Meetily **summary generation engine** integrates with LLM providers through a single `generate_summary` dispatch function in [`frontend/src-tauri/src/summary/llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/frontend/src-tauri/src/summary/llm_client.rs).
- **OpenAI-compatible** back ends—Ollama, Groq, OpenRouter, and CustomOpenAI—share the `ChatRequest` struct, while **Claude** uses the dedicated `ClaudeRequest` format.
- Authentication headers and endpoint URLs are provider-specific: Bearer tokens for most, `x-api-key` for Claude, and none for local Ollama.
- Every request supports **cancellation** via `tokio::select!` and a `CancellationToken`.
- The **BuiltInAI** variant bypasses HTTP and talks to a local Llama sidecar via [`summary/summary_engine/client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/summary/summary_engine/client.rs).
- Helper modules in [`ollama/ollama.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/ollama/ollama.rs), [`groq/groq.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/groq/groq.rs), and [`openrouter/openrouter.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/openrouter/openrouter.rs) handle model discovery and caching.

## Frequently Asked Questions

### How does Meetily switch between OpenAI-compatible and Claude request formats?

The `LLMProvider` enum parsed in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs) determines which struct the engine serializes. If the variant is `Claude`, the engine builds a `ClaudeRequest` with a top-level `system` field and `max_tokens`. For every other variant, it builds the generic `ChatRequest` used by OpenAI-compatible endpoints.

### Can I use a self-hosted OpenAI-compatible server instead of Groq or OpenRouter?

Yes. The `CustomOpenAI` variant accepts a user-supplied endpoint URL and optional API key. The engine sends the standard `ChatRequest` to that custom base URL, making it compatible with self-hosted inference servers such as vLLM or LocalAI.

### What happens if the user cancels a summary request mid-flight?

The `generate_summary` function accepts an optional `CancellationToken`. The request future races against the token inside a `tokio::select!` block in [`llm_client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/llm_client.rs). If the token triggers, the future aborts and the function returns early, preventing unnecessary LLM charges and freeing the HTTP client.

### Does Meetily require an internet connection for all summarization?

No. The `BuiltInAI` provider communicates with a local sidecar process defined in [`summary/summary_engine/client.rs`](https://github.com/Zackriya-Solutions/meetily/blob/main/summary/summary_engine/client.rs), running a Llama-style model entirely offline. Ollama can also operate offline once the model weights are pulled to the local machine via `pull_ollama_model`.