How the Meetily Summary Generation Engine Integrates with Ollama, Claude, Groq, and OpenRouter

The Meetily summary generation engine integrates with LLM providers through a single generate_summary function in frontend/src-tauri/src/summary/llm_client.rs, dynamically switching request formats, authentication headers, and endpoints to support Ollama, Claude, Groq, OpenRouter, and a built-in local AI sidecar.

The Zackriya-Solutions/meetily repository implements this Rust-based dispatch layer inside the frontend/src-tauri/src/summary subsystem. Whether the front end selects a local Ollama instance or a remote Claude API key, the engine manages serialization, HTTP dispatch, and response parsing without UI changes. This article walks through the exact integration mechanisms—from enum dispatch to sidecar fallback—based on the current source tree.

Provider Selection and Runtime Dispatch

At runtime, the front end passes a provider string such as "ollama" or "claude". The engine converts this string into a typed variant via LLMProvider::from_str defined in frontend/src-tauri/src/summary/llm_client.rs.

This enum drives every downstream decision, including which request struct to build, which headers to attach, and which URL to target. Because the provider is resolved before any network call, adding a new backend requires changes only inside the summary subsystem.

Request Builder and Provider Types

The engine maintains two request families: one for OpenAI-compatible providers and one for Anthropic Claude.

OpenAI-Compatible Providers

Ollama, Groq, OpenRouter, and CustomOpenAI all use the ChatRequest struct. This JSON payload carries model, messages, and optional generation parameters. The definition lives in llm_client.rs around lines 18-28 and is reused across all four backends by serializing the same struct to different endpoints.

Claude

Claude deviates from the OpenAI shape. The engine constructs a ClaudeRequest struct that includes a top-level system field and a fixed max_tokens value. This format aligns with Anthropic’s /v1/messages endpoint rather than the standard chat-completions path.

Authentication and Endpoint Routing

Each provider carries its own URL and header scheme, hard-coded or configured in llm_client.rs.

• OpenAI targets https://api.openai.com/v1/chat/completions
• Groq targets https://api.groq.com/openai/v1/chat/completions
• OpenRouter targets https://openrouter.ai/api/v1/chat/completions
• Ollama defaults to http://localhost:11434/v1/chat/completions but accepts a custom override
• CustomOpenAI uses a user-supplied base URL for self-hosted OpenAI-compatible servers

For authentication, the engine injects Authorization: Bearer <api_key> for every remote provider except Claude. Claude instead requires x-api-key and anthropic-version headers. Ollama typically omits the header entirely when running locally without API key validation.

HTTP Transport with Cancellation

Every outbound request is wrapped in an async tokio::select! block. The engine accepts a CancellationToken from the caller, allowing the summary generation to abort immediately if the user dismisses the dialog or closes the window.

This pattern appears in llm_client.rs between lines 66-82. The timeout and cancellation logic is provider-agnostic, meaning Groq and OpenRouter requests receive the same lifecycle management as a local Ollama call.

Response Parsing

After the HTTP client returns bytes, the engine deserializes the payload based on the active provider.

For Claude, the JSON maps to ClaudeChatResponse, and the extractor returns the first text block in the content array. For all other providers, the engine uses the generic ChatResponse shape and extracts choices[0].message.content. These two parsers live in llm_client.rs around lines 100-124.

Built-In AI Sidecar

When the provider variant is BuiltInAI, the engine bypasses HTTP entirely. It communicates with a local sidecar process running a Llama-style model.

This path is orchestrated by frontend/src-tauri/src/summary/summary_engine/client.rs. The sidecar client manages process lifecycle and exchanges messages over local IPC, offering fully offline summarization without an external API key.

Supporting Modules for Model Discovery

Before a summary request fires, the UI often queries available models. Meetily maintains provider-specific helpers for this.

Ollama Model Management

The file frontend/src-tauri/src/ollama/ollama.rs exposes get_ollama_models, pull_ollama_model, delete_ollama_model, and get_ollama_model_context. These commands populate the dropdown and ensure the correct model weights are present locally before the engine attempts inference.

Groq Model Caching

frontend/src-tauri/src/groq/groq.rs provides get_groq_models, which lists chat-capable Groq models and caches the result for five minutes. The Tauri command layer re-exports this to the front end.

OpenRouter Catalog

frontend/src-tauri/src/openrouter/openrouter.rs implements get_openrouter_models via a blocking HTTP call to the OpenRouter API. The returned catalog is streamed straight to the UI for selection.

Practical Integration Examples

Below are Rust snippets that mirror the patterns found in the Meetily source.

Calling the Generic Generator

use reqwest::Client;
use tauri::async_runtime::spawn;
use crate::summary::llm_client::{generate_summary, LLMProvider};
use tokio_util::sync::CancellationToken;

#[tauri::command]
async fn summarize_meeting(
    provider: String,
    model: String,
    system_prompt: String,
    user_prompt: String,
) -> Result<String, String> {
    let client = Client::new();
    let provider = LLMProvider::from_str(&provider)?;
    let cancel_token = CancellationToken::new();

    generate_summary(
        &client,
        &provider,
        &model,
        "",                  // API key (empty for local Ollama)
        &system_prompt,
        &user_prompt,
        None,                // optional Ollama endpoint
        None,                // optional custom OpenAI endpoint
        None, None, None,    // optional token/temperature/top_p
        None,                // app_data_dir
        Some(&cancel_token),
    )
    .await
}

Ensuring an Ollama Model Exists

use crate::ollama::ollama::{get_ollama_models, pull_ollama_model};
use tauri::AppHandle;

pub async fn ensure_ollama_model(
    app: &AppHandle,
    model_name: &str,
) -> Result<(), String> {
    let models = get_ollama_models(None).await?;
    if !models.iter().any(|m| m.name == model_name) {
        pull_ollama_model(app.clone(), model_name.to_string(), None).await?;
    }
    Ok(())
}

Listing Groq Models in a Tauri Command

use crate::groq::groq::get_groq_models;

#[tauri::command]
async fn list_groq_models(
    api_key: Option<String>,
) -> Result<Vec<crate::groq::groq::GroqModel>, String> {
    get_groq_models(api_key).await
}

Summary

• The Meetily summary generation engine integrates with LLM providers through a single generate_summary dispatch function in frontend/src-tauri/src/summary/llm_client.rs.
• OpenAI-compatible back ends—Ollama, Groq, OpenRouter, and CustomOpenAI—share the ChatRequest struct, while Claude uses the dedicated ClaudeRequest format.
• Authentication headers and endpoint URLs are provider-specific: Bearer tokens for most, x-api-key for Claude, and none for local Ollama.
• Every request supports cancellation via tokio::select! and a CancellationToken.
• The BuiltInAI variant bypasses HTTP and talks to a local Llama sidecar via summary/summary_engine/client.rs.
• Helper modules in ollama/ollama.rs, groq/groq.rs, and openrouter/openrouter.rs handle model discovery and caching.

Frequently Asked Questions

How does Meetily switch between OpenAI-compatible and Claude request formats?

The LLMProvider enum parsed in llm_client.rs determines which struct the engine serializes. If the variant is Claude, the engine builds a ClaudeRequest with a top-level system field and max_tokens. For every other variant, it builds the generic ChatRequest used by OpenAI-compatible endpoints.

Can I use a self-hosted OpenAI-compatible server instead of Groq or OpenRouter?

Yes. The CustomOpenAI variant accepts a user-supplied endpoint URL and optional API key. The engine sends the standard ChatRequest to that custom base URL, making it compatible with self-hosted inference servers such as vLLM or LocalAI.

What happens if the user cancels a summary request mid-flight?

The generate_summary function accepts an optional CancellationToken. The request future races against the token inside a tokio::select! block in llm_client.rs. If the token triggers, the future aborts and the function returns early, preventing unnecessary LLM charges and freeing the HTTP client.

Does Meetily require an internet connection for all summarization?

No. The BuiltInAI provider communicates with a local sidecar process defined in summary/summary_engine/client.rs, running a Llama-style model entirely offline. Ollama can also operate offline once the model weights are pulled to the local machine via pull_ollama_model.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}