How to Set Up Podcast Transcription with Groq or OpenAI API in Agent-Reach

Agent-Reach provides a self-contained transcription pipeline that automatically routes audio to Groq's Whisper-large-v3 or OpenAI's Whisper-1 models, requiring only API key configuration and the installation of yt-dlp and ffmpeg to convert podcast URLs into plain text.

Agent-Reach is an open-source Python framework that ships with a built-in transcription pipeline for converting podcast episodes into searchable text. The system supports both Groq and OpenAI Whisper APIs through a unified interface that handles audio downloading, compression, chunking, and intelligent provider fallback automatically. All transcription logic resides in agent_reach/transcribe.py, which orchestrates the entire workflow from URL to plaintext.

Architecture Overview

The transcription system is built around a provider-agnostic pipeline that abstracts away the differences between Groq and OpenAI implementations while enforcing consistent audio preprocessing standards.

Provider Configuration and Routing

At the core of the routing logic is the PROVIDERS dictionary defined in agent_reach/transcribe.py (lines 30-40). This mapping associates provider names with their respective endpoints, model identifiers, and configuration keys. When you call the transcription functions, the system uses this dictionary to determine which API endpoint to hit and which API key to retrieve from your configuration.

The Config object (utilized via _provider_key at lines 57-61) reads keys from ~/.agent-reach/config.yaml or environment variables using the field names defined in PROVIDERS—specifically groq_api_key and openai_api_key.

Audio Processing Pipeline

Before sending data to any API, the system normalizes audio through three distinct phases:

1. Acquisition: The download_audio function (lines 77-95) uses yt-dlp to fetch audio from URLs into temporary files.
2. Compression: The compress_audio function (lines 101-123) normalizes tracks to mono, 16 kHz, 32 kbps m4a format, ensuring files remain under the 25 MiB Whisper API limit.
3. Chunking: If compression still yields files exceeding size limits, chunk_audio (lines 126-154) splits audio into ≤10-minute segments that fit within single API requests.

Fallback Orchestration

The _transcribe_with_fallback method (lines 49-61) implements resilient error handling by iterating over an ordered list of providers. When provider="auto" is specified, the system attempts Groq first, then falls back to OpenAI if the primary request fails, aggregating errors only if all providers exhaust their retry logic.

Configuration Steps

Obtain API Credentials

You need at least one valid API key to use the transcription feature:

• Groq: Create a free account at groq.com and generate a groq_api_key.
• OpenAI: Generate an openai_api_key from platform.openai.com/account/api-keys.

Store Keys via CLI

Agent-Reach includes a configuration helper that writes keys to the correct location in YAML format. Run the following commands to store your credentials:

agent-reach configure groq-key <your-groq-key>
agent-reach configure openai-key <your-openai-key>

This populates ~/.agent-reach/config.yaml with the field names expected by the PROVIDERS dictionary, eliminating manual YAML editing.

Install External Dependencies

The pipeline requires two external binaries that must be available in your system PATH:

• yt-dlp: Install via pip install yt-dlp or your system package manager.
• ffmpeg: Install via apt, brew, or download from ffmpeg.org.

Verify Installation

Validate your setup by running the built-in diagnostic command:

python -m agent_reach.cli doctor

This checks for the presence of yt-dlp, ffmpeg, and confirms that at least one provider key is configured correctly.

Transcribing Podcasts

The public entry point transcribe() (lines 68-88 in agent_reach/transcribe.py) orchestrates the complete workflow: validation → download/compression → chunking → provider fallback → concatenation of results.

Automatic Provider Selection

To transcribe a podcast with automatic provider fallback, use the default "auto" setting:

from agent_reach.transcribe import transcribe

text = transcribe(
    "https://www.youtube.com/watch?v=example_podcast_episode",
    provider="auto",  # Tries Groq first, falls back to OpenAI

)

print(text[:500])  # Preview first 500 characters

Forcing a Specific Provider

To bypass the fallback logic and force a specific provider:

from agent_reach.transcribe import transcribe

transcript = transcribe(
    "https://example.com/podcast.mp3",
    provider="groq",  # Explicitly use Groq; errors will not fall back to OpenAI

)

Preserving Intermediate Files

For debugging purposes, you can retain the downloaded source, compressed audio, and chunk files by specifying an output directory:

from pathlib import Path
from agent_reach.transcribe import transcribe

out = Path("/tmp/podcast_debug")
out.mkdir(parents=True, exist_ok=True)

transcript = transcribe(
    "https://example.com/long-episode.m4a",
    provider="openai",
    out_dir=out,
)

# Directory now contains: source.<ext>, compressed.m4a, chunk_001.m4a, etc.

Low-Level API Access

If you need to transcribe pre-processed audio chunks directly without the full pipeline, use the transcribe_chunk function:

from pathlib import Path
from agent_reach.transcribe import transcribe_chunk, Config

cfg = Config()  # Loads keys from ~/.agent-reach/config.yaml

chunk_path = Path("/tmp/chunk_001.m4a")
text = transcribe_chunk(chunk_path, "groq", config=cfg)
print(text)

This function sends a single audio chunk to the specified provider's /audio/transcriptions endpoint and returns the plaintext response.

Summary

• Agent-Reach provides a complete podcast transcription pipeline in agent_reach/transcribe.py that supports both Groq and OpenAI Whisper models.
• Configuration requires storing groq_api_key or openai_api_key via the CLI helper, plus installing yt-dlp and ffmpeg.
• The transcribe() function handles the entire workflow: downloading via yt-dlp, compressing to 32 kbps m4a, chunking if necessary, and routing through the PROVIDERS fallback logic.
• Use provider="auto" for resilient transcription that tries Groq first, then OpenAI, or specify a single provider to force specific API usage.
• Intermediate files can be preserved by passing an out_dir parameter for debugging long episodes.

Frequently Asked Questions

What audio sources are supported by the transcription feature?

The download_audio function utilizes yt-dlp, which supports virtually any URL that yt-dlp can process—including YouTube videos, direct MP3/M4A links, and most podcast hosting platforms. For local files, you can pass a file path directly to the transcribe() function, bypassing the download step while still processing through compression and chunking.

Why does my transcription fail with a file size error?

The Whisper API enforces a 25 MiB limit per request. The compress_audio function automatically normalizes audio to mono, 16 kHz, 32 kbps m4a to stay under this limit. If the compressed file still exceeds 25 MiB, the chunk_audio function splits it into ≤10-minute segments. Ensure you have ffmpeg installed and accessible in your PATH, as missing binaries will cause compression failures that lead to size errors.

How does the fallback mechanism work when using provider="auto"?

The _transcribe_with_fallback method iterates over an ordered list starting with Groq, then OpenAI, attempting each provider in sequence until one returns a successful transcript. If Groq's API returns an error or times out, the system automatically retries with OpenAI using the same audio chunks. Only if all configured providers fail will the function raise an aggregated error containing all failure messages.

Can I use both Groq and OpenAI keys simultaneously?

Yes. You can configure both providers using the CLI commands agent-reach configure groq-key and agent-reach configure openai-key. When both keys are present, setting provider="auto" enables the fallback logic to use Groq as the primary and OpenAI as the backup. You can also explicitly choose either provider on a per-call basis by passing provider="groq" or provider="openai" to the transcribe() function.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}