Why Groq Uses Different Rate Limit Headers for Whisper vs Chat Completion Models

Groq exposes specialized rate limit headers because audio transcription consumes processing time (measured in seconds) while chat completion consumes token generation capacity, requiring distinct resource metering for each endpoint.

The cheahjs/free-llm-api-resources repository reveals that Groq's API returns endpoint-specific rate limit headers that reflect the underlying computational resources each service consumes. Understanding these differences is crucial for building robust integrations that respect Groq's throttling mechanisms for speech-to-text versus text-generation workloads.

Why Audio Transcription Uses Audio-Seconds Headers

When you call Groq's Whisper endpoint for speech-to-text conversion, the API returns x-ratelimit-limit-audio-seconds instead of token-based headers. This reflects that Whisper models process audio duration rather than generating text tokens.

The x-ratelimit-limit-audio-seconds Header

According to the source code in src/pull_available_models.py, after a successful transcription request to POST https://api.groq.com/openai/v1/audio/transcriptions, the script extracts the audio processing quota from the response headers:

# Lines 71-75 in src/pull_available_models.py

audio_seconds_per_minute = int(r.headers["x-ratelimit-limit-audio-seconds"])
rpd = int(r.headers["x-ratelimit-limit-requests"])

The x-ratelimit-limit-audio-seconds value represents the maximum number of audio seconds you may submit per minute, while x-ratelimit-limit-requests indicates your daily transcription request quota.

Why Chat Completion Uses Token-Based Headers

For LLM inference endpoints, Groq measures consumption in tokens rather than time because text generation scales with output length and model complexity, not input duration.

The x-ratelimit-limit-tokens Header

As implemented in the get_groq_limits_for_model function (lines 86-108 of src/pull_available_models.py), chat completion responses from POST https://api.groq.com/openai/v1/chat/completions return different metering headers:

# Lines 86-108 in src/pull_available_models.py

rpd = int(r.headers["x-ratelimit-limit-requests"])
tpm = int(r.headers["x-ratelimit-limit-tokens"])

Here, x-ratelimit-limit-tokens specifies the maximum number of tokens you can generate per minute, while x-ratelimit-limit-requests tracks your daily request allowance.

Shared vs. Distinct Rate Limit Headers

Both endpoints share the daily request quota header but diverge on per-minute throttling metrics:

• Both endpoints return: x-ratelimit-limit-requests (daily request quota)
• Whisper (STT) adds: x-ratelimit-limit-audio-seconds (audio processing per minute)
• Chat completion adds: x-ratelimit-limit-tokens (token generation per minute)

This architectural distinction ensures that rate limiting aligns with the actual computational resource being consumed—audio processing time for transcription versus token throughput for generation.

Accessing These Headers in Practice

To inspect your current limits programmatically, you must make an actual request to each endpoint, as Groq only returns these headers on successful API calls. The free-llm-api-resources repository demonstrates this pattern for both service types.

Checking Whisper Limits

import os, requests

def get_whisper_limits(model_id: str) -> dict:
    r = requests.post(
        "https://api.groq.com/openai/v1/audio/transcriptions",
        headers={"Authorization": f'Bearer {os.getenv("GROQ_API_KEY")}'},
        data={"model": model_id},
        files={"file": open("1-second-of-silence.mp3", "rb")},
    )
    r.raise_for_status()
    return {
        "audio_seconds_per_minute": int(r.headers["x-ratelimit-limit-audio-seconds"]),
        "requests_per_day": int(r.headers["x-ratelimit-limit-requests"]),
    }

print(get_whisper_limits("whisper-large-v3"))

# → {'audio_seconds_per_minute': 7200, 'requests_per_day': 2000}

Checking Chat Completion Limits

def get_chat_limits(model_id: str) -> dict:
    r = requests.post(
        "https://api.groq.com/openai/v1/chat/completions",
        headers={
            "Authorization": f'Bearer {os.getenv("GROQ_API_KEY")}',
            "Content-Type": "application/json",
        },
        json={
            "model": model_id,
            "messages": [{"role": "user", "content": "Hello"}],
            "max_tokens": 1,
            "stream": True,
        },
        stream=True,
    )
    r.raise_for_status()
    return {
        "requests_per_day": int(r.headers["x-ratelimit-limit-requests"]),
        "tokens_per_minute": int(r.headers["x-ratelimit-limit-tokens"]),
    }

print(get_chat_limits("llama3-groq-8b-8192-tool-use-preview"))

# → {'requests_per_day': 250, 'tokens_per_minute': 70000}

Summary

• Resource-specific metering: Groq tailors rate limit headers to match the computational resource each endpoint consumes—audio seconds for Whisper and tokens for chat models.
• Daily quotas are shared: Both endpoints return x-ratelimit-limit-requests for daily limits, but per-minute throttling uses endpoint-specific headers.
• Implementation details: The src/pull_available_models.py file in cheahjs/free-llm-api-resources handles these differences through separate functions (get_groq_limits_for_stt_model and get_groq_limits_for_model) that parse the distinct header sets.
• Header extraction requires live requests: You must make actual API calls to receive rate limit headers; they are not available through a dedicated metadata endpoint.

Frequently Asked Questions

What is the difference between Groq's audio-seconds and tokens rate limits?

Audio-seconds rate limits cap the total duration of audio you can submit to Whisper models per minute (typically 7,200 seconds/minute), while tokens rate limits restrict the number of tokens your chat completion requests can generate per minute (often 70,000 tokens/minute for high-tier models). These metrics reflect fundamentally different computational bottlenecks—audio processing time versus text generation throughput.

Do all Groq models share the same rate limit headers?

No, the specific rate limit headers returned depend on the endpoint type, not individual model variants. All Whisper models accessed through the /audio/transcriptions endpoint return x-ratelimit-limit-audio-seconds, while all chat models accessed through /chat/completions return x-ratelimit-limit-tokens. However, the numerical values (the actual limits) vary by model tier and your account status.

How can I check my current Groq rate limit usage programmatically?

You must make a successful API request and inspect the response headers. The free-llm-api-resources repository demonstrates this approach in src/pull_available_models.py, where functions make minimal requests (1-second audio for Whisper, 1-token chat completions) specifically to harvest the x-ratelimit-limit-* headers without consuming significant quota.

Why does Groq limit audio transcription by seconds rather than tokens?

Groq limits transcription by audio seconds because Whisper models process audio streams sequentially, consuming GPU time proportional to input duration rather than output token count. Unlike text generation where computational cost scales with token production, speech-to-text processing is bounded by the length of the audio file being transcribed, making seconds the appropriate metering unit for throttling.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}