How Provider-Specific API Fetching Functions Handle Rate Limit Errors and Pagination in free-llm-api-resources

The provider-specific API fetching functions in src/pull_available_models.py catch HTTP exceptions—including 429 rate limit errors—and return empty collections to prevent script crashes, while implementing explicit pagination only for the GitHub Marketplace fetcher via a page-based loop with polite delays.

The cheahjs/free-llm-api-resources repository automates the aggregation of free LLM model availability from multiple providers. Understanding how these provider-specific API fetching functions handle rate limit errors and pagination reveals the robust error-handling patterns and architectural decisions that keep the data pipeline resilient against service disruptions.

Common Error Handling Patterns Across Providers

All provider-specific functions in src/pull_available_models.py follow a consistent defensive programming strategy to handle network instability and rate limiting.

HTTP Exception Catching and Empty Result Fallbacks

Each function wraps its HTTP requests in a try/except block targeting requests.exceptions.RequestException. After executing a request, the code immediately calls r.raise_for_status() to trigger exceptions on 4xx and 5xx responses, including 429 "Too Many Requests" errors. When caught, the exception is logged and the function returns either an empty list [] or empty dictionary {}, ensuring the main aggregation loop continues uninterrupted even if individual providers fail.

try:
    r = requests.get(url, headers=headers)
    r.raise_for_status()
except requests.exceptions.RequestException as e:
    logger.error(f"Error fetching provider models: {e}")
    return []

Provider-Specific Rate Limit Handling Strategies

While error handling follows a common template, each provider implements distinct rate limit awareness mechanisms.

Groq – Dynamic Rate Limit Extraction from Response Headers

The fetch_groq_models and get_groq_limits_for_model functions extract dynamic rate limit metadata directly from HTTP response headers. Upon successful requests, the code parses x-ratelimit-limit-requests and x-ratelimit-limit-tokens to populate the limits field with the actual quotas enforced by the Groq API.

rpd = int(r.headers["x-ratelimit-limit-requests"])
tpm = int(r.headers["x-ratelimit-limit-tokens"])

This approach allows the script to report real-time quota availability rather than relying on hardcoded values.

OpenRouter, Hyperbolic, and OVH – Static and Validated Limits

Several providers enforce static rate limits that the code respects through configuration rather than dynamic header inspection:

• OpenRouter: The fetch_openrouter_models function filters for :free tier models and applies static limits of 20 requests per minute and 50 requests per day before returning the dataset.
• Hyperbolic: The fetch_hyperbolic_models function enforces a static limit of 60 requests per minute in its return payload.
• OVH: The fetch_ovh_models function mitigates pagination needs by requesting a fixed limit=100 with id.desc sorting, handling the entire dataset in a single validated request.

Kluster, Cloudflare, and GitHub – Exception-Based Recovery

The fetch_kluster_models, fetch_cloudflare_models, and fetch_github_models functions rely solely on exception handling for rate limit management. Rather than parsing headers, they depend on r.raise_for_status() to detect throttling responses. For GitHub specifically, the fetcher also treats a 404 response as a termination signal during pagination, gracefully handling cases where the expected page count exceeds actual available data.

Pagination Implementation in API Fetching

The repository employs two distinct strategies for handling large datasets: single-page ingestion and explicit pagination.

Single-Page Response Pattern

Six of the seven providers—Groq, Kluster, OpenRouter, Cloudflare, OVH, and Hyperbolic—return their complete model catalogs in a single API response. The corresponding fetch functions contain no pagination logic, making a single HTTP request per provider and processing the entire result set immediately. The OVH implementation explicitly sets limit=100 to ensure the full dataset fits within this single-request pattern.

GitHub Marketplace – Explicit Pagination with Rate Limiting

Only the fetch_github_models function implements explicit pagination logic to handle the GitHub Marketplace API's paginated responses. The implementation follows a controlled iteration pattern:

1. Initializes page at 1 and extracts total_pages from the initial JSON response
2. Iterates while page <= total_pages
3. Appends a 0.5-second delay (time.sleep(0.5)) between requests to avoid triggering rate limits
4. Terminates early if a 404 status is encountered, treating it as an end-of-data signal

page = 1
total_pages = 1

while page <= total_pages:
    url = f"https://github.com/marketplace?type=models&page={page}"
    r = requests.get(url, headers=headers)
    r.raise_for_status()
    data = r.json()
    total_pages = data.get("totalPages", 0)
    
    # Process models...

    
    page += 1
    time.sleep(0.5)

This defensive approach prevents aggressive polling that could result in IP throttling or blacklisting.

Practical Implementation Examples

Example 1: Extracting Groq Rate Limit Metadata

import logging
from src.pull_available_models import fetch_groq_models

logger = logging.getLogger(__name__)
groq_models = fetch_groq_models(logger)

for model in groq_models:
    if 'limits' in model:
        print(f"{model['id']}: {model['limits']}")

Example 2: Iterating GitHub Marketplace Pages Safely

import logging
from src.pull_available_models import fetch_github_models

logger = logging.getLogger(__name__)
github_models = fetch_github_models(logger)

print(f"Fetched {len(github_models)} models from GitHub Marketplace")

Example 3: Aggregating All Providers with Unified Error Handling

import logging
from src.pull_available_models import (
    fetch_groq_models,
    fetch_kluster_models,
    fetch_openrouter_models,
    fetch_cloudflare_models,
    fetch_ovh_models,
    fetch_hyperbolic_models,
    fetch_github_models
)

logger = logging.getLogger(__name__)
all_models = []

providers = [
    fetch_groq_models,
    fetch_kluster_models,
    fetch_openrouter_models,
    fetch_cloudflare_models,
    fetch_ovh_models,
    fetch_hyperbolic_models,
    fetch_github_models,
]

for fetch_fn in providers:
    models = fetch_fn(logger)
    all_models.extend(models)
    print(f"{fetch_fn.__name__}: {len(models)} models fetched")

Summary

• Exception-based resilience: All provider functions in src/pull_available_models.py use requests.exceptions.RequestException handling to catch 429 errors and return empty collections, preventing pipeline failures.
• Dynamic vs. static limits: Groq extracts real-time quotas from response headers (x-ratelimit-limit-requests), while OpenRouter, Hyperbolic, and OVH rely on documented static limits.
• Single-request architecture: Six providers return complete datasets without pagination, simplifying the aggregation logic.
• Explicit GitHub pagination: Only fetch_github_models implements page-based iteration with time.sleep(0.5) delays and 404 termination handling to manage the GitHub Marketplace API's paginated structure.

Frequently Asked Questions

How do the provider-specific functions handle HTTP 429 rate limit errors?

Each function wraps its HTTP request in a try-except block that catches requests.exceptions.RequestException. When r.raise_for_status() encounters a 429 response, the exception propagates to the except block, where it is logged and the function returns an empty list or dictionary. This prevents the error from crashing the main aggregation loop while recording the failure for debugging.

Which providers require pagination handling?

Only the GitHub Marketplace fetcher (fetch_github_models) implements explicit pagination logic. The remaining six providers—Groq, Kluster, OpenRouter, Cloudflare, OVH, and Hyperbolic—return their complete model catalogs in single API responses, eliminating the need for page-based iteration.

How does the GitHub fetcher prevent rate limiting during pagination?

The fetch_github_models function inserts a 0.5-second delay between page requests using time.sleep(0.5). This polite polling interval reduces the risk of triggering GitHub's rate limits during the sequential fetching of multiple pages. Additionally, the function treats 404 responses as a termination signal, preventing infinite loops if the API reports fewer pages than initially indicated.

Can I adjust the rate limit values for OpenRouter or Hyperbolic in the code?

The rate limits for OpenRouter (20 requests/minute, 50 requests/day) and Hyperbolic (60 requests/minute) are hardcoded constants within their respective fetch functions. To adjust these values, you must modify the static dictionary assignments in src/pull_available_models.py within fetch_openrouter_models or fetch_hyperbolic_models, as these providers do not expose dynamic limit headers like Groq does.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}