Price API Rate Limiting and Caching Strategies in AI-Trader: A Complete Guide

AI-Trader implements a layered defense system in service/server/price_fetcher.py that combines provider-wide cooldowns, automatic rate-limit detection, exponential back-off, and in-memory caching to prevent external API saturation while minimizing redundant network traffic.

The HKUDS/AI-Trader repository handles high-frequency price fetching from multiple market data providers. To maintain reliable access without triggering provider bans or wasting bandwidth on duplicate lookups, the codebase employs sophisticated price API rate limiting and caching techniques that are both self-healing and configurable at runtime.

Provider-Wide Rate Limiting

The foundation of AI-Trader's rate limiting is a global cooldown mechanism that tracks provider health across all requests. This prevents the system from hammering APIs that have already signaled distress.

Cooldown Map Implementation

At the module level, _provider_cooldowns (lines 41-44) stores the earliest timestamp when each provider may be called again. Before every network request, _provider_cooldown_remaining() (lines 59-60) checks this map. If the current time is earlier than the stored timestamp, the request aborts immediately with an error, ensuring zero additional load on struggling endpoints.

# Internal logic flow (simplified)

if _provider_cooldown_remaining("alphavantage") > 0:
    raise CooldownError(f"Provider cooling down for {remaining}s")

Automatic HTTP 429 Handling

When a provider returns HTTP 429 (Too Many Requests), the fetcher automatically invokes _activate_provider_cooldown() (lines 122-128). This sets a mandatory pause using the PRICE_FETCH_RATE_LIMIT_COOLDOWN_SECONDS environment variable (default 60 seconds), preventing further calls until the cooldown expires.

Server Error Protection (5xx)

Transient server errors trigger a shorter safety pause. Any status code ≥ 500 activates a cooldown defined by PRICE_FETCH_ERROR_COOLDOWN_SECONDS (default 20 seconds) via lines 128-134. This protects against cascading failures while allowing quicker recovery than hard rate limits.

Intelligent Retry Logic

Exponential Back-off with Jitter

For recoverable errors including 429s, 5xxs, timeouts, and connection failures, AI-Trader implements _retry_delay() (lines 73-78). The delay calculates as:

delay = PRICE_FETCH_BACKOFF_BASE_SECONDS × (2 ^ attempt_number) + random_jitter

The _request_json_with_retry() function (lines 112-145) orchestrates this logic, sleeping between attempts according to the computed delay and respecting the PRICE_FETCH_MAX_RETRIES limit.

Caching Mechanisms

Polymarket Token Resolution Cache

To eliminate redundant blockchain lookups, the system maintains _polymarket_token_cache (lines 54-57), an in-memory dictionary storing resolved token IDs for (reference, token_id, outcome) tuples. Entries expire after _POLYMARKET_TOKEN_CACHE_TTL_S (300 seconds/5 minutes), after which the next call refreshes the data.

The cache check occurs in _polymarket_resolve_reference() (lines 26-33), short-circuiting network requests when valid cached data exists.

Environment-Based Configuration

All behavioral parameters are externally configurable without code changes:

• PRICE_FETCH_RATE_LIMIT_COOLDOWN_SECONDS: Duration to pause after HTTP 429
• PRICE_FETCH_ERROR_COOLDOWN_SECONDS: Duration to pause after 5xx errors
• PRICE_FETCH_MAX_RETRIES: Maximum retry attempts per request
• PRICE_FETCH_BACKOFF_BASE_SECONDS: Base delay for exponential back-off
• PRICE_FETCH_TIMEOUT_SECONDS: Request timeout threshold

Implementation Examples

Configure the rate limiting behavior via environment variables before initializing the fetcher:

import os

os.environ["PRICE_FETCH_RATE_LIMIT_COOLDOWN_SECONDS"] = "45"   # 45s pause after 429

os.environ["PRICE_FETCH_ERROR_COOLDOWN_SECONDS"] = "15"      # 15s pause after 5xx

os.environ["PRICE_FETCH_MAX_RETRIES"] = "3"                 # maximum 3 retries

os.environ["PRICE_FETCH_BACKOFF_BASE_SECONDS"] = "0.5"      # 0.5s base back-off

os.environ["PRICE_FETCH_TIMEOUT_SECONDS"] = "8"            # 8s request timeout

Fetch prices with automatic rate-limit handling:

from service.server.price_fetcher import get_price_from_market

price = get_price_from_market(
    symbol="AAPL",
    executed_at="2024-11-01T14:30:00Z",
    market="us-stock"
)

if price is not None:
    print(f"Fetched price: ${price:.2f}")
else:
    print("Price unavailable (provider in cooldown)")

Simulate and check cooldown status programmatically:

from service.server.price_fetcher import _activate_provider_cooldown, _provider_cooldown_remaining

# Simulate hitting a rate limit

_activate_provider_cooldown("alphavantage", 60, "HTTP 429")
remaining = _provider_cooldown_remaining("alphavantage")
print(f"Cooldown remaining: {remaining}s")  # Value close to 60

Leverage the Polymarket cache for repeated lookups:

from service.server.price_fetcher import _polymarket_resolve_reference

# First call performs network request

contract = _polymarket_resolve_reference("btc-usd")

# Second call within 5 minutes returns cached data instantly

contract_cached = _polymarket_resolve_reference("btc-usd")

Summary

AI-Trader's approach to price API rate limiting and caching delivers a self-throttling client through five key mechanisms:

• Pre-flight cooldown checks guarantee no requests hit providers already flagged as rate-limited
• Automatic 429 detection triggers configurable cooling periods to respect provider limits
• 5xx error handling applies shorter pauses to handle transient server issues gracefully
• Jittered exponential back-off reduces burst traffic while maximizing eventual success rates
• TTL-based in-memory caching for Polymarket token resolution eliminates redundant HTTP round-trips for up to five minutes

Frequently Asked Questions

How does AI-Trader prevent hitting rate limits repeatedly?

The system maintains a global _provider_cooldowns dictionary in service/server/price_fetcher.py that tracks when each provider was last rate-limited. Before every request, _provider_cooldown_remaining() verifies that enough time has passed since the last 429 or 5xx error. If the cooldown is still active, the request fails fast locally without touching the external API.

What happens when a price API returns HTTP 429?

When the fetcher receives an HTTP 429 status, _activate_provider_cooldown() immediately registers a cooldown period (default 60 seconds) for that specific provider. Subsequent calls to get_price_from_market() for that provider will return errors until the cooldown expires, preventing further rate limit violations.

Can I adjust the retry behavior without modifying the source code?

Yes, all retry and timing parameters are controlled through environment variables. Set PRICE_FETCH_MAX_RETRIES to change attempt counts, PRICE_FETCH_BACKOFF_BASE_SECONDS to adjust exponential delay scaling, and PRICE_FETCH_RATE_LIMIT_COOLDOWN_SECONDS to customize how long the system waits after hitting rate limits.

How long does the Polymarket token cache remain valid?

The _polymarket_token_cache stores resolved token IDs with a TTL (time-to-live) of 300 seconds (5 minutes) as defined by _POLYMARKET_TOKEN_CACHE_TTL_S. After this period expires, the next lookup for the same reference tuple will trigger a fresh network request to retrieve current token data.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}