# Price API Rate Limiting and Caching Strategies in AI-Trader: A Complete Guide

> Learn AI-Trader's Price API rate limiting and caching strategies. Discover provider cooldowns, back-off, and in-memory caching for efficient API usage.

- Repository: [✨Data Intelligence Lab@HKU✨/AI-Trader](https://github.com/HKUDS/AI-Trader)
- Tags: how-to-guide
- Published: 2026-05-09

---

**AI-Trader implements a layered defense system in [`service/server/price_fetcher.py`](https://github.com/HKUDS/AI-Trader/blob/main/service/server/price_fetcher.py) that combines provider-wide cooldowns, automatic rate-limit detection, exponential back-off, and in-memory caching to prevent external API saturation while minimizing redundant network traffic.**

The HKUDS/AI-Trader repository handles high-frequency price fetching from multiple market data providers. To maintain reliable access without triggering provider bans or wasting bandwidth on duplicate lookups, the codebase employs sophisticated **price API rate limiting and caching** techniques that are both self-healing and configurable at runtime.

## Provider-Wide Rate Limiting

The foundation of AI-Trader's rate limiting is a global cooldown mechanism that tracks provider health across all requests. This prevents the system from hammering APIs that have already signaled distress.

### Cooldown Map Implementation

At the module level, **`_provider_cooldowns`** (lines 41-44) stores the earliest timestamp when each provider may be called again. Before every network request, **`_provider_cooldown_remaining()`** (lines 59-60) checks this map. If the current time is earlier than the stored timestamp, the request aborts immediately with an error, ensuring zero additional load on struggling endpoints.

```python

# Internal logic flow (simplified)

if _provider_cooldown_remaining("alphavantage") > 0:
    raise CooldownError(f"Provider cooling down for {remaining}s")

```

### Automatic HTTP 429 Handling

When a provider returns **HTTP 429 (Too Many Requests)**, the fetcher automatically invokes **`_activate_provider_cooldown()`** (lines 122-128). This sets a mandatory pause using the **`PRICE_FETCH_RATE_LIMIT_COOLDOWN_SECONDS`** environment variable (default 60 seconds), preventing further calls until the cooldown expires.

### Server Error Protection (5xx)

Transient server errors trigger a shorter safety pause. Any status code ≥ 500 activates a cooldown defined by **`PRICE_FETCH_ERROR_COOLDOWN_SECONDS`** (default 20 seconds) via lines 128-134. This protects against cascading failures while allowing quicker recovery than hard rate limits.

## Intelligent Retry Logic

### Exponential Back-off with Jitter

For recoverable errors including 429s, 5xxs, timeouts, and connection failures, AI-Trader implements **`_retry_delay()`** (lines 73-78). The delay calculates as:

```

delay = PRICE_FETCH_BACKOFF_BASE_SECONDS × (2 ^ attempt_number) + random_jitter

```

The **`_request_json_with_retry()`** function (lines 112-145) orchestrates this logic, sleeping between attempts according to the computed delay and respecting the **`PRICE_FETCH_MAX_RETRIES`** limit.

## Caching Mechanisms

### Polymarket Token Resolution Cache

To eliminate redundant blockchain lookups, the system maintains **`_polymarket_token_cache`** (lines 54-57), an in-memory dictionary storing resolved token IDs for `(reference, token_id, outcome)` tuples. Entries expire after **`_POLYMARKET_TOKEN_CACHE_TTL_S`** (300 seconds/5 minutes), after which the next call refreshes the data.

The cache check occurs in **`_polymarket_resolve_reference()`** (lines 26-33), short-circuiting network requests when valid cached data exists.

### Environment-Based Configuration

All behavioral parameters are externally configurable without code changes:

- **`PRICE_FETCH_RATE_LIMIT_COOLDOWN_SECONDS`**: Duration to pause after HTTP 429
- **`PRICE_FETCH_ERROR_COOLDOWN_SECONDS`**: Duration to pause after 5xx errors  
- **`PRICE_FETCH_MAX_RETRIES`**: Maximum retry attempts per request
- **`PRICE_FETCH_BACKOFF_BASE_SECONDS`**: Base delay for exponential back-off
- **`PRICE_FETCH_TIMEOUT_SECONDS`**: Request timeout threshold

## Implementation Examples

Configure the rate limiting behavior via environment variables before initializing the fetcher:

```python
import os

os.environ["PRICE_FETCH_RATE_LIMIT_COOLDOWN_SECONDS"] = "45"   # 45s pause after 429

os.environ["PRICE_FETCH_ERROR_COOLDOWN_SECONDS"] = "15"      # 15s pause after 5xx

os.environ["PRICE_FETCH_MAX_RETRIES"] = "3"                 # maximum 3 retries

os.environ["PRICE_FETCH_BACKOFF_BASE_SECONDS"] = "0.5"      # 0.5s base back-off

os.environ["PRICE_FETCH_TIMEOUT_SECONDS"] = "8"            # 8s request timeout

```

Fetch prices with automatic rate-limit handling:

```python
from service.server.price_fetcher import get_price_from_market

price = get_price_from_market(
    symbol="AAPL",
    executed_at="2024-11-01T14:30:00Z",
    market="us-stock"
)

if price is not None:
    print(f"Fetched price: ${price:.2f}")
else:
    print("Price unavailable (provider in cooldown)")

```

Simulate and check cooldown status programmatically:

```python
from service.server.price_fetcher import _activate_provider_cooldown, _provider_cooldown_remaining

# Simulate hitting a rate limit

_activate_provider_cooldown("alphavantage", 60, "HTTP 429")
remaining = _provider_cooldown_remaining("alphavantage")
print(f"Cooldown remaining: {remaining}s")  # Value close to 60

```

Leverage the Polymarket cache for repeated lookups:

```python
from service.server.price_fetcher import _polymarket_resolve_reference

# First call performs network request

contract = _polymarket_resolve_reference("btc-usd")

# Second call within 5 minutes returns cached data instantly

contract_cached = _polymarket_resolve_reference("btc-usd")

```

## Summary

AI-Trader's approach to **price API rate limiting and caching** delivers a self-throttling client through five key mechanisms:

- **Pre-flight cooldown checks** guarantee no requests hit providers already flagged as rate-limited
- **Automatic 429 detection** triggers configurable cooling periods to respect provider limits
- **5xx error handling** applies shorter pauses to handle transient server issues gracefully
- **Jittered exponential back-off** reduces burst traffic while maximizing eventual success rates
- **TTL-based in-memory caching** for Polymarket token resolution eliminates redundant HTTP round-trips for up to five minutes

## Frequently Asked Questions

### How does AI-Trader prevent hitting rate limits repeatedly?

The system maintains a global `_provider_cooldowns` dictionary in [`service/server/price_fetcher.py`](https://github.com/HKUDS/AI-Trader/blob/main/service/server/price_fetcher.py) that tracks when each provider was last rate-limited. Before every request, `_provider_cooldown_remaining()` verifies that enough time has passed since the last 429 or 5xx error. If the cooldown is still active, the request fails fast locally without touching the external API.

### What happens when a price API returns HTTP 429?

When the fetcher receives an HTTP 429 status, `_activate_provider_cooldown()` immediately registers a cooldown period (default 60 seconds) for that specific provider. Subsequent calls to `get_price_from_market()` for that provider will return errors until the cooldown expires, preventing further rate limit violations.

### Can I adjust the retry behavior without modifying the source code?

Yes, all retry and timing parameters are controlled through environment variables. Set `PRICE_FETCH_MAX_RETRIES` to change attempt counts, `PRICE_FETCH_BACKOFF_BASE_SECONDS` to adjust exponential delay scaling, and `PRICE_FETCH_RATE_LIMIT_COOLDOWN_SECONDS` to customize how long the system waits after hitting rate limits.

### How long does the Polymarket token cache remain valid?

The `_polymarket_token_cache` stores resolved token IDs with a TTL (time-to-live) of 300 seconds (5 minutes) as defined by `_POLYMARKET_TOKEN_CACHE_TTL_S`. After this period expires, the next lookup for the same reference tuple will trigger a fresh network request to retrieve current token data.