# Rate Limiting System in Free-Claude-Code: Implementing Strict Rolling Windows

> Discover how Free-Claude-Code uses a strict rolling-window rate limiting system with reactive blocking, proactive throttling, and concurrency caps to prevent API quota violations. Learn about the GlobalRateLimiter class impleme...

- Repository: [Ali Khokhar/free-claude-code](https://github.com/Alishahryar1/free-claude-code)
- Tags: internals
- Published: 2026-04-24

---

**Free-Claude-Code implements a strict rolling-window rate limiting system through a singleton `GlobalRateLimiter` class that combines reactive blocking, proactive throttling using a monotonic timestamp deque, and concurrency caps to prevent API quota violations.**

The `Alishahryar1/free-claude-code` repository enforces polite API consumption through a sophisticated throttling mechanism defined in [`providers/rate_limit.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/rate_limit.py). At its core sits the **`GlobalRateLimiter`**, a singleton that coordinates request pacing across all LLM providers using a strict rolling-window algorithm rather than fixed-interval buckets.

## Core Architecture of the Global Rate Limiter

### The Singleton Pattern for Global Coordination

The rate limiting system relies on **`GlobalRateLimiter.get_instance()`** to lazily construct a single shared instance on first invocation. This singleton design enables global coordination across all provider modules, ensuring that disparate API calls respect a unified quota regardless of which specific provider (OpenAI, LMStudio, NVIDIA NIM) initiates the request.

### Three-Layer Defense Strategy

The implementation combines three complementary mechanisms to enforce quota compliance:

- **Reactive Blocking**: Uses `self._blocked_until` and `wait_if_blocked()` to pause execution when a provider returns a 429 error. The `set_blocked(delay)` method records a future timestamp, and subsequent calls to `wait_if_blocked()` sleep until that time passes.
- **Proactive Throttling**: Implements a strict rolling window using `_request_times: deque[float]` and `_acquire_proactive_slot()`. This guarantees no more than `rate_limit` requests occur within any sliding interval of length `rate_window`.
- **Concurrency Caps**: An `asyncio.Semaphore` managed through `concurrency_slot()` limits simultaneous open connections (e.g., SSE streams), preventing resource oversubscription.

## How the Rolling Window Algorithm Works

### The Timestamp Deque

Inside [`providers/rate_limit.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/rate_limit.py), the limiter maintains **`_request_times`**, a `collections.deque` storing monotonic timestamps of successful requests. Before granting a new request slot, the algorithm trims this deque to remove entries older than `now - self._rate_window`.

### Slot Acquisition Logic

The **`_acquire_proactive_slot()`** method executes the core rolling-window logic under an `asyncio.Lock`:

1. **Window Pruning**: Removes timestamps falling outside the current window.
2. **Limit Check**: Grants the slot immediately if fewer than `self._rate_limit` timestamps remain.
3. **Calculated Delay**: If the limit is reached, computes the exact sleep time required for the oldest timestamp to expire: `(oldest + self._rate_window) - now`.
4. **Non-Blocking Sleep**: Releases the lock before sleeping, allowing other coroutines to queue.

This approach eliminates burst behavior at window boundaries, ensuring **strict quota enforcement** where any arbitrary interval of length `rate_window` contains at most `rate_limit` requests.

## Reactive Blocking and Retry Handling

### Handling 429 Errors with set_blocked

When a provider returns a `RateLimitError`, the **`set_blocked(delay)`** method records a future timestamp in `self._blocked_until`. The **`wait_if_blocked()`** coroutine checks this timestamp before proceeding, implementing an immediate circuit-breaker pattern that prevents retry storms.

### Exponential Back-off with execute_with_retry

The **`execute_with_retry()`** helper combines proactive throttling with reactive recovery:

```python
async def execute_with_retry(self, func, max_retries=3):
    for attempt in range(max_retries):
        await self.wait_if_blocked()  # Check reactive + proactive limits

        try:
            return await func()
        except RateLimitError:
            delay = calculate_backoff(attempt)
            self.set_blocked(delay)  # Trigger reactive block

            await asyncio.sleep(delay)

```

This wrapper ensures that 429 responses trigger both the rolling-window adjustment and exponential back-off before subsequent attempts.

## Concurrency Control with Async Semaphores

Independent of the sliding window, the **`concurrency_slot()`** async context manager acquires `self._concurrency_sem`, an `asyncio.Semaphore` initialized with `max_concurrency`. This limits simultaneously open provider streams—critical for SSE connections or batch operations—ensuring that rate limits are not circumvented through parallel request flooding.

## Practical Implementation Examples

### Basic Provider Integration

```python
from providers.rate_limit import GlobalRateLimiter

limiter = GlobalRateLimiter.get_instance(rate_limit=40, rate_window=60)

async def call_api(payload):
    # Wait for both reactive blocks and proactive slots

    await limiter.wait_if_blocked()
    response = await http_client.post("/v1/chat", json=payload)
    return response

```

### Automatic Retry Wrapping

```python
async def chat(messages):
    async def do_request():
        return await http_client.post("/v1/chat", json={"messages": messages})

    limiter = GlobalRateLimiter.get_instance()
    # Retries 3 times on 429 with back-off

    return await limiter.execute_with_retry(do_request, max_retries=3)

```

### Limiting Concurrent SSE Streams

```python
limiter = GlobalRateLimiter.get_instance(max_concurrency=2)

async def stream_events():
    async with limiter.concurrency_slot():
        async for event in sse_client.iter_events():
            process(event)  # Only 2 streams run in parallel

```

### Validating Rolling-Window Guarantees

```python
import asyncio
import time
from providers.rate_limit import GlobalRateLimiter

async def demo():
    GlobalRateLimiter.reset_instance()
    lim = GlobalRateLimiter.get_instance(rate_limit=2, rate_window=0.5)
    
    timestamps = []
    
    async def acquire():
        await lim.wait_if_blocked()
        timestamps.append(time.monotonic())
    
    await asyncio.gather(*(acquire() for _ in range(5)))
    timestamps.sort()
    print(timestamps)  # Observe ≥0.5s spacing between every 2nd request

```

## Summary

- **GlobalRateLimiter** in [`providers/rate_limit.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/providers/rate_limit.py) acts as a singleton coordinator for all API requests.
- **Strict rolling windows** use a `deque` of monotonic timestamps to guarantee no more than `rate_limit` requests occur within any `rate_window` interval.
- **_acquire_proactive_slot()** calculates precise sleep times based on the oldest timestamp in the window, preventing burst behavior.
- **Reactive blocking** via `set_blocked()` and `wait_if_blocked()` handles 429 errors with immediate back-off.
- **Concurrency limits** are enforced independently through an `asyncio.Semaphore` via `concurrency_slot()`.
- **Comprehensive tests** in [`tests/providers/test_provider_rate_limit.py`](https://github.com/Alishahryar1/free-claude-code/blob/main/tests/providers/test_provider_rate_limit.py) validate singleton semantics, zero-value guards, and strict window enforcement.

## Frequently Asked Questions

### What is a strict rolling window in rate limiting?

A strict rolling window (or sliding window) tracks the exact timestamps of recent requests and enforces that no more than `rate_limit` requests have occurred within the past `rate_window` seconds. Unlike fixed windows that reset at interval boundaries, this approach prevents burst attacks at window edges and guarantees quota compliance for any arbitrary time slice of the specified duration.

### How does GlobalRateLimiter handle 429 rate limit errors?

When a provider returns a 429 status, the code calls `set_blocked(delay)`, which sets `self._blocked_until` to a future timestamp. The `wait_if_blocked()` method checks this timestamp before each request and pauses execution until the block expires. This reactive mechanism works alongside the proactive rolling window to prevent retry loops while the API quota recovers.

### Why use a deque instead of a fixed bucket for the rolling window?

The `_request_times` deque stores actual request timestamps, allowing the algorithm to calculate the exact moment the oldest request falls outside the window. Fixed-token buckets allow bursts at refill time, whereas the deque-based approach enforces **strict** pacing by requiring the oldest timestamp to age out before granting new slots, matching the semantics of commercial APIs like OpenAI's rate limits.

### Can the concurrency limit be adjusted independently of the rate window?

Yes. The `max_concurrency` parameter controls the `asyncio.Semaphore` initialized separately from `rate_limit` and `rate_window`. You can configure `GlobalRateLimiter.get_instance(rate_limit=20, rate_window=60, max_concurrency=5)` to allow five simultaneous connections while still enforcing only 20 requests per minute, preventing resource exhaustion without affecting the rolling-window throttle.