# How the Sliding Window Log Rate Limiter Works and Its Drawbacks

> Explore the Sliding Window Log rate limiter's timestamp tracking for accurate limits. Understand its drawbacks: high memory use and computational overhead.

- Repository: [Gaurav Kumar/system-design-notes](https://github.com/liquidslr/system-design-notes)
- Tags: deep-dive
- Published: 2026-09-11

---

**The Sliding Window Log rate limiter maintains a precise log of request timestamps to enforce rolling-window limits, offering accuracy at the cost of high memory usage and computational overhead.**

The Sliding Window Log algorithm provides precise, rolling-window rate limiting by tracking every request timestamp within a configurable time frame. According to the `liquidslr/system-design-notes` repository, this method stores individual arrival times to calculate exact request counts over any interval, eliminating the burst-edge issues common in fixed-window approaches. While it delivers superior accuracy, the design incurs significant memory and performance penalties that make it unsuitable for high-throughput distributed systems without careful optimization.

## How Sliding Window Log Rate Limiting Works

As documented in `04. Rate Limiter/Readme.md`, the algorithm operates through a continuous cycle of recording, pruning, and counting【L86-L94】. For every incoming request, the limiter executes three distinct phases:

1. **Record the request timestamp**: The system captures the current time (typically Unix epoch in milliseconds) and appends it to a sorted log structure such as a Redis Sorted Set or an in-memory slice.
2. **Remove expired entries**: The limiter calculates the window cutoff (`now - windowSize`) and deletes all timestamps falling outside this rolling boundary.
3. **Evaluate the quota**: If the remaining entry count exceeds the configured limit, the request is rejected; otherwise, it is allowed and the timestamp persists in the log.

### Precision Without Edge Cases

Unlike fixed-window counters that reset at rigid interval boundaries, the Sliding Window Log continuously discards old entries as time progresses. This rolling calculation ensures that a burst of requests straddling two adjacent windows cannot exceed the quota, providing what the source notes describe as "accurate rate limiting" without window-edge spikes【L86-L94】.

## Critical Drawbacks of Sliding Window Log Rate Limiting

The repository explicitly flags "high memory consumption" as a primary architectural concern【L91-L94】. The algorithm's accuracy comes with three major trade-offs:

### High Memory Consumption

Every accepted request requires persistent storage of its timestamp. Under heavy traffic, the log grows linearly with request volume, consuming significant RAM whether using local slices or external data stores like Redis. This consumption scales proportionally with the request rate and window duration, making the approach memory-inefficient for high-throughput services.

### Performance Overhead

Each request triggers both a write operation (inserting the new timestamp) and a cleanup scan (pruning expired entries). In naive implementations, the pruning phase requires iterating through the log to find the cutoff point, resulting in O(n) complexity where *n* is the number of stored timestamps. This dual operation creates CPU overhead that increases as the log grows.

### Distributed Scalability Challenges

Synchronizing timestamp logs across multiple nodes adds significant latency and complexity. Distributed deployments require strongly consistent data stores or complex Lua scripts in Redis to maintain atomicity during the read-prune-write cycle. These coordination mechanisms introduce network roundtrips and potential contention points that limit horizontal scalability.

## Implementation Examples

The following code samples illustrate the core logic: inserting the current timestamp, removing entries outside the window, counting the remainder, and enforcing the limit.

### Python Implementation with Redis

```python
import time
import redis

redis_client = redis.StrictRedis(host='localhost', port=6379, db=0)

def sliding_window_log(key: str, limit: int, window_seconds: int) -> bool:
    now = int(time.time() * 1000)                     # current timestamp in ms

    window_start = now - (window_seconds * 1000)

    # Add current request timestamp

    redis_client.zadd(key, {now: now})

    # Remove timestamps outside the window

    redis_client.zremrangebyscore(key, 0, window_start)

    # Get current request count

    count = redis_client.zcard(key)

    # Optional: set TTL to auto‑expire the key

    redis_client.expire(key, window_seconds + 1)

    return count <= limit   # True → allowed, False → throttled

```

### Go Implementation with In-Memory Slice

```go
type SlidingWindow struct {
    timestamps []int64 // stored in ascending order
    limit      int
    window    time.Duration
    mu        sync.Mutex
}

func NewSlidingWindow(limit int, window time.Duration) *SlidingWindow {
    return &SlidingWindow{limit: limit, window: window}
}

func (sw *SlidingWindow) Allow() bool {
    sw.mu.Lock()
    defer sw.mu.Unlock()

    now := time.Now().UnixNano()
    cutoff := now - sw.window.Nanoseconds()

    // Discard old timestamps
    i := 0
    for ; i < len(sw.timestamps); i++ {
        if sw.timestamps[i] >= cutoff {
            break
        }
    }
    sw.timestamps = sw.timestamps[i:]

    // Check limit
    if len(sw.timestamps) >= sw.limit {
        return false // throttled
    }

    // Record this request
    sw.timestamps = append(sw.timestamps, now)
    return true
}

```

## Summary

- The Sliding Window Log algorithm tracks every request timestamp to enforce precise, rolling-window rate limits without edge-case bursts.
- As noted in `04. Rate Limiter/Readme.md`, the approach provides "accurate rate limiting" by continuously pruning expired entries and counting the remaining log【L86-L94】.
- **High memory consumption** represents the primary drawback, as the log size scales linearly with request traffic【L91-L94】.
- Performance overhead increases with log size due to mandatory cleanup operations on every request.
- Distributed implementations require complex coordination mechanisms, making horizontal scaling challenging compared to token bucket or fixed-window alternatives.

## Frequently Asked Questions

### What makes Sliding Window Log different from Fixed Window rate limiting?

Fixed Window counters reset at specific interval boundaries (e.g., every 60 seconds), allowing potential bursts of twice the limit at window edges. Sliding Window Log calculates the exact number of requests within the trailing time window continuously, preventing these edge-case violations by strictly enforcing the quota over any arbitrary interval.

### Why is memory consumption high in Sliding Window Log implementations?

The algorithm stores a discrete timestamp for every accepted request within the active window. For a service handling thousands of requests per second with a 60-second window, this requires storing 60,000+ timestamps per rate-limiting key. Redis Sorted Sets or in-memory slices must maintain this entire dataset, consuming RAM proportional to request volume and window duration.

### Can Sliding Window Log scale horizontally across multiple servers?

Horizontal scaling requires shared state between instances, typically using Redis or similar data stores with atomic Lua scripts to handle the read-prune-write cycle. This introduces network latency and potential bottlenecks at the central data store. Unlike stateless token bucket algorithms, Sliding Window Log requires strong consistency to maintain accuracy, complicating distributed deployments.

### When should I use Sliding Window Log over other rate-limiting algorithms?

Use Sliding Window Log when accuracy is critical and request volume is moderate—such as API tiers requiring strict compliance with SLAs or payment processing endpoints. Avoid it for high-throughput public gateways or edge services where memory efficiency and horizontal scalability take precedence over sub-second precision.