# Rate Limiting Techniques for Java Microservices: 5 Algorithms and Frameworks Explained

> Explore rate limiting techniques for Java microservices. Discover 5 algorithms and frameworks like Token Bucket and Spring Cloud Gateway to protect your system from traffic spikes.

- Repository: [Doocs/advanced-java](https://github.com/doocs/advanced-java)
- Tags: deep-dive
- Published: 2026-02-28

---

**Rate limiting techniques for Java microservices protect distributed systems from traffic spikes using algorithms like Token Bucket and framework integrations such as Spring Cloud Gateway with Redis.**

The **doocs/advanced-java** repository provides production-ready implementations of classic throttling algorithms and enterprise framework configurations. Understanding these patterns is essential for maintaining system stability and preventing cascading failures in high-concurrency environments.

## Core Rate Limiting Algorithms

The repository documents four fundamental algorithms in [`docs/high-concurrency/how-to-limit-current.md`](https://github.com/doocs/advanced-java/blob/main/docs/high-concurrency/how-to-limit-current.md), each offering different trade-offs between simplicity and burst handling.

### Fixed Window Counter

The simplest approach tracks request counts within a fixed time window. While easy to implement, it suffers from burst issues at window boundaries.

In the source, the `Counter` class uses an `AtomicInteger` to track requests against a configurable `limit` per `timeout` milliseconds:

```java
public class Counter {
    private final int limit = 10;
    private final long timeout = 1000L;
    private long windowStart = System.currentTimeMillis();
    private final AtomicInteger reqCount = new AtomicInteger(0);

    public boolean limit() {
        long now = System.currentTimeMillis();
        if (now - windowStart < timeout) {
            return reqCount.incrementAndGet() <= limit;
        } else {
            windowStart = now;
            reqCount.set(1);
            return true;
        }
    }
}

```

This implementation resets the counter atomically when the window expires, as shown in lines 15-49 of the concurrency documentation.

### Sliding Window Counter

To smooth traffic spikes, the **sliding window** divides time into smaller slots. The `TimeWindow` class uses a `ConcurrentLinkedQueue` to store timestamps and a background cleaner thread to remove expired entries.

```java
public class TimeWindow {
    private final ConcurrentLinkedQueue<Long> queue = new ConcurrentLinkedQueue<>();
    private final int max;          // max requests per window
    private final int seconds;      // window length

    public TimeWindow(int max, int seconds) {
        this.max = max;
        this.seconds = seconds;
    }

    public boolean tryAcquire() {
        long now = System.currentTimeMillis();
        clean(now);
        if (queue.size() >= max) {
            return false;
        }
        queue.offer(now);
        return true;
    }

    private void clean(long now) {
        long limit = now - seconds * 1000L;
        while (true) {
            Long ts = queue.peek();
            if (ts == null || ts > limit) break;
            queue.poll();
        }
    }
}

```

This approach provides smoother rate distribution compared to the fixed window, implemented in lines 60-84.

### Leaky Bucket Algorithm

The **leaky bucket** models traffic as water flowing into a bucket with a fixed outflow rate. When the bucket overflows, requests drop. The `LeakBucket` class maintains a `nowSize` field representing current water level and drains it at a constant `rate`:

```java
public class LeakBucket {
    private final int capacity;     // bucket capacity
    private final int rate;         // outflow rate per second
    private int nowSize;            // current water level
    private long lastLeakTime;

    public synchronized boolean tryConsume() {
        leak();
        if (nowSize < capacity) {
            nowSize++;
            return true;
        }
        return false;
    }

    private void leak() {
        long now = System.currentTimeMillis();
        int leakAmount = (int) ((now - lastLeakTime) / 1000 * rate);
        nowSize = Math.max(0, nowSize - leakAmount);
        lastLeakTime = now;
    }
}

```

This algorithm enforces a strict output rate regardless of input burstiness, detailed in lines 90-124.

### Token Bucket Algorithm

The most flexible algorithm for microservices, **token bucket** adds tokens at a steady rate while requests consume them. The `TokenBucket` implementation refills tokens based on elapsed time between requests:

```java
public class TokenBucket {
    private final double capacity;   // max tokens
    private final double rate;       // tokens added per millisecond
    private double tokens;
    private long lastRefill;

    public TokenBucket(double capacity, double rate) {
        this.capacity = capacity;
        this.rate = rate;
        this.tokens = capacity;
        this.lastRefill = System.currentTimeMillis();
    }

    public synchronized boolean tryConsume() {
        refill();
        if (tokens < 1) return false;
        tokens -= 1;
        return true;
    }

    private void refill() {
        long now = System.currentTimeMillis();
        double added = (now - lastRefill) * rate;
        tokens = Math.min(capacity, tokens + added);
        lastRefill = now;
    }
}

```

This allows configurable burst capacity while maintaining long-term rate limits, as implemented in lines 130-168.

## Framework-Based Rate Limiting Solutions

For production microservices, the repository recommends integrating with established frameworks rather than custom implementations.

### Spring Cloud Gateway with Redis

**Spring Cloud Gateway** provides distributed rate limiting using Redis as a backing store for a token-bucket implementation. Configuration uses [`application.yml`](https://github.com/doocs/advanced-java/blob/main/application.yml) with a `KeyResolver` bean to extract client identifiers (typically IP addresses):

```yaml
spring:
  cloud:
    gateway:
      routes:
        - id: requestratelimiter_route
          uri: lb://my-service
          predicates:
            - Path=/api/**
          filters:
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 5
                redis-rate-limiter.burstCapacity: 10
                key-resolver: '#{@remoteAddrKeyResolver}'

```

The corresponding `KeyResolver` bean:

```java
@Bean
public KeyResolver remoteAddrKeyResolver() {
    return exchange -> Mono.just(
        Objects.requireNonNull(exchange.getRequest().getRemoteAddress())
               .getAddress().getHostAddress());
}

```

This configuration applies per-IP limiting across all gateway instances, documented in the repository's gateway section.

### Alibaba Sentinel

**Sentinel** offers declarative flow control via rules stored in Nacos. It supports QPS limiting, thread-count constraints, and advanced strategies like warm-up and queueing:

```json
[
  {
    "resource": "/order",
    "limitApp": "default",
    "grade": 1,
    "count": 20,
    "strategy": 0,
    "controlBehavior": 0,
    "clusterMode": false
  }
]

```

The `grade: 1` indicates QPS-based limiting, while `grade: 0` would indicate thread-count limiting. This approach centralizes rate limiting policies in Nacos for dynamic updates without deployment.

### Hystrix Thread Pool Isolation

For concurrency-based limiting rather than rate-based, **Hystrix** isolates commands in dedicated thread pools. According to [`docs/high-availability/hystrix-thread-pool-current-limiting.md`](https://github.com/doocs/advanced-java/blob/main/docs/high-availability/hystrix-thread-pool-current-limiting.md), this approach limits simultaneous calls per command key, preventing slow dependencies from exhausting shared thread pools.

Configure thread pool size and queue capacity:

```java
HystrixCommand.Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"))
    .andCommandPropertiesDefaults(
        HystrixCommandProperties.Setter()
            .withExecutionIsolationThreadTimeoutInMilliseconds(500))
    .andThreadPoolPropertiesDefaults(
        HystrixThreadPoolProperties.Setter()
            .withCoreSize(10)
            .withMaxQueueSize(100));

```

This protects downstream services from being overwhelmed by concurrent requests, complementing rate-based algorithms.

## Architectural Best Practices for Microservices

When implementing rate limiting techniques for Java microservices, consider these strategies from the **doocs/advanced-java** repository:

- **Choose appropriate granularity**: Per-endpoint limits protect specific resources, while per-user limits prevent individual clients from monopolizing capacity. The Spring Cloud Gateway `KeyResolver` demonstrates per-IP limiting.
- **Use distributed stores for multi-instance deployments**: Local counters only protect a single JVM. Redis-backed solutions ensure consistent limiting across horizontally scaled services.
- **Layer multiple algorithms**: Combine an in-memory token bucket for fast-path rejection with a Redis fallback for distributed burst protection.
- **Monitor and instrument**: Expose QPS, rejection counts, and bucket fill levels via Micrometer or Prometheus. The repository emphasizes tuning limits based on observed metrics rather than theoretical calculations.

## Summary

- **Fixed window counters** offer simple implementation but allow bursts at window boundaries, implemented via `AtomicInteger` in [`docs/high-concurrency/how-to-limit-current.md`](https://github.com/doocs/advanced-java/blob/main/docs/high-concurrency/how-to-limit-current.md).
- **Sliding window counters** smooth traffic using `ConcurrentLinkedQueue` to track individual request timestamps.
- **Leaky bucket** algorithms enforce strict output rates using a drain mechanism with `nowSize` tracking.
- **Token bucket** implementations provide flexible burst handling by calculating token refill based on elapsed time between requests.
- **Framework integrations** like Spring Cloud Gateway (Redis), Sentinel (Nacos rules), and Hystrix (thread pools) provide production-ready distributed limiting for microservices architectures.

## Frequently Asked Questions

### What is the difference between leaky bucket and token bucket algorithms?

The **leaky bucket** enforces a strict, constant outflow rate regardless of input patterns, dropping excess traffic immediately. It is ideal for smoothing bursty traffic into steady streams. The **token bucket** allows bursts up to the bucket capacity while maintaining a long-term average rate, making it more flexible for microservices that need to handle occasional traffic spikes without dropping requests.

### How does Spring Cloud Gateway implement distributed rate limiting?

Spring Cloud Gateway uses **Redis** as a shared state store to implement a distributed token bucket. The `RequestRateLimiter` filter calculates token availability across all gateway instances using Redis Lua scripts for atomic operations. The `KeyResolver` bean determines the limiting granularity (IP, user, or endpoint), ensuring consistent enforcement in clustered deployments.

### When should I use Sentinel instead of Gateway rate limiting?

Use **Alibaba Sentinel** when you need **centralized, dynamic rule management** via Nacos or when requiring advanced control behaviors like warm-up (gradual ramp-up) or uniform rate pacing. Use **Spring Cloud Gateway** limiting for edge-level protection at the API gateway layer. Often, both are used together: Gateway for coarse-grained perimeter defense and Sentinel for fine-grained service-level protection.

### Can thread pool limiting replace rate limiting in Hystrix?

No, they serve different purposes. **Thread pool limiting** restricts concurrent execution to prevent resource exhaustion from slow dependencies, but it does not limit requests per second. A service could receive 1000 QPS while the thread pool only allows 10 concurrent executions, queueing or rejecting the rest. For complete protection, combine Hystrix thread pools with a rate limiter (Token Bucket or Sentinel) to control both throughput and concurrency.