Easegress Rate Limiting: Architecture, Configuration, and Token-Bucket Implementation

Easegress implements rate limiting through a dual-layer token-bucket algorithm with three operational states (Normal, Limiting, Disabled), enabling per-URL throttling via the RateLimiter filter that returns HTTP 429 Too Many Requests when limits are exceeded.

Easegress rate limiting is built into the easegress-io/easegress repository as a configurable filter that protects backend services from overload. The implementation combines a generic token-bucket utility with an HTTP-specific filter layer, allowing administrators to define fine-grained policies that match specific URL patterns and enforce throttling with configurable timeouts.

Core Token-Bucket Implementation

The foundation of Easegress rate limiting resides in pkg/util/ratelimiter/ratelimiter.go. This utility provides a reusable, thread-safe rate limiter that tracks tokens across configurable time windows.

Policy Structure and State Machine

The Policy struct defines three critical parameters (lines 34-38):

• TimeoutDuration: Maximum time a request waits before rejection
• LimitRefreshPeriod: Interval for refilling tokens
• LimitForPeriod: Maximum tokens available per period

The state machine operates in three distinct states (lines 62-66):

• StateNormal: Traffic flows freely
• StateLimiting: Bucket exhausted; requests face delays
• StateDisabled: Limiter inactive; all requests pass through

Token Acquisition Logic

The acquirePermission function (lines 29-85) implements the core algorithm. It calculates remaining tokens for the current cycle and either grants permission immediately or returns a wait duration. When the bucket empties, it transitions to StateLimiting and triggers notifyListener to inform observers of state changes.

The public API exposes AcquirePermission(), AcquireNPermission(), and WaitPermission() for synchronous blocking operations.

HTTP Filter Layer

The HTTP-specific implementation in pkg/filters/ratelimiter/ratelimiter.go wraps the utility limiter, binding policies to URL patterns and managing request lifecycle.

Configuration Model

The Spec struct (lines 73-84) defines the filter configuration:

• policies: Named policy definitions
• defaultPolicyRef: Fallback policy name
• urls: URL patterns with associated policy references

Each URLRule embeds urlrule.URLRule and maintains a runtime *librl.RateLimiter instance (lines 66-71).

Request Handling and State Transitions

During Init/Inherit, each URLRule executes three steps:

1. Policy Binding (bindPolicyToURL): Selects the appropriate policy, falling back to DefaultPolicyRef (lines 64-74)
2. Limiter Creation (createRateLimiter): Instantiates the utility limiter with the resolved policy (lines 13-35)
3. Listener Attachment (setStateListenerForURL): Registers a callback that logs state transitions via the Easegress logger (lines 52-60)

The Handle method processes each request:

• Matches the request URL against configured rules (u.Match) (lines 51-55)
• Attempts token acquisition (u.rl.AcquirePermission()) (line 57)

When denied (!permitted), the filter returns 429 Too Many Requests with the header X-EG-Rate-Limiter: too-many-requests and the result rateLimited (lines 58-70).

When allowed with delay (d > 0), the filter asynchronously waits for the specified duration or request cancellation before continuing (lines 73-85).

Configuration Examples

YAML Configuration

Configure the RateLimiter filter in your Easegress pipeline:

apiVersion: v2.easegress.io/v2
kind: RateLimiter
metadata:
  name: api-rate-limiter
spec:
  defaultPolicyRef: "default"
  policies:
    - name: "default"
      timeoutDuration: "100ms"
      limitRefreshPeriod: "10ms"
      limitForPeriod: 50
    - name: "login"
      timeoutDuration: "200ms"
      limitRefreshPeriod: "20ms"
      limitForPeriod: 10
  urls:
    - id: "all-api"
      url: "/api/*"
      policyRef: "default"
    - id: "login-endpoint"
      url: "/api/login"
      policyRef: "login"

This configuration creates separate token buckets for general API traffic (50 requests per 10ms) and the login endpoint (10 requests per 20ms). The urls list ties a URL pattern to a named policy, and the filter creates a separate token bucket for each rule.

Programmatic Usage

Import the utility limiter directly for custom components:

import (
    "time"
    rlutil "github.com/megaease/easegress/v2/pkg/util/ratelimiter"
    filt "github.com/megaease/easegress/v2/pkg/filters/ratelimiter"
)

// Create a utility limiter directly (useful for custom components)
policy := rlutil.Policy{
    LimitForPeriod:     100,
    LimitRefreshPeriod: 10 * time.Millisecond,
    TimeoutDuration:    100 * time.Millisecond,
}
limiter := rlutil.New(&policy)

// Acquire a token
ok, wait := limiter.AcquirePermission()
if !ok {
    // reject request
}
if wait > 0 {
    time.Sleep(wait) // respect the required delay
}

The code mirrors the methods defined in the utility file (AcquirePermission, AcquireNPermission, WaitPermission).

Monitor state transitions:

limiter.SetStateListener(func(e *rlutil.Event) {
    fmt.Printf("Rate limiter transitioned to %s at %v\n", e.State, e.Time)
})

The filter automatically registers a listener that logs to the Easegress logger (see setStateListenerForURL in the filter source).

Summary

• Easegress rate limiting combines a generic token-bucket utility with an HTTP-specific filter for per-URL throttling.
• The core algorithm in pkg/util/ratelimiter/ratelimiter.go manages three states (StateNormal, StateLimiting, StateDisabled) and calculates wait times when tokens are exhausted.
• The HTTP filter in pkg/filters/ratelimiter/ratelimiter.go binds policies to URL patterns, returns 429 Too Many Requests with the X-EG-Rate-Limiter header when limits are exceeded, and supports asynchronous waiting for delayed permits.
• Configuration uses YAML policies defining timeoutDuration, limitRefreshPeriod, and limitForPeriod, allowing different buckets for different endpoints.
• The utility limiter is reusable across components, as demonstrated by the MQTT proxy in pkg/object/mqttproxy/ratelimiter.go.

Frequently Asked Questions

What algorithm does Easegress use for rate limiting?

Easegress uses a token-bucket algorithm with a configurable timeout mechanism. The implementation in pkg/util/ratelimiter/ratelimiter.go tracks tokens per time slice, refreshes the bucket on a defined interval, and returns the necessary wait duration when tokens are exhausted. Unlike a simple fixed-window counter, this approach allows for bursty traffic patterns while maintaining an average rate limit, and the TimeoutDuration parameter prevents requests from waiting indefinitely.

How do I configure different rate limits for different API endpoints?

Define multiple policies in the RateLimiter filter spec and reference them in the urls list. Each URL rule can point to a specific policy via the policyRef field, falling back to defaultPolicyRef if unspecified. For example, you can assign a generous limit to /api/health and a strict limit to /api/login by creating separate policies with different limitForPeriod values and binding each to the appropriate URL pattern. The filter instantiates a separate token bucket for each rule.

What happens when a request exceeds the rate limit?

When the token bucket is empty and the request cannot acquire permission within the TimeoutDuration, the filter returns HTTP 429 Too Many Requests with the response header X-EG-Rate-Limiter: too-many-requests. The request context is tagged with rateLimited and the result rateLimited is returned, preventing the request from reaching the backend. Alternatively, if the policy permits waiting and the calculated delay is within the timeout window, the filter asynchronously waits for the specified duration (or until the request is canceled) before allowing the request to proceed.

Can I use the rate limiter outside of HTTP filters?

Yes. The pkg/util/ratelimiter package provides a generic RateLimiter struct that is independent of HTTP concerns. You can instantiate it directly with a Policy struct and call AcquirePermission() or WaitPermission() in any Go component. The MQTT proxy implementation in pkg/object/mqttproxy/ratelimiter.go demonstrates this reuse pattern for non-HTTP traffic, using the same token-bucket algorithm to throttle MQTT connections.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}