How the Ratelimit Middleware in Kratos Enforces Request Throttling

The ratelimit middleware in Kratos wraps server handlers and enforces throttling by delegating admission decisions to a pluggable limiter interface, returning HTTP 429 when the limiter denies requests and notifying it of completion via a DoneFunc callback.

The go-kratos/kratos framework provides a robust middleware/ratelimit package that intercepts incoming requests to prevent system overload. This middleware acts as a gatekeeper, using configurable algorithms to determine whether traffic should proceed or be rejected with standard rate-limiting semantics.

How Rate Limiting Works in Kratos

The ratelimit middleware operates as a standard Kratos middleware that wraps your service handlers. It delegates the actual admission logic to an implementation of the ratelimit.Limiter interface, allowing you to swap algorithms without changing your handler code.

The Middleware Architecture

At its core, the middleware maintains an options struct that holds the limiter instance. According to the source in middleware/ratelimit/ratelimit.go, the default configuration instantiates a BBR (Bottleneck Bandwidth-Delay-Product) limiter:

options := &options{
    limiter: bbr.NewLimiter(),   // default from github.com/go-kratos/aegis/ratelimit/bbr
}

This default leverages the BBR algorithm to detect system bottlenecks by monitoring request latency and in-flight requests, automatically adjusting admission rates based on observed throughput.

The Limiter Interface

The middleware expects any limiter to satisfy this interface contract:

type Limiter interface {
    Allow() (DoneFunc, error)
}

The Allow() method determines admission. When successful, it returns a DoneFunc callback that the middleware must invoke after request completion, passing ratelimit.DoneInfo containing error details from the handler.

Configuring the Ratelimit Middleware

You can deploy the ratelimit middleware with zero configuration for sensible defaults, or inject custom limiters for specific traffic-shaping requirements.

Default BBR Limiter Setup

For most services, importing and applying the middleware provides immediate protection. In middleware/ratelimit/ratelimit.go, the Server() function creates a middleware instance with the BBR limiter:

import (
    "github.com/go-kratos/kratos/v2/middleware/ratelimit"
)

// Apply to HTTP server
httpSrv := http.NewServer(
    http.Address(":8080"),
    http.Middleware(
        ratelimit.Server(), // Uses bbr.NewLimiter() internally
    ),
)

The BBR algorithm continuously samples windows of traffic to calculate the maximum sustainable concurrency, rejecting requests when the system approaches capacity.

Custom Limiter Injection

To override the default, use the WithLimiter option defined in middleware/ratelimit/ratelimit.go:

func WithLimiter(limiter ratelimit.Limiter) Option {
    return func(o *options) { o.limiter = limiter }
}

This functional option replaces the default BBR instance with your implementation. For example, you might implement a token bucket or integrate with an external rate-limiting service like Redis or Polaris.

Request Lifecycle and Admission Control

Understanding the exact flow helps troubleshoot throttling behavior and optimize performance. The middleware executes a strict admission protocol for every request.

The Allow Check

When a request arrives, the middleware immediately queries the limiter. In middleware/ratelimit/ratelimit.go lines 41-45, the logic is:

done, e := options.limiter.Allow()
if e != nil {
    return nil, ErrLimitExceed
}

If Allow() returns an error, the middleware aborts processing and returns ErrLimitExceed without invoking your handler. This prevents resource consumption by rejected requests.

The error constant is defined as:

var ErrLimitExceed = errors.New(429, "RATELIMIT", "service unavailable due to rate limit exceeded")

This generates an HTTP 429 (Too Many Requests) response code, which clients should handle with exponential backoff.

Post-Processing with DoneFunc

When Allow() succeeds, it returns a done callback that the middleware must execute after handler completion. This occurs in lines 47-48 of middleware/ratelimit/ratelimit.go:

reply, err = handler(ctx, req)
done(ratelimit.DoneInfo{Err: err})

This callback allows the limiter to update internal metrics—such as recording latency, tracking successes versus failures, or replenishing token buckets. The BBR limiter uses this data to adjust its sliding windows and calculate new concurrency limits.

Error Handling and HTTP Status Codes

The ratelimit middleware standardizes error responses across transport protocols. Whether using HTTP or gRPC, the ErrLimitExceed error maps to appropriate status codes:

• HTTP: Returns 429 Too Many Requests
• gRPC: Maps to status code 8 (Resource Exhausted)

This standardization ensures API consumers receive consistent signals to implement retry logic and circuit breakers.

Integration Examples

Basic HTTP Server Setup

Deploying rate limiting requires minimal boilerplate. Here's a complete example using the default BBR limiter:

package main

import (
    "github.com/go-kratos/kratos/v2"
    "github.com/go-kratos/kratos/v2/transport/http"
    "github.com/go-kratos/kratos/v2/middleware/ratelimit"
)

func main() {
    srv := kratos.New(
        kratos.Server(
            http.NewServer(
                http.Address(":8080"),
                http.Middleware(
                    ratelimit.Server(), // Default BBR protection
                ),
            ),
        ),
    )
    if err := srv.Run(); err != nil {
        panic(err)
    }
}

Custom Token Bucket Implementation

For custom logic, implement the Limiter interface and inject it via WithLimiter():

type tokenBucket struct {
    tokens int64
    mu     sync.Mutex
}

func (t *tokenBucket) Allow() (ratelimit.DoneFunc, error) {
    t.mu.Lock()
    defer t.mu.Unlock()
    
    if t.tokens <= 0 {
        return nil, ratelimit.ErrLimitExceed
    }
    t.tokens--
    
    return func(_ ratelimit.DoneInfo) {
        t.mu.Lock()
        t.tokens++ // Replenish on completion
        t.mu.Unlock()
    }, nil
}

// Usage
ratelimit.Server(ratelimit.WithLimiter(&tokenBucket{tokens: 100}))

External Integration Pattern

The contrib/polaris/ratelimit.go file demonstrates integrating with Tencent Polaris, a distributed rate-limiting service. This pattern shows how external limiters can implement the same interface and slot into the middleware without code changes.

Summary

• The ratelimit middleware delegates admission decisions to a ratelimit.Limiter interface implementation.
• BBR serves as the default algorithm, auto-tuning based on system latency and throughput measurements.
• Requests are rejected with HTTP 429 when Allow() returns an error, preventing resource consumption.
• The DoneFunc callback ensures limiters receive post-execution metrics to update state (success/failure counts, latency samples).
• Custom limiters integrate via the WithLimiter() functional option, supporting everything from token buckets to external services like Polaris.

Frequently Asked Questions

What algorithm does Kratos ratelimit use by default?

By default, the ratelimit middleware uses the BBR (Bottleneck Bandwidth-Delay-Product) algorithm from the github.com/go-kratos/aegis package. This algorithm detects system bottlenecks by monitoring request latency and in-flight request counts, automatically adjusting the admission rate to maximize throughput without causing overload.

How do I customize the rate limiter in Kratos?

You can provide a custom implementation by using the ratelimit.WithLimiter() option when creating the middleware. Your struct must implement the ratelimit.Limiter interface with an Allow() (DoneFunc, error) method. Pass your instance to ratelimit.Server(ratelimit.WithLimiter(&myLimiter{})) to override the default BBR behavior.

What HTTP status code does Kratos return when rate limiting?

The middleware returns HTTP 429 Too Many Requests when a request exceeds the configured rate limit. This is encapsulated in the ErrLimitExceed error variable defined in middleware/ratelimit/ratelimit.go, which carries the code 429 and the reason phrase "RATELIMIT".

Can I use external rate limiters like Polaris with Kratos?

Yes. The contrib/polaris/ratelimit.go file provides a reference implementation showing how to integrate external rate-limiting services. You implement the ratelimit.Limiter interface to wrap calls to your external service (Redis, Polaris, Sentinel, etc.), then inject it via WithLimiter() to maintain compatibility with the Kratos middleware chain.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}