# How FastProxy Implements Traffic Throttling: Token Bucket Rate Limiting in Go

> Discover how FastProxy uses Go's token bucket rate limiting to control traffic. Learn about its sync map implementation and HTTP 429 error handling for efficient request management.

- Repository: [Kingson4Wu/fast_proxy](https://github.com/kingson4wu/fast_proxy)
- Tags: how-to-guide
- Published: 2026-03-05

---

**FastProxy implements traffic throttling using a per-service token bucket rate limiter from `golang.org/x/time/rate`, storing limiters in a `sync.Map` and rejecting excess requests with HTTP 429 status codes.**

FastProxy is an open-source Go proxy that protects upstream services through configurable traffic throttling. This article examines how the `inproxy` module implements per-service and per-endpoint rate limiting using token bucket algorithms, based on the actual source code in the `kingson4wu/fast_proxy` repository.

## Token Bucket Architecture

FastProxy employs a **token bucket rate limiter** to control inbound traffic. This algorithm allows bursts up to a configured capacity while maintaining a steady long-term rate, making it ideal for API gateway scenarios.

### Limiter Storage with sync.Map

The core throttling logic resides in [`inproxy/internal/limiter/limitManager.go`](https://github.com/kingson4wu/fast_proxy/blob/main/inproxy/internal/limiter/limitManager.go). A global `sync.Map` named `limitMap` stores `*rate.Limiter` instances, keyed by a composite string `serviceName_uri`.

This design provides lock-free reads for the hot path, minimizing latency for requests that fall within rate limits.

### Lazy Initialization and Per-Key Mutexes

When a request arrives, the `IsLimit` function checks `limitMap` for an existing limiter:

- If found, it immediately calls `Allow()` on the limiter. A `false` return triggers throttling.
- If not found, the function acquires a per-key mutex from a `keyLocks` map to prevent duplicate limiter creation.

The initialization sequence reads the configured QPS from `inconfig.Get().ServiceQps(serviceName, uri)`, then creates a new limiter:

```go
rate.NewLimiter(rate.Limit(qps), qps)

```

The bucket capacity equals the QPS value, allowing temporary bursts while enforcing the average rate.

## Configuration and QPS Settings

Rate limits are defined in the in-proxy YAML configuration. Administrators specify QPS values per service and per URI pattern:

```yaml

# examples/inproxy/config.yaml

services:
  example-service:
    "/api/v1/resource": 100   # 100 requests per second for this endpoint

    "/health": 500            # Higher limit for health checks

```

The `ServiceQps` method in [`inproxy/inconfig/config.go`](https://github.com/kingson4wu/fast_proxy/blob/main/inproxy/inconfig/config.go) retrieves these values at runtime, enabling dynamic throttling without code changes.

## Request Handling Integration

Throttling is enforced in [`inproxy/internal/proxy/httpclientProxy.go`](https://github.com/kingson4wu/fast_proxy/blob/main/inproxy/internal/proxy/httpclientProxy.go) before forwarding requests upstream. The proxy checks the limiter status:

```go
// inproxy/internal/proxy/httpclientProxy.go (excerpt)
if limiter.IsLimit(clientServiceName, requestPath) {
    // request exceeds the allowed rate → reject with 429 Too Many Requests
    w.WriteHeader(http.StatusTooManyRequests)
    return
}

```

Rejected requests receive an **HTTP 429 Too Many Requests** status code, signaling clients to implement backoff strategies.

## Concurrency Safety Mechanisms

FastProxy's throttling implementation balances performance and correctness through two synchronization primitives:

- **`sync.Map`** provides lock-free reads for the common case where a limiter already exists, reducing contention on high-traffic endpoints.
- **`keyLocks` map** stores `sync.Mutex` instances for each unique service/URI key. This prevents race conditions during lazy initialization when multiple concurrent requests arrive for a new endpoint.

## Practical Implementation Examples

### Custom Handler Integration

You can leverage the limiter in custom handlers:

```go
package myhandler

import (
    "net/http"
    "github.com/Kingson4Wu/fast_proxy/inproxy/internal/limiter"
)

func MyHandler(w http.ResponseWriter, r *http.Request) {
    svc := "example-service"
    uri := r.URL.Path

    if limiter.IsLimit(svc, uri) {
        http.Error(w, "Too Many Requests", http.StatusTooManyRequests)
        return
    }

    // normal processing …
    w.Write([]byte("OK"))
}

```

### Manual Limiter Creation

For testing or specialized use cases, create limiters directly:

```go
import (
    "golang.org/x/time/rate"
    "time"
)

func NewCustomLimiter(qps int) *rate.Limiter {
    // bucket size == burst capacity == qps
    return rate.NewLimiter(rate.Limit(qps), qps)
}

```

## Summary

- FastProxy uses a **token bucket algorithm** via `golang.org/x/time/rate` to enforce per-service, per-endpoint rate limits.
- Limiters are stored in a `sync.Map` keyed by `serviceName_uri` and initialized lazily with per-key mutexes to prevent race conditions.
- Configuration occurs through YAML files specifying QPS values, read via `inconfig.Get().ServiceQps()`.
- Exceeded requests receive **HTTP 429** responses before reaching upstream services.
- The implementation prioritizes performance through lock-free reads and targeted synchronization only during limiter creation.

## Frequently Asked Questions

### What algorithm does FastProxy use for traffic throttling?

FastProxy implements the **token bucket algorithm** using the standard Go package `golang.org/x/time/rate`. This allows configurable QPS limits with burst capacity equal to the QPS value, accommodating temporary traffic spikes while maintaining steady-state rate control.

### How does FastProxy handle concurrent requests to the same endpoint?

FastProxy uses a two-tier concurrency strategy. A global `sync.Map` provides lock-free reads for existing limiters, ensuring minimal overhead for high-traffic endpoints. For new endpoints, a per-key `sync.Mutex` (stored in a separate `keyLocks` map) serializes limiter creation, preventing duplicate limiters for the same `serviceName_uri` key.

### What HTTP status code does FastProxy return when throttling requests?

When a request exceeds the configured rate limit, FastProxy returns **HTTP 429 Too Many Requests**. This occurs in [`inproxy/internal/proxy/httpclientProxy.go`](https://github.com/kingson4wu/fast_proxy/blob/main/inproxy/internal/proxy/httpclientProxy.go) before the request reaches upstream services, allowing clients to detect throttling and implement appropriate backoff strategies.

### Can I configure different rate limits for different API endpoints?

Yes. FastProxy supports granular rate limiting through YAML configuration. You define QPS values per service and per URI pattern in the configuration file, such as `"/api/v1/resource": 100` for 100 requests per second on that specific endpoint, while allowing different limits for other paths like `"/health": 500`.