# How to Configure Traffic Throttling at Index, Node, and User Levels in INFINI Gateway

> Master traffic throttling in INFINI Gateway at index, node, and user levels. Learn to configure limits using traffic_control, request_path_limiter, and request_user_limiter filters.

- Repository: [INFINI Labs/gateway](https://github.com/infinilabs/gateway)
- Tags: how-to-guide
- Published: 2026-03-04

---

**Configure traffic throttling in INFINI Gateway by enabling `traffic_control` for node-level limits, or by adding `request_path_limiter` and `request_user_limiter` filters to your flows for index-level and user-level rate limiting.**

INFINI Gateway provides a flexible, multi-layered traffic throttling configuration that allows you to control request rates at different granularities. Whether you need to protect individual Elasticsearch nodes from overload, enforce different quality-of-service tiers per index, or apply usage caps per user account, the gateway implements these limits through a shared **GenericLimiter** framework located in [`proxy/filters/throttle/request_limiter_base.go`](https://github.com/infinilabs/gateway/blob/main/proxy/filters/throttle/request_limiter_base.go).

## Understanding the Rate-Limiting Framework

All throttling mechanisms in INFINI Gateway rely on the **GenericLimiter** implementation in [`proxy/filters/throttle/request_limiter_base.go`](https://github.com/infinilabs/gateway/blob/main/proxy/filters/throttle/request_limiter_base.go). This component provides consistent rate-limiting logic across node, index, and user levels with the following configurable parameters:

- **`max_requests`** / **`burst_requests`** – Control request-count limits per time window
- **`max_bytes`** / **`burst_bytes`** – Enforce bandwidth limits
- **`action`** – Choose between `retry` (default) or `drop` behavior when limits are exceeded
- **`max_retry_times`**, **`retry_delay_in_ms`**, **`status`**, **`message`** – Fine-tune retry behavior and client responses

## Node-Level Traffic Throttling

Node-level traffic throttling controls the overall request volume, bandwidth, and connection count for each individual Elasticsearch node in your cluster. This is the coarsest granularity and serves as your first line of defense against cluster overload.

### Configuration Parameters

Enable node-level throttling by adding the `traffic_control` section to your Elasticsearch connection configuration in [`gateway.yml`](https://github.com/infinilabs/gateway/blob/main/gateway.yml):

```yaml
es:
  endpoints:
    - "http://es-node1:9200"
    - "http://es-node2:9200"
  discovery:
    enabled: true
    refresh:
      enabled: true
      interval: "30s"
  traffic_control:
    enabled: true                 # Enable throttling

    max_qps_per_node: 5000       # Max requests per second per node

    max_bytes_per_node: 10485760 # 10 MiB/s per node

    max_connection_per_node: 200 # Max concurrent TCP connections

    max_wait_time_in_ms: 10000    # Max wait time before rejection

```

### Runtime Enforcement

The gateway enforces node-level limits at runtime through the `metadata.CheckNodeTrafficThrottle` function, which is invoked in two critical locations:

1. **[`proxy/output/elastic/reverseproxy.go`](https://github.com/infinilabs/gateway/blob/main/proxy/output/elastic/reverseproxy.go)** (line 558) – Applies throttling to normal HTTP proxy traffic
2. **[`pipeline/queue_consumer/diskqueue_consumer.go`](https://github.com/infinilabs/gateway/blob/main/pipeline/queue_consumer/diskqueue_consumer.go)** (line 339) – Enforces limits on queued bulk requests

These calls forward the node host identifier to the **GenericLimiter**, ensuring that traffic to each backend node stays within configured bounds.

## Index-Level Traffic Throttling

Index-level throttling allows you to enforce different rate limits for specific Elasticsearch indices, enabling quality-of-service tiers where critical indices receive dedicated bandwidth while others share a common pool.

### Configuration Example

Create a flow that includes the `request_path_limiter` filter to implement index-level throttling:

```yaml
flow:
  - name: index_rate_limit
    filter:
      - request_path_limiter:
          message: "Rate limit exceeded for this index"
          rules:
            - pattern: "/(?P<index_name>medcl)/_search"
              max_qps: 3
              group: index_name
            - pattern: "/(?P<index_name>.*?)/_search"
              max_qps: 100
              group: index_name

```

The filter implementation resides in [`proxy/filters/throttle/request_path_limiter.go`](https://github.com/infinilabs/gateway/blob/main/proxy/filters/throttle/request_path_limiter.go).

### How It Works

The `request_path_limiter` filter processes requests through the following steps:

1. **Path Matching** – The request URL is matched against each rule in order using regular expressions
2. **Token Extraction** – When a rule matches, the captured group (specified by `group`, such as `index_name`) becomes the **token** for the generic limiter
3. **Rate Enforcement** – The **GenericLimiter** checks the QPS (`max_qps`) for that specific index token
4. **Action Execution** – If the limit is exceeded, the filter returns HTTP 429 with the configured `message`, or retries according to the `action` field

## User-Level Traffic Throttling

User-level throttling enables you to enforce request quotas based on HTTP Basic Authentication credentials, allowing you to differentiate between premium users, standard accounts, and potential abusers.

### Configuration Example

Add the `request_user_limiter` filter to your flow to implement per-user rate limiting:

```yaml
flow:
  - name: user_rate_limit
    filter:
      - request_user_limiter:
          user:
            - elastic
            - medcl
          max_requests: 256
          max_bytes: 102400        # 100 KiB/s per user

          action: retry            # or "drop"

          message: "you reached our limit"

```

The filter implementation is located in [`proxy/filters/throttle/request_user_limiter.go`](https://github.com/infinilabs/gateway/blob/main/proxy/filters/throttle/request_user_limiter.go).

### Processing Steps

The `request_user_limiter` filter executes the following logic:

1. **Credential Extraction** – Parses the HTTP Basic-Auth header from the request using `ctx.Request.ParseBasicAuth()`
2. **User Matching** – Checks if the extracted username exists in the configured `user` list
3. **Token Generation** – If matched, creates a limiter token with type `user` and the username as the token value
4. **Limit Enforcement** – Invokes the **GenericLimiter** to enforce `max_requests` and `max_bytes` limits
5. **Response Handling** – Returns the configured `message` with appropriate HTTP status when limits are exceeded, or applies retry logic based on the `action` setting

## Additional Granularities

Beyond the three primary levels, INFINI Gateway provides several specialized throttling filters that use the same **GenericLimiter** infrastructure:

- **`request_host_limiter`** – Throttles by HTTP `Host` header, useful for multi-tenant deployments with virtual hosts
- **`request_client_ip_limiter`** – Limits requests per client IP address to prevent abuse from specific sources
- **`request_api_key_limiter`** – Enforces quotas based on API key headers for third-party integrations
- **`bulk_request_throttle`** – Controls bulk write throughput per index by parsing bulk request payloads

All filters reside in `proxy/filters/throttle/` and share the configuration schema defined in [`request_limiter_base.go`](https://github.com/infinilabs/gateway/blob/main/request_limiter_base.go).

## Summary

- **Node-level throttling** is configured via the `traffic_control` section in your Elasticsearch connection settings and enforced by `metadata.CheckNodeTrafficThrottle` in [`proxy/output/elastic/reverseproxy.go`](https://github.com/infinilabs/gateway/blob/main/proxy/output/elastic/reverseproxy.go) and [`pipeline/queue_consumer/diskqueue_consumer.go`](https://github.com/infinilabs/gateway/blob/main/pipeline/queue_consumer/diskqueue_consumer.go).

- **Index-level throttling** uses the `request_path_limiter` filter to extract index names from URL paths via regular expressions and apply per-index QPS and bandwidth limits.

- **User-level throttling** leverages the `request_user_limiter` filter to parse HTTP Basic-Auth credentials and enforce request quotas per username.

- All granularities rely on the **GenericLimiter** in [`proxy/filters/throttle/request_limiter_base.go`](https://github.com/infinilabs/gateway/blob/main/proxy/filters/throttle/request_limiter_base.go), providing consistent behavior for request counts, byte limits, retry logic, and error responses.

## Frequently Asked Questions

### How do I combine multiple throttling levels in a single flow?

You can chain multiple limiter filters in sequence within your flow definition. For example, place `request_user_limiter` first to enforce per-user caps, followed by `request_path_limiter` to apply index-specific limits. Each filter operates independently using the shared **GenericLimiter**, so a request must pass all configured limits to proceed.

### What happens when a throttling limit is exceeded?

By default, the **GenericLimiter** returns HTTP 429 (Too Many Requests) with your configured `message`. If you set `action: retry`, the gateway will automatically retry the request up to `max_retry_times` with `retry_delay_in_ms` intervals before finally rejecting it. For queued bulk operations in [`diskqueue_consumer.go`](https://github.com/infinilabs/gateway/blob/main/diskqueue_consumer.go), the retry behavior ensures data durability while respecting node-level bandwidth constraints.

### Can I throttle by API key instead of username?

Yes, use the `request_api_key_limiter` filter instead of `request_user_limiter`. This filter extracts tokens from a configurable header (typically `Authorization` or `X-API-Key`) and applies the same **GenericLimiter** logic. This is ideal for third-party API access management where Basic Auth is not used.

### Where is the node-level throttling enforced in the codebase?

Node-level throttling is enforced in two critical locations: [`proxy/output/elastic/reverseproxy.go`](https://github.com/infinilabs/gateway/blob/main/proxy/output/elastic/reverseproxy.go) at line 558 for standard HTTP proxy traffic, and [`pipeline/queue_consumer/diskqueue_consumer.go`](https://github.com/infinilabs/gateway/blob/main/pipeline/queue_consumer/diskqueue_consumer.go) at line 339 for queued bulk ingestion. Both locations invoke `metadata.CheckNodeTrafficThrottle`, which delegates to the **GenericLimiter** to check against the `traffic_control` configuration parameters.