How Bulk Indexing Acceleration Works in INFINI Gateway: Pipeline Deep Dive and Optimization Guide

INFINI Gateway accelerates Elasticsearch bulk indexing by decomposing incoming _bulk requests into routed sub-buffers that are asynchronously reordered, batched, and delivered to target nodes, eliminating network bottlenecks and enabling back-pressure management through configurable throttling and partitioning stages.

Bulk indexing acceleration in INFINI Gateway reimagines the data path between clients and Elasticsearch clusters. Instead of proxying _bulk requests verbatim, the gateway—implemented in the infinilabs/gateway repository—intercepts the NDJSON payload and processes it through a specialized pipeline that optimizes for both throughput and reliability. Understanding the internal flow through filters like bulk_request_throttle, bulk_reshuffle, and bulk_request_resort is essential for tuning high-volume ingestion workloads.

How the Bulk Acceleration Pipeline Works

The pipeline consists of four distinct stages that transform a standard bulk request into an optimized, asynchronously delivered payload. Each stage is implemented as a dedicated filter in the gateway's proxy layer.

Stage	Function	Source File
Throttle	Applies per-index rate limiting based on operation count and payload size	proxy/filters/throttle/bulk_request_throttle.go
Reshuffle	Parses NDJSON payload, computes routing, splits into sub-bulks, and queues by partition	proxy/filters/elastic/bulk_reshuffle.go
Response Processing	Generates synthetic bulk response to acknowledge client immediately	proxy/filters/elastic/bulk_reshuffle.go
Resort	Reorders documents by version or timestamp, batches, and commits to downstream	proxy/filters/elastic/bulk_request_resort.go

Stage 1: Throttle (Early Back-Pressure)

The ElasticsearchBulkRequestThrottle filter acts as the first line of defense against downstream overload. Located in proxy/filters/throttle/bulk_request_throttle.go, this stage counts bulk operations and payload size per index using a GenericLimiter. When configured with per-index limits, it automatically delays or drops excess traffic before it enters the reshuffle queue, preventing memory pressure and Elasticsearch throttling on hot indices.

Stage 2: Reshuffle (Splitting and Routing)

The core transformation happens in proxy/filters/elastic/bulk_reshuffle.go. Here, the filter parses the incoming _bulk NDJSON stream using elastic.WalkBulkRequests from the framework/core/elastic/bulk.go utility. For each document, it optionally fixes missing _id fields via FixNullID and collects metadata through IndexStatsAnalysis and ActionStatsAnalysis.

The filter then determines routing based on the configured Level parameter:

• Cluster: Routes to any available cluster node
• Node: Targets the specific node holding the primary shard
• Index: Groups by index name
• Shard: Targets specific shard IDs

When PartitionSize exceeds 1, the filter computes a deterministic partition ID using xxhash, creating queue keys like async_bulk##node##es-1##node-abc##partition-3. Each sub-bulk is buffered in a bytebufferpool.ByteBuffer (when bytes_buffer_enabled is true) to minimize GC overhead.

Stage 3: Response Processing (Synthetic Acknowledgment)

Immediately after queuing, the reshuffle filter constructs a synthetic bulk response using startPart, itemPart, and endPart templates. This fake response streams back to the client with HTTP 200 status, allowing the client to proceed while the actual indexing happens asynchronously. The filter also injects bulk_index_stats and bulk_action_stats into the request context for downstream metric collection.

Stage 4: Resort (Reordering and Batching)

The final stage runs in proxy/filters/elastic/bulk_request_resort.go within dedicated goroutines (Sorter.run). Workers consume from per-partition output queues, assembling documents into new bulk payloads. They reorder documents using either SortDocumentsByVersion (for optimistic concurrency) or SortDocumentsByTime (for chronological sequencing).

Batches are flushed based on BatchSizeInDocs, BatchSizeInMB, or IdleTimeoutInSeconds thresholds. After successful delivery, the filter atomically commits offsets to guarantee at-least-once delivery while preserving ordering guarantees.

Data Flow Summary

1. Client sends _bulk request → Throttle (optional rate limiting) → Reshuffle (split and queue)
2. Reshuffle returns synthetic 200 OK to client immediately
3. Resort workers consume queues, reorder, batch, and deliver real bulk requests to Elasticsearch nodes

Performance Tuning and Optimization

Optimizing bulk indexing acceleration requires balancing memory usage, network efficiency, and delivery latency. The following configurations control the pipeline behavior.

Throttle Configuration for Hot Index Protection

Configure per-index limits in bulk_request_throttle to protect specific indices from traffic spikes. Use the indices map to define limit (operations per second) and burst (maximum temporary exceedance) values.

bulk_request_throttle:
  indices:
    "logs-*":
      limit: 5000
      burst: 10000

Reshuffle Level and Partitioning Strategies

Set level: node in bulk_reshuffle to route documents directly to nodes holding primary shards, eliminating inter-node forwarding within Elasticsearch. For clusters exceeding 50 nodes, enable partition_size to distribute load across independent queues using xxhash-based partitioning.

bulk_reshuffle:
  level: node
  partition_size: 8

Buffer Pool and Memory Management

Enable bytes_buffer_enabled: true to reuse ByteBuffer instances from framework/lib/bytebufferpool/pool.go instead of allocating new buffers per request. Adjust max_buffer_size and max_buffer_count to match available heap memory—defaults of 1 GB and 10,000 buffers suit moderate workloads, but high-volume scenarios require increased limits.

bulk_reshuffle:
  bytes_buffer_enabled: true
  max_buffer_size: 2000000
  max_buffer_count: 50000

Resort Batch Sizing and Idle Timeouts

Tune batch_size_in_docs and batch_size_in_mb in bulk_request_resort to match Elasticsearch's http.max_content_length (default 100 MB). Larger batches improve throughput but increase latency. Set idle_timeout_in_seconds lower than the default 10s to ensure timely delivery during low-traffic periods.

bulk_request_resort:
  batch_size_in_docs: 8000
  batch_size_in_mb: 20
  idle_timeout_in_seconds: "2s"
  commit_offset:
    interval: "8s"

Complete Production Configuration

The following gateway.yml combines all optimization strategies for a high-throughput, low-latency pipeline:

filters:
  - name: bulk_request_throttle
    config:
      indices:
        "logs-*":
          limit: 4000
          burst: 8000

  - name: bulk_reshuffle
    config:
      elasticsearch: es-cluster
      level: node
      partition_size: 4
      bytes_buffer_enabled: true
      max_buffer_size: 5000000
      fix_null_id: true
      index_stats_analysis: true
      action_stats_analysis: true

  - name: bulk_request_resort
    config:
      batch_size_in_docs: 6000
      batch_size_in_mb: 15
      idle_timeout_in_seconds: "5s"
      sort_by: version
      commit_offset:
        interval: "8s"

Client Integration Example

No client changes are required. Applications continue using standard Elasticsearch bulk APIs while the gateway handles all acceleration transparently.

// Standard HTTP client example - works unchanged with INFINI Gateway
bulkBody := `
{ "index": { "_index": "logs-2026.03", "_id": "123" } }
{ "msg": "entry", "ts": "2026-03-04T12:00:00Z" }
`
resp, err := http.Post("http://gateway:8080/_bulk", 
    "application/x-ndjson", 
    strings.NewReader(bulkBody))
// Returns 200 OK immediately (synthetic response)

Summary

• Throttle applies per-index back-pressure via GenericLimiter in bulk_request_throttle.go to prevent downstream overload.
• Reshuffle splits requests using elastic.WalkBulkRequests, routes by node/shard level, partitions via xxhash, and buffers using pooled ByteBuffer instances.
• Resort asynchronously reorders documents using SortDocumentsByVersion or SortDocumentsByTime, then batches according to size and idle timeout constraints before final delivery.
• Optimization requires tuning partition_size for parallelism, enabling buffer pooling to reduce GC, and configuring batch thresholds to balance latency against throughput.

Frequently Asked Questions

How does INFINI Gateway handle client responses during bulk acceleration?

The gateway immediately returns a synthetic bulk response constructed from startPart, itemPart, and endPart templates after queuing sub-bulks. This allows clients to proceed without waiting for actual Elasticsearch indexing, while background Resort workers handle the real delivery and retry logic.

What is the purpose of the partition size setting in bulk reshuffle?

partition_size determines how many independent queues the gateway creates per routing level. When set above 1, the filter uses xxhash to distribute documents across partitions like async_bulk##node##es-1##node-abc##partition-3, enabling parallel consumption by multiple resort workers and preventing single-queue bottlenecks in large clusters.

How can I prevent out-of-memory errors when processing large bulk payloads?

Enable bytes_buffer_enabled to reuse buffers from bytebufferpool, and configure max_buffer_size and max_buffer_count to cap memory consumption per partition. Additionally, use bulk_request_throttle to rate-limit hot indices and prevent unbounded queue growth during traffic spikes.

Does the gateway preserve document ordering during bulk acceleration?

Yes, when configured appropriately. The Resort filter supports SortDocumentsByVersion for optimistic concurrency control or SortDocumentsByTime for chronological ordering. Documents within each partition are reordered before batching, and atomic offset commits ensure at-least-once delivery with ordering guarantees maintained per partition.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}