How FastProxy Handles Data Compression: Configuration, Algorithms, and Middleware Pipeline

FastProxy implements a pluggable compression layer that supports Snappy and Gzip algorithms, configurable per service via ServiceConfig, and integrated into request/response pipelines through middleware in the outproxy and inproxy packages.

FastProxy is an open-source proxy framework designed for high-performance service mesh scenarios. According to the kingson4wu/fast_proxy source code, the project implements a modular compression system that allows operators to enable and configure compression algorithms on a per-service basis without modifying core routing logic. This design separates compression concerns into configuration, algorithm registration, and middleware execution phases.

Configuration-Driven Compression Settings

Compression in FastProxy is toggled through service-specific configuration rather than global flags. Each service definition contains explicit controls for enabling compression and selecting the algorithm.

The ServiceConfig Structure

In common/config/config.go, the ServiceConfig struct defines two critical fields:

• CompressEnable (bool) – Toggles compression on or off for the service
• CompressAlgorithm (int32) – Selects the algorithm where 0 represents Snappy and 1 represents Gzip

This configuration lives in YAML files typically located at outproxy/outconfig/config.yaml. The following snippet demonstrates enabling Gzip compression for a specific service:

services:
  my-service:
    compress:
      enable: true
      algorithm: 1   # 0 = Snappy, 1 = Gzip

The Compression Algorithm Registry

FastProxy uses a registry pattern to decouple algorithm selection from implementation details. This allows the middleware to treat all compression algorithms uniformly through a common interface.

The Compress Interface

The contract for compression implementations is defined in common/compress/interfaces/interfaces.go. This interface requires Encode and Decode methods that accept byte slices and return compressed or decompressed data along with error handling. Both Snappy and Gzip implementations satisfy this interface, enabling polymorphic usage throughout the codebase.

Algorithm Registration and Initialization

At startup, common/compress/compress.go executes an init() function that builds a mapping between the Algorithm enum values and their concrete implementations. The registry maps:

• Algorithm 0 to the Snappy implementation in common/compress/snappy/snappy.go
• Algorithm 1 to the Gzip implementation in common/compress/gzip/gzip.go

This registration pattern allows the middleware to invoke compress.Encode() and compress.Decode() without knowing which specific algorithm is configured for the current service.

Middleware Integration

The compression logic is woven into the request/response lifecycle through dedicated packing layers in both the outbound and inbound proxy components.

Outgoing Request Compression (outproxy)

When FastProxy processes outgoing traffic from server to client, the outproxy/internal/pack/pack.go file handles the compression step. After encryption but before protobuf encoding, the code checks serviceConfig.CompressEnable. If enabled, it calls compress.Encode() with the raw payload and the configured algorithm identifier from serviceConfig.CompressAlgorithm.

The typical flow follows: Encrypt → Compress → ProtobufEncode

func EncodeRequest(body []byte, svcName string) ([]byte, error) {
    // Fast Proxy's Encode function automatically runs Encrypt → Compress → ProtobufEncode
    return pack.Encode(body, svcName) // pack from outproxy/internal/pack
}

Incoming Response Decompression (inproxy)

For incoming traffic from client to server, inproxy/internal/pack/pack.go manages decompression. After unmarshalling the protobuf payload, the code checks the reData.Compress boolean flag. When true, it invokes compress.Decode() to restore the original payload before further processing.

The inbound flow reverses the operations: Protobuf → Decompress → Decrypt

func DecodeResponse(respData []byte) ([]byte, error) {
    // Fast Proxy's Decode runs Protobuf → Decompress → Decrypt
    return pack.Decode(respData) // pack from inproxy/internal/pack
}

Supported Compression Algorithms

FastProxy ships with two production-ready compression implementations, each optimized for different latency and compression ratio requirements.

Snappy Implementation

Located in common/compress/snappy/snappy.go, the Snappy implementation prioritizes speed over compression ratio. It wraps the github.com/golang/snappy library:

• Encode calls snappy.Encode() for high-speed compression
• Decode calls snappy.Decode() for rapid decompression

This algorithm suits scenarios where CPU overhead must remain minimal and network bandwidth is less constrained.

Gzip Implementation

The Gzip implementation in common/compress/gzip/gzip.go provides higher compression ratios at the cost of increased CPU usage:

• Encode creates a gzip.NewWriter, writes the payload, and flushes the compressed bytes
• Decode instantiates a gzip.NewReader to stream and decompress the data

This option benefits high-latency or bandwidth-constrained networks where payload size reduction outweighs processing overhead.

Practical Implementation Examples

Beyond configuration, developers can interact with the compression layer programmatically:

Programmatic encoding using the middleware:

import (
    "github.com/Kingson4Wu/fast_proxy/common/compress"
    "github.com/Kingson4Wu/fast_proxy/common/config"
)

// Create a ServiceConfig that enables Gzip
sc := &config.ServiceConfig{
    CompressEnable:    true,
    CompressAlgorithm: int32(compress.Gzip),
}

// Raw payload
payload := []byte(`{"msg":"hello world"}`)

// Apply compression (middleware style)
compressed, err := compress.Encode(payload, sc.CompressAlgorithm)
if err != nil {
    // handle error
}

Decoding a compressed response:

import (
    "github.com/Kingson4Wu/fast_proxy/common/compress"
)

// Assume we received `data` and we know the algorithm is Snappy
original, err := compress.Decode(data, int32(compress.Snappy))
if err != nil {
    // handle error
}

Summary

• Configuration-driven: Compression is controlled per service via ServiceConfig.CompressEnable and CompressAlgorithm fields in common/config/config.go
• Pluggable architecture: The init() function in common/compress/compress.go registers algorithms to enum values, supporting Snappy (0) and Gzip (1)
• Interface-based design: Both algorithms implement the Compress interface from common/compress/interfaces/interfaces.go, allowing uniform middleware treatment
• Pipeline integration: outproxy/internal/pack/pack.go compresses outbound data, while inproxy/internal/pack/pack.go decompresses inbound responses
• Algorithm characteristics: Snappy offers low-latency compression via common/compress/snappy/snappy.go, while Gzip provides higher compression ratios via common/compress/gzip/gzip.go

Frequently Asked Questions

How do I enable compression for a specific service in FastProxy?

Set compress.enable to true and specify compress.algorithm (0 for Snappy, 1 for Gzip) in the service's YAML configuration block. FastProxy reads these values into the ServiceConfig struct at startup and applies them automatically during request processing.

Where does compression occur in the FastProxy request lifecycle?

For outgoing requests, compression happens in outproxy/internal/pack/pack.go after encryption but before protobuf encoding. For incoming responses, decompression occurs in inproxy/internal/pack/pack.go after protobuf unmarshalling but before decryption. This ordering ensures compressed data travels over the wire while maintaining end-to-end payload integrity.

Can I add custom compression algorithms to FastProxy?

Yes. Implement the Compress interface defined in common/compress/interfaces/interfaces.go, then register your implementation in the algorithm map inside common/compress/compress.go. Add a new enum constant for your algorithm and ensure the init() function maps that constant to your implementation, following the pattern used for Snappy and Gzip.

What is the performance difference between Snappy and Gzip in FastProxy?

According to the source implementations, Snappy prioritizes speed and low CPU overhead using the github.com/golang/snappy library, making it ideal for low-latency requirements. Gzip, implemented via standard library gzip.NewWriter and gzip.NewReader, achieves higher compression ratios but consumes more CPU cycles, making it suitable for bandwidth-constrained scenarios where payload size reduction is critical.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}