# How FastProxy Handles Data Compression: Configuration, Algorithms, and Middleware Pipeline

> Discover how FastProxy handles data compression with Snappy and Gzip. Learn about configuration and middleware integration in request/response pipelines for optimized performance.

- Repository: [Kingson4Wu/fast_proxy](https://github.com/kingson4wu/fast_proxy)
- Tags: deep-dive
- Published: 2026-03-05

---

**FastProxy implements a pluggable compression layer that supports Snappy and Gzip algorithms, configurable per service via `ServiceConfig`, and integrated into request/response pipelines through middleware in the `outproxy` and `inproxy` packages.**

FastProxy is an open-source proxy framework designed for high-performance service mesh scenarios. According to the kingson4wu/fast_proxy source code, the project implements a modular compression system that allows operators to enable and configure compression algorithms on a per-service basis without modifying core routing logic. This design separates compression concerns into configuration, algorithm registration, and middleware execution phases.

## Configuration-Driven Compression Settings

Compression in FastProxy is toggled through service-specific configuration rather than global flags. Each service definition contains explicit controls for enabling compression and selecting the algorithm.

### The ServiceConfig Structure

In [`common/config/config.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/config/config.go), the `ServiceConfig` struct defines two critical fields:

- **`CompressEnable`** (`bool`) – Toggles compression on or off for the service
- **`CompressAlgorithm`** (`int32`) – Selects the algorithm where `0` represents Snappy and `1` represents Gzip

This configuration lives in YAML files typically located at [`outproxy/outconfig/config.yaml`](https://github.com/kingson4wu/fast_proxy/blob/main/outproxy/outconfig/config.yaml). The following snippet demonstrates enabling Gzip compression for a specific service:

```yaml
services:
  my-service:
    compress:
      enable: true
      algorithm: 1   # 0 = Snappy, 1 = Gzip

```

## The Compression Algorithm Registry

FastProxy uses a registry pattern to decouple algorithm selection from implementation details. This allows the middleware to treat all compression algorithms uniformly through a common interface.

### The Compress Interface

The contract for compression implementations is defined in [`common/compress/interfaces/interfaces.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/interfaces/interfaces.go). This interface requires `Encode` and `Decode` methods that accept byte slices and return compressed or decompressed data along with error handling. Both Snappy and Gzip implementations satisfy this interface, enabling polymorphic usage throughout the codebase.

### Algorithm Registration and Initialization

At startup, [`common/compress/compress.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/compress.go) executes an `init()` function that builds a mapping between the `Algorithm` enum values and their concrete implementations. The registry maps:

- **Algorithm 0** to the Snappy implementation in [`common/compress/snappy/snappy.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/snappy/snappy.go)
- **Algorithm 1** to the Gzip implementation in [`common/compress/gzip/gzip.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/gzip/gzip.go)

This registration pattern allows the middleware to invoke `compress.Encode()` and `compress.Decode()` without knowing which specific algorithm is configured for the current service.

## Middleware Integration

The compression logic is woven into the request/response lifecycle through dedicated packing layers in both the outbound and inbound proxy components.

### Outgoing Request Compression (outproxy)

When FastProxy processes outgoing traffic from server to client, the [`outproxy/internal/pack/pack.go`](https://github.com/kingson4wu/fast_proxy/blob/main/outproxy/internal/pack/pack.go) file handles the compression step. After encryption but before protobuf encoding, the code checks `serviceConfig.CompressEnable`. If enabled, it calls `compress.Encode()` with the raw payload and the configured algorithm identifier from `serviceConfig.CompressAlgorithm`.

The typical flow follows: **Encrypt → Compress → ProtobufEncode**

```go
func EncodeRequest(body []byte, svcName string) ([]byte, error) {
    // Fast Proxy's Encode function automatically runs Encrypt → Compress → ProtobufEncode
    return pack.Encode(body, svcName) // pack from outproxy/internal/pack
}

```

### Incoming Response Decompression (inproxy)

For incoming traffic from client to server, [`inproxy/internal/pack/pack.go`](https://github.com/kingson4wu/fast_proxy/blob/main/inproxy/internal/pack/pack.go) manages decompression. After unmarshalling the protobuf payload, the code checks the `reData.Compress` boolean flag. When true, it invokes `compress.Decode()` to restore the original payload before further processing.

The inbound flow reverses the operations: **Protobuf → Decompress → Decrypt**

```go
func DecodeResponse(respData []byte) ([]byte, error) {
    // Fast Proxy's Decode runs Protobuf → Decompress → Decrypt
    return pack.Decode(respData) // pack from inproxy/internal/pack
}

```

## Supported Compression Algorithms

FastProxy ships with two production-ready compression implementations, each optimized for different latency and compression ratio requirements.

### Snappy Implementation

Located in [`common/compress/snappy/snappy.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/snappy/snappy.go), the Snappy implementation prioritizes speed over compression ratio. It wraps the `github.com/golang/snappy` library:

- **`Encode`** calls `snappy.Encode()` for high-speed compression
- **`Decode`** calls `snappy.Decode()` for rapid decompression

This algorithm suits scenarios where CPU overhead must remain minimal and network bandwidth is less constrained.

### Gzip Implementation

The Gzip implementation in [`common/compress/gzip/gzip.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/gzip/gzip.go) provides higher compression ratios at the cost of increased CPU usage:

- **`Encode`** creates a `gzip.NewWriter`, writes the payload, and flushes the compressed bytes
- **`Decode`** instantiates a `gzip.NewReader` to stream and decompress the data

This option benefits high-latency or bandwidth-constrained networks where payload size reduction outweighs processing overhead.

## Practical Implementation Examples

Beyond configuration, developers can interact with the compression layer programmatically:

**Programmatic encoding using the middleware:**

```go
import (
    "github.com/Kingson4Wu/fast_proxy/common/compress"
    "github.com/Kingson4Wu/fast_proxy/common/config"
)

// Create a ServiceConfig that enables Gzip
sc := &config.ServiceConfig{
    CompressEnable:    true,
    CompressAlgorithm: int32(compress.Gzip),
}

// Raw payload
payload := []byte(`{"msg":"hello world"}`)

// Apply compression (middleware style)
compressed, err := compress.Encode(payload, sc.CompressAlgorithm)
if err != nil {
    // handle error
}

```

**Decoding a compressed response:**

```go
import (
    "github.com/Kingson4Wu/fast_proxy/common/compress"
)

// Assume we received `data` and we know the algorithm is Snappy
original, err := compress.Decode(data, int32(compress.Snappy))
if err != nil {
    // handle error
}

```

## Summary

- **Configuration-driven**: Compression is controlled per service via `ServiceConfig.CompressEnable` and `CompressAlgorithm` fields in [`common/config/config.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/config/config.go)
- **Pluggable architecture**: The `init()` function in [`common/compress/compress.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/compress.go) registers algorithms to enum values, supporting Snappy (0) and Gzip (1)
- **Interface-based design**: Both algorithms implement the `Compress` interface from [`common/compress/interfaces/interfaces.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/interfaces/interfaces.go), allowing uniform middleware treatment
- **Pipeline integration**: [`outproxy/internal/pack/pack.go`](https://github.com/kingson4wu/fast_proxy/blob/main/outproxy/internal/pack/pack.go) compresses outbound data, while [`inproxy/internal/pack/pack.go`](https://github.com/kingson4wu/fast_proxy/blob/main/inproxy/internal/pack/pack.go) decompresses inbound responses
- **Algorithm characteristics**: Snappy offers low-latency compression via [`common/compress/snappy/snappy.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/snappy/snappy.go), while Gzip provides higher compression ratios via [`common/compress/gzip/gzip.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/gzip/gzip.go)

## Frequently Asked Questions

### How do I enable compression for a specific service in FastProxy?

Set `compress.enable` to `true` and specify `compress.algorithm` (0 for Snappy, 1 for Gzip) in the service's YAML configuration block. FastProxy reads these values into the `ServiceConfig` struct at startup and applies them automatically during request processing.

### Where does compression occur in the FastProxy request lifecycle?

For outgoing requests, compression happens in [`outproxy/internal/pack/pack.go`](https://github.com/kingson4wu/fast_proxy/blob/main/outproxy/internal/pack/pack.go) after encryption but before protobuf encoding. For incoming responses, decompression occurs in [`inproxy/internal/pack/pack.go`](https://github.com/kingson4wu/fast_proxy/blob/main/inproxy/internal/pack/pack.go) after protobuf unmarshalling but before decryption. This ordering ensures compressed data travels over the wire while maintaining end-to-end payload integrity.

### Can I add custom compression algorithms to FastProxy?

Yes. Implement the `Compress` interface defined in [`common/compress/interfaces/interfaces.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/interfaces/interfaces.go), then register your implementation in the algorithm map inside [`common/compress/compress.go`](https://github.com/kingson4wu/fast_proxy/blob/main/common/compress/compress.go). Add a new enum constant for your algorithm and ensure the `init()` function maps that constant to your implementation, following the pattern used for Snappy and Gzip.

### What is the performance difference between Snappy and Gzip in FastProxy?

According to the source implementations, Snappy prioritizes speed and low CPU overhead using the `github.com/golang/snappy` library, making it ideal for low-latency requirements. Gzip, implemented via standard library `gzip.NewWriter` and `gzip.NewReader`, achieves higher compression ratios but consumes more CPU cycles, making it suitable for bandwidth-constrained scenarios where payload size reduction is critical.