# How Compression Is Implemented in OpenFlux Transports: A Deep Dive into LZ4 Optimization

> Discover how OpenFlux implements transport compression with LZ4 optimization. Learn about the CompressedTransport decorator and its byte-size threshold for efficient data transfer.

- Repository: [p1neappleXpress/OpenFlux](https://github.com/p1neappleXpress/OpenFlux)
- Tags: deep-dive
- Published: 2026-09-14

---

**OpenFlux implements transport compression using a** **`CompressedTransport`** **decorator pattern that wraps any concrete transport, applying LZ4 compression only when payloads exceed 200 bytes and actually reduce in size.**

The `p1neappleXpress/OpenFlux` repository provides a lightweight, transparent compression layer for TCP transport implementations. By wrapping existing transports with a decorator, the system enables on-the-fly payload compression without modifying individual transport logic.

## The CompressedTransport Decorator Pattern

OpenFlux uses the **decorator design pattern** to add compression capabilities to any transport implementation. The wrapper resides in [`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go) and implements the same `Transport` interface defined in [`transport/transport.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/transport.go), allowing seamless integration with existing code.

The decorator intercepts `Send` and `Receive` calls, automatically applying compression logic based on payload characteristics. This approach decouples compression concerns from transport-specific implementations like YandexDocs, OneMe, or Cups.online transports.

## Compression Thresholds and Markers

The compression system uses specific constants to determine when and how to compress data.

### Size-Based Compression Logic

OpenFlux only attempts compression when the payload meets the minimum threshold:

- **`MinCompressSize = 200` bytes** – Packets smaller than this limit are transmitted uncompressed to avoid overhead
- **Size comparison** – Even for packets exceeding 200 bytes, the wrapper compares the compressed output against the original size; if compression does not reduce the payload, it falls back to uncompressed transmission

### Protocol Markers for Transport Negotiation

The system prefixes all packets with marker bytes to allow the receiver to distinguish between compressed and uncompressed data:

- **`CompressionMarker = 0x1F`** – Indicates the following bytes contain LZ4-compressed data
- **`0x00`** – Indicates the remaining bytes are uncompressed raw payload

These single-byte headers enable stateless decompression on the receiving end without requiring out-of-band negotiation.

## Send and Receive Flows

The `CompressedTransport` wrapper intercepts data flows through two primary paths.

### Compression Flow (Send)

When `Send(data)` is called on a wrapped transport, the following logic executes according to the source in [`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go):

1. Check if `len(data) <= MinCompressSize` – if true, prefix with `0x00` and forward unchanged
2. For larger payloads, write the `CompressionMarker` (`0x1F`)
3. Create an LZ4 writer using `github.com/pierrec/lz4/v4`
4. Stream data into the LZ4 writer and close it
5. Compare compressed buffer size against original size
6. If compressed size is **not smaller**, discard and send `0x00` + original data instead

```go
inner := yandex.NewYandexDocsTransport(docURL, cfg) // any Transport implementation
comp := transport.NewCompressedTransport(inner)

if err := comp.Start(); err != nil {
    log.Fatalf("failed to start transport: %v", err)
}
defer comp.Stop()

// Sending data – automatically compressed if large enough
payload := []byte("...large TCP payload...")
if err := comp.Send(payload); err != nil {
    log.Printf("send error: %v", err)
}

```

### Decompression Flow (Receive)

The `Receive` callback in the wrapper inspects incoming packets via the `decompress(data)` function:

- If the first byte is `0x00`, the payload is simply `data[1:]` (uncompressed)
- If the first byte is `0x1F`, the remaining bytes feed into an LZ4 reader to restore the original data
- Any decompression error triggers a safety fallback, passing the raw data through unchanged

```go
// Receiving data – automatically decompressed
comp.Receive(func(pkt []byte) {
    // pkt is the original uncompressed payload
    handlePacket(pkt)
})

```

## Integration Points and Usage Examples

The compressed transport is instantiated at key entry points in the codebase. In [`main.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/main.go) at line 111, the wrapper decorates the transport pipeline:

```go
// main.go (line 111)
trans := transport.NewCompressedTransport(inner)

```

For iOS static library builds, the wrapper appears in [`export_ios.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/export_ios.go) between lines 170-173:

```go
// export_ios.go (lines 170-173)
t = transport.NewCompressedTransport(yandex.NewYandexDocsTransport(docURL, config))

```

These integration points demonstrate that **every transport** automatically benefits from LZ4 compression without requiring changes to the underlying transport implementations.

## Performance Characteristics and Fallback Behavior

The compression layer prioritizes **throughput over compression ratio** by using the LZ4 algorithm, known for extremely fast compression and decompression speeds. The implementation includes intelligent fallbacks:

- **Small packet optimization** – Packets under 200 bytes bypass compression entirely, avoiding CPU overhead for trivial payloads
- **Negative compression handling** – When LZ4 produces output larger than the input (possible with incompressible data), the system transparently falls back to uncompressed transmission
- **Error resilience** – Decompression failures result in raw data passthrough, ensuring connectivity even with corrupted or malformed packets

## Summary

- **Decorator architecture** – The `CompressedTransport` wrapper in [`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go) wraps any concrete transport implementation without modifying underlying code
- **Smart compression triggers** – Only payloads exceeding **200 bytes** are candidates for compression, and only if the compressed result is actually smaller
- **Lightweight protocol** – Single-byte markers (`0x1F` for compressed, `0x00` for raw) enable transparent protocol negotiation
- **LZ4 implementation** – Uses `github.com/pierrec/lz4/v4` for high-speed compression with automatic fallback on errors or incompressible data
- **Universal integration** – Activated in [`main.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/main.go) and [`export_ios.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/export_ios.go), providing compression across all transport types including Yandex, OneMe, and Cups.online

## Frequently Asked Questions

### What compression algorithm does OpenFlux use?

OpenFlux uses **LZ4 compression** via the `github.com/pierrec/lz4/v4` library. LZ4 was chosen for its extremely fast compression and decompression speeds, making it ideal for real-time network transport where latency matters more than maximum compression ratios.

### At what payload size does OpenFlux start compressing data?

OpenFlux only considers compression for payloads larger than **200 bytes**, defined by the `MinCompressSize` constant in [`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go). Packets at or below this threshold are prefixed with `0x00` and transmitted uncompressed to avoid unnecessary CPU overhead.

### How does OpenFlux handle decompression errors?

The `decompress` function implements a safety fallback: if any error occurs during LZ4 decompression, the system passes the raw data through unchanged rather than failing the transport. This ensures that corrupted packets or protocol mismatches do not break the connection, though the application layer receives uncompressed data in these edge cases.

### Can I disable compression for specific transports?

While the base [`transport/transport.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/transport.go) interface does not expose a toggle, you can disable compression simply by **not wrapping** the transport with `NewCompressedTransport()`. Since compression is implemented as an optional decorator rather than a core feature, using the underlying transport directly (e.g., `yandex.NewYandexDocsTransport()` without the wrapper) bypasses compression entirely while maintaining full functionality.