# Understanding the Framing Mechanism for Compressed Data in OpenFlux

> OpenFlux uses a simple single-byte prefix framing mechanism for compressed data employing 0x1F for LZ4 and 0x00 for uncompressed payloads. Learn more about its transport/compressor.go implementation.

- Repository: [p1neappleXpress/OpenFlux](https://github.com/p1neappleXpress/OpenFlux)
- Tags: deep-dive
- Published: 2026-09-14

---

**OpenFlux implements a lightweight single-byte prefix framing mechanism in [`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go) that uses marker `0x1F` for LZ4-compressed payloads and `0x00` for uncompressed data.**

The OpenFlux messaging library employs a custom framing protocol to distinguish between compressed and uncompressed network payloads. This deterministic approach, defined in the `transport` package, enables receivers to transparently handle both small messages and large compressed blocks without external metadata headers.

## How the Framing Mechanism Works

OpenFlux uses a **single-byte prefix** at the start of every packet to indicate the compression state of the following bytes.

### The Single-Byte Marker Protocol

The protocol defines two constant markers at the top of [`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go):

- **`0x00`**: Indicates the remainder of the packet contains raw, uncompressed data.
- **`0x1F`**: Indicates the remainder contains an LZ4-compressed block.

During reception, the `decompress` function examines the first byte to determine processing:

```go
if data[0] == 0x00 {
    return data[1:], nil // raw payload
}
r := lz4.NewReader(bytes.NewReader(data[1:]))
return io.ReadAll(r)    // LZ4-decompressed payload

```

### Compression Threshold Logic

The implementation includes intelligent fallback logic to avoid compression overhead on small payloads. The `MinCompressSize` constant (set to **200 bytes**) defines the minimum payload size eligible for compression.

When `compress` processes data larger than 200 bytes, it attempts LZ4 compression. If the resulting compressed size exceeds the original payload size plus the one-byte marker overhead, OpenFlux automatically falls back to the uncompressed format using marker `0x00`.

## Implementation in transport/compressor.go

The core constants and logic reside in [`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go):

```go
const (
    MinCompressSize   = 200
    CompressionMarker = 0x1F // <-- indicates LZ4-compressed payload
)

```

The `compress` function creates an LZ4 writer, writes the payload, and prefixes the result with `CompressionMarker`. The `decompress` function strips the marker and returns either the raw bytes or the inflated LZ4 stream.

## Working with CompressedTransport

OpenFlux wraps existing transports with compression support through `NewCompressedTransport`.

### Sending Compressed Data

Automatic compression occurs when wrapping any transport implementation:

```go
import "github.com/p1neappleXpress/OpenFlux/main/transport"

func sendExample(t transport.Transport, payload []byte) error {
    // Wrap the inner transport with compression support
    comp := transport.NewCompressedTransport(t)

    // The transport will add the appropriate marker automatically
    return comp.Send(payload)
}

```

### Receiving and Decompressing

The `CompressedTransport.Receive` method handles framing transparently:

```go
func receiveExample(t transport.Transport) {
    comp := transport.NewCompressedTransport(t)

    comp.Receive(func(msg []byte) {
        // `msg` is already decompressed if it was sent with the 0x1F marker
        fmt.Printf("Received %d bytes: %s\n", len(msg), string(msg))
    })
}

```

## Manual Frame Inspection

For debugging or protocol analysis, you can inspect the framing marker directly:

```go
func inspectMarker(data []byte) string {
    if len(data) == 0 {
        return "empty"
    }
    switch data[0] {
    case 0x00:
        return "uncompressed"
    case 0x1F:
        return "lz4-compressed"
    default:
        return "unknown"
    }
}

```

## Summary

- OpenFlux uses a **single-byte prefix framing mechanism** where `0x1F` signals LZ4 compression and `0x00` signals raw data.
- The logic is implemented in **[`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go)** with `CompressionMarker` and `MinCompressSize` constants.
- Payloads smaller than **200 bytes** are transmitted uncompressed to avoid overhead.
- The system automatically falls back to uncompressed format when compression would not reduce total size.
- **LZ4** is the compression algorithm used for all compressed payloads.
- `NewCompressedTransport` wraps any existing transport to add transparent compression support.

## Frequently Asked Questions

### What compression algorithm does OpenFlux use for framed data?

OpenFlux uses **LZ4** compression for all payloads marked with `0x1F`. The implementation creates an `lz4.Writer` during compression and an `lz4.NewReader` during decompression, providing high-speed compression with minimal CPU overhead according to the source code in [`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go).

### How does OpenFlux handle small messages that don't benefit from compression?

Messages smaller than `MinCompressSize` (200 bytes) are automatically sent uncompressed with the `0x00` marker. Additionally, if compression would result in a larger payload than the original data plus marker overhead, the system falls back to the uncompressed format to optimize bandwidth.

### Can I use the compressed transport with encrypted connections?

Yes. OpenFlux supports transport layering, allowing `CompressedTransport` to wrap encryption transports or vice versa. The [`transport/encrypted.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/encrypted.go) file demonstrates how compressed transports compose with additional transport layers for end-to-end encryption while maintaining the single-byte framing protocol.

### Where is the framing logic implemented in the OpenFlux source code?

The framing mechanism is implemented in **[`transport/compressor.go`](https://github.com/p1neappleXpress/OpenFlux/blob/main/transport/compressor.go)**, which defines the marker constants, compression threshold logic, and the `compress` and `decompress` functions that handle the single-byte prefix protocol.