How BitChat's Binary Protocol Optimizes Bluetooth LE Bandwidth: A Technical Deep Dive

BitChat minimizes Bluetooth Low Energy airtime and battery consumption by encoding all messages into a compact binary wire format featuring fixed-size headers, bit-packed flags, optional TLV sections, and z-standard compression, specifically engineered to fit within BLE's constrained ATT payload limits.

Bluetooth Low Energy imposes strict bandwidth constraints, typically limiting ATT payloads to 20 bytes per packet. BitChat, an open-source messaging protocol available in the permissionlesstech/bitchat repository, addresses this challenge through a purpose-built binary serialization layer defined in BinaryProtocol.swift. This implementation eliminates textual encoding overhead and maximizes data density to reduce radio airtime on mobile and wearable devices.

Fixed-Size Header Architecture

The protocol begins with a compact, versioned header of approximately 12 bytes containing only essential fields: version, flags, sender ID, recipient ID, and payload length. Unlike text-based protocols such as JSON, this fixed structure eliminates variable-length field delimiters and parsing overhead.

In localPackages/BitFoundation/Sources/BitFoundation/BinaryProtocol.swift, the header is structured to ensure deterministic memory layout. By avoiding variable-length text fields in the header, BitChat guarantees that every byte transmitted carries substantive protocol information rather than formatting markup.

Bit-Packed Flags and Boolean Compression

Boolean states are encoded using bit-field flags rather than separate bytes. A single flags byte encodes multiple boolean values including hasRoute, isCompressed, and isSigned.

As implemented in BinaryProtocol.swift (lines 25-34), this technique packs several boolean indicators into a single byte, reducing per-packet overhead by 75% compared to byte-aligned boolean storage. This micro-optimization compounds significantly when transmitting thousands of messages across BLE's high-latency links.

Optional Section Serialization

The protocol employs an optional section pattern for non-essential data such as route hop lists, digital signatures, and TLV metadata. These sections are only serialized when the corresponding flags indicate their presence.

The decoder skips absent sections automatically, ensuring that simple messages remain at minimal size. This conditional serialization prevents empty fields from consuming valuable payload space in the 20-byte BLE packets.

Payload Compression and Raw Identifier Encoding

Large payloads undergo automatic compression using z-standard before transmission. The isCompressed flag in the header indicates whether the receiver must decompress the payload, allowing a 1 KB message to shrink to a few hundred bytes.

Additionally, all identifiers—including peer IDs and route hops—are stored as raw 8-byte arrays rather than UTF-8 strings. This design choice, verified in localPackages/BitFoundation/Tests/BitFoundationTests/PeerIDTests.swift (line 480), eliminates string encoding overhead and reduces identifier transmission costs by approximately 50% compared to hexadecimal string representations.

BLE-Specific Fragmentation and Reassembly

When messages exceed the BLE MTU, the protocol implements intelligent fragmentation to minimize retransmission costs.

MTU-Aware Fragment Planning

The BLEOutboundFragmentPlanner (tested in bitchatTests/Services/BLEOutboundFragmentPlannerTests.swift) divides binary envelopes into fragments that fit within the current MTU. Each fragment carries minimal reassembly context, avoiding redundant header retransmission across multiple packets.

Fragment Assembly Buffer

On the receiving side, BLEFragmentAssemblyBuffer (located in bitchat/Services/BLE/BLEFragmentAssemblyBuffer.swift) reassembles incoming fragments using sequence information embedded in small fragment headers. This buffer manages memory efficiently, discarding partial packets that exceed timeout thresholds to prevent denial-of-service via memory exhaustion.

End-to-End Encoding Workflow

The binary protocol follows a strict encode-transmit-decode pipeline optimized for BLE constraints:

1. Message Creation: Applications instantiate BitchatMessage or BitchatPacket structures with recipient and payload data.
2. Binary Encoding: The BinaryProtocol.encode(packet, padding: false) method (lines 131-150 in BinaryProtocol.swift) computes the compact header, serializes optional sections, compresses the payload if beneficial, and returns a Data buffer ready for transmission.
3. BLE Transmission: BLEService (in bitchat/Services/BLE/BLEService.swift) transmits the binary data via CoreBluetooth, invoking fragmentation logic when necessary.
4. Reassembly and Decoding: The receiver's BLEFragmentAssemblyBuffer reconstructs complete envelopes, which BinaryProtocol.decode(data) then deserializes into BitchatPacket instances for application processing.

Practical Implementation Example

The following Swift code demonstrates the encoding and decoding cycle:

import BitFoundation

// Create a message for BLE transmission
let message = BitchatMessage(text: "Mesh network test", recipient: peerID)
let packet = BitchatPacket(message: message, senderID: localPeerID)

// Encode to binary wire format (minimal overhead)
let binaryData = try BinaryProtocol.encode(packet, padding: false)

// Transmit via BLE service
BLEService.shared.send(binaryData: binaryData)

// --- Receiver Implementation ---
BLEService.shared.onDataReceived = { fragmentData in
    // Accumulate fragments until complete
    if let completeData = BLEFragmentAssemblyBuffer.shared.append(fragmentData) {
        // Decode binary envelope back to packet
        if let receivedPacket = BinaryProtocol.decode(completeData) {
            processMessage(receivedPacket)
        }
    }
}

Summary

• Fixed-size headers in BinaryProtocol.swift eliminate parsing overhead and ensure predictable 12-byte minimum packet sizes.
• Bit-packed flags compress multiple boolean states into single bytes, reducing header bloat.
• Optional sections and raw byte identifiers prevent empty fields and string encoding from consuming limited BLE bandwidth.
• Z-standard compression with the isCompressed flag dramatically reduces payload sizes for larger messages.
• Intelligent fragmentation via BLEOutboundFragmentPlanner and BLEFragmentAssemblyBuffer handles BLE MTU limitations without redundant data transmission.

Frequently Asked Questions

Why does BitChat use a binary protocol instead of JSON for Bluetooth communication?

JSON requires extensive delimiter characters and UTF-8 encoding overhead that consumes 40-60% more bytes than equivalent binary representations. BitChat's binary protocol stores peer IDs as raw 8-byte arrays and uses fixed-size headers, ensuring that messages fit within BLE's 20-byte ATT payload limits without fragmentation when possible.

How does BitChat handle messages larger than the BLE MTU?

The BLEOutboundFragmentPlanner automatically splits binary envelopes into MTU-sized fragments, each containing a small fragment header referencing the original packet's flags. The receiver's BLEFragmentAssemblyBuffer reconstructs the complete envelope before BinaryProtocol.decode() processes the final payload, minimizing retransmission costs by avoiding duplicate header data.

What compression algorithm does BitChat use for payload optimization?

BitChat employs z-standard compression automatically when payload sizes warrant the CPU cost. The isCompressed flag in the binary header alerts receivers to decompress the payload after reassembly, typically reducing kilobyte-sized messages to a few hundred bytes suitable for efficient BLE transmission.

Where is the binary protocol implementation located in the repository?

The core encoding logic resides in localPackages/BitFoundation/Sources/BitFoundation/BinaryProtocol.swift, with BLE-specific fragmentation handled in bitchat/Services/BLE/BLEFragmentAssemblyBuffer.swift and transmission logic in bitchat/Services/BLE/BLEService.swift. Comprehensive test coverage exists in BinaryProtocolTests.swift and BLEOutboundFragmentPlannerTests.swift.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}