BitChat Protocol Binary Packet Format and Message Fragmentation Explained

The BitChat protocol uses a compact, versioned binary wire format with bitmasked flags for optional fields, and automatically fragments oversized messages using dedicated fragment packets with 8-byte fragment IDs that are reassembled via BLEFragmentAssemblyBuffer.

The BitChat protocol powers decentralized, peer-to-peer messaging over Bluetooth Low Energy (BLE). At its core lies a carefully designed binary packet format optimized for constrained networks, paired with transparent message fragmentation that handles BLE's small MTU without burdening application developers. This deep dive examines the wire format implemented in BinaryProtocol.swift and the fragmentation machinery orchestrated by BLEOutboundFragmentPlanner and BLEFragmentAssemblyBuffer.

BitChat Protocol Binary Packet Structure

The protocol defines two versions: v1 for basic packets and v2 for routed messages with extended length fields. All packets share a common header structure with version-specific variations.

Fixed Header Fields

Offset (bytes)	Field	Size	Description
0	Version	1	0x01 for v1; 0x02 for v2 (adds 4-byte length and route support)
1	Type	1	MessageType enum value (.message, .fragment, .announce, etc.)
2	TTL	1	Hop count limit for message propagation
3–10	Timestamp	8	Unix epoch milliseconds (big-endian)
11	Flags	1	Bitmask controlling optional field presence

Variable-Length Fields

Following the fixed header, the packet contains:

• Payload length: 2 bytes (v1) or 4 bytes (v2)
• Sender-ID: 8 bytes (always present)
• Recipient-ID: 8 bytes (present when Flags.hasRecipient is set)
• Route: 1 byte hop count + N×8 byte hop IDs (v2 only, when Flags.hasRoute is set)
• Original size: 2 or 4 bytes (when Flags.isCompressed is set)
• Payload: Variable-length data (compressed or raw)
• Signature: 64 bytes (when Flags.hasSignature is set)

Flags Bitmask (BinaryProtocol.Flags)

Defined in BinaryProtocol.swift, the flags byte uses the following bit positions:

Bit	Constant	Meaning
0	hasRecipient	Directed message with explicit recipient
1	hasSignature	Ed25519 signature appended
2	isCompressed	Payload compressed with zlib
3	hasRoute	Source routing information included (v2 only)
4	isRSR	Return source routing flag
5–7	—	Reserved for future use

The protocol automatically selects v2 when route information is present, as implemented in BLEOutboundFragmentPlanner.swift.

Message Fragmentation Architecture

BLE's typical MTU of 512 bytes cannot accommodate the protocol's maximum packet size of 65,535 bytes. The fragmentation system splits large messages transparently.

Fragment Packet Type

The MessageType enum defines .fragment = 0x20 for fragment packets. When a BitchatPacket exceeds the link MTU, the encoder:

1. Compresses the payload if it exceeds 256 bytes
2. Computes optimal chunk sizes via BLEOutboundFragmentPlanner
3. Generates fragment packets with extended headers

BLEOutboundFragmentPlanner

This component determines fragmentation strategy:

// From BLEOutboundFragmentPlanner.swift - chunk size calculation
func fragmentChunkSize(for mtu: Int, version: ProtocolVersion) -> Int {
    // Account for header overhead, flags, and optional fields
    let headerOverhead = version == .v1 ? 13 : 17  // base header + length field
    let maxPayloadPerPacket = mtu - headerOverhead - MemoryLayout<BLEFragmentHeader>.size
    
    // Ensure alignment for optimal BLE transmission
    return maxPayloadPerPacket & ~0x03  // 4-byte alignment
}

The planner considers:

• Link MTU: Negotiated BLE connection parameter
• Protocol version: v2 required for routed fragments
• Compression state: Fragments contain compressed chunks, not individual compression
• Padding requirements: Final fragment padded via MessagePadding.pad

BLEFragmentHeader Structure

Each fragment packet carries an 8-byte fragment identifier plus sequencing information:

Field	Size	Purpose
Fragment ID	8 bytes	Unique identifier linking fragments of same message
Total fragments	2 bytes	Expected fragment count
Fragment index	2 bytes	Position in sequence (0-based)
Reserved	4 bytes	Future extensions, currently zero

BLEFragmentAssemblyBuffer

Reassembly happens in BLEFragmentAssemblyBuffer.swift:

// Reassembly workflow
class BLEFragmentAssemblyBuffer {
    private var assemblies: [UInt64: FragmentAssembly] = [:]
    
    func processFragment(_ fragment: BitchatPacket) -> BitchatPacket? {
        guard fragment.type == .fragment else { return fragment }
        
        let header = extractFragmentHeader(from: fragment)
        var assembly = assemblies[header.fragmentID, default: FragmentAssembly(
            totalFragments: header.totalFragments,
            expectedSize: computeExpectedSize(header)
        )]
        
        assembly.addFragment(header.fragmentIndex, payload: fragment.payload)
        
        // Eviction: drop assemblies older than 30 seconds or with gaps > 5 fragments
        if assembly.isComplete {
            assemblies.removeValue(forKey: header.fragmentID)
            return assembly.reassemble()
        } else if assembly.shouldEvict {
            assemblies.removeValue(forKey: header.fragmentID)
            return nil
        }
        
        assemblies[header.fragmentID] = assembly
        return nil  // waiting for more fragments
    }
}

The buffer implements:

• Timeout eviction: Incomplete assemblies expire after 30 seconds
• Gap detection: Missing fragments trigger selective retransmission requests
• Memory bounds: Maximum 16 concurrent assemblies per peer
• Validation: CRC32 checksum verification on completed reassembly

Complete Encoding and Decoding Example

import BitFoundation

// Create a message that will exceed typical BLE MTU
let largePayload = Data(repeating: 0xA5, count: 2048)
let packet = BitchatPacket(
    version: .v2,  // Required for routing
    type: .message,
    senderID: myPeerID,           // 8-byte identifier
    recipientID: targetPeerID,    // Triggers hasRecipient flag
    timestamp: UInt64(Date().timeIntervalSince1970 * 1000),
    payload: largePayload,
    signature: nil,
    route: Route(hops: [relayPeerID]),  // Triggers hasRoute flag, forces v2
    ttl: 7
)

// Encode: compression + fragmentation applied automatically
guard let encoded = BinaryProtocol.encode(packet) else {
    fatalError("Encoding failed: \(BinaryProtocol.lastError)")
}

//(encoded is now [Data] if fragmented, or single Data)
for fragmentData in encoded {
    blePeripheral.writeValue(fragmentData, for: characteristic, type: .withoutResponse)
}

// ---- Receiver implementation ----
func peripheral(_ peripheral: CBPeripheral, 
                didUpdateValueFor characteristic: CBCharacteristic,
                error: Error?) {
    
    guard let data = characteristic.value else { return }
    
    // Decoding handles both fragments and complete packets
    if let packet = BinaryProtocol.decode(data) {
        // Returns nil for incomplete fragments, BitchatPacket when reassembled
        handleIncomingMessage(packet)
    }
}

Key Protocol Design Decisions

The BitChat protocol's binary format reflects several deliberate engineering choices:

• Version in first byte: Enables parsers to branch immediately without parsing variable fields
• Big-endian timestamps: Consistent network byte order regardless of platform
• Compression before fragmentation: Reduces total fragment count and improves throughput
• 8-byte fragment IDs: Sufficiently large to avoid collisions in high-throughput scenarios
• Implicit v2 upgrade: Protocol version selected automatically based on feature requirements

Summary

• Binary format: Versioned header (1 byte) + flags (1 byte) + variable optional fields determined by bitmask; v2 adds 4-byte length fields and route support
• Key files: BinaryProtocol.swift (wire format), MessageType.swift (type enum including fragment = 0x20), BitchatPacket.swift (packet struct)
• Fragmentation: BLEOutboundFragmentPlanner computes chunk sizes considering MTU and version; fragment packets carry 8-byte IDs with sequence metadata
• Reassembly: BLEFragmentAssemblyBuffer manages partial assemblies with timeout eviction, gap detection, and CRC32 validation
• Transparency: Application code uses standard encode()/decode() calls; fragmentation occurs automatically for oversized payloads

Frequently Asked Questions

What triggers protocol version 2 in BitChat packets?

Version 2 is automatically selected when any packet includes route information (the hasRoute flag) or when the payload length exceeds 65,535 bytes. The BLEOutboundFragmentPlanner upgrades the version during encoding if it detects a non-nil route property or extended payload requirements. Application code does not manually specify the version.

How does BitChat handle fragment loss or out-of-order delivery?

The BLEFragmentAssemblyBuffer tracks received fragments by their 8-byte fragment ID and 2-byte fragment index. It accepts out-of-order fragments and buffers them until the complete sequence arrives. If gaps exceed 5 fragments or 30 seconds elapse without completion, the assembly is evicted and higher-layer retransmission (handled by the reliability protocol) occurs.

Why compress before fragmentation rather than compressing each fragment?

Compressing the full payload first achieves better compression ratios due to dictionary reuse across the entire message, then fragmenting the compressed output. This reduces total bytes transmitted and fragment count. Per-fragment compression would add overhead and reduce efficiency, particularly for small final fragments.

What is the maximum message size supported by BitChat?

The protocol supports payloads up to 65,535 bytes uncompressed (v1 length field limit) or 4,294,967,295 bytes with v2's 4-byte length field. Practical limits are constrained by BLEFragmentAssemblyBuffer memory bounds (16 concurrent assemblies) and BLE connection reliability. The reference implementation optimizes for messages under 1 MB.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}