OpenLogi IPC Wire Format: Bincode Serialization with Length-Delimited Framing

OpenLogi uses bincode v1.3 with length-delimited, varint-encoded framing, serializing all IPC messages using little-endian varint integers and append-only enum variant indices.

The AprilNEA/OpenLogi repository implements a high-performance inter-process communication system that relies on a compact binary IPC wire format to exchange messages between agents, the GUI, and CLI tools. Understanding this format is essential for extending the protocol or integrating external tools with the OpenLogi ecosystem.

Core Components of the OpenLogi IPC Wire Format

Bincode Serialization with DefaultOptions

All messages crossing the IPC boundary are encoded using bincode with bincode::DefaultOptions. According to the test suite in crates/openlogi-ipc/tests/wire_format.rs (lines 4-9), this configuration:

• Uses little-endian byte order for all multi-byte values
• Encodes integers as variable-length varints to minimize payload size
• Rejects trailing data during deserialization to prevent corruption

This standardization ensures that every component—from the Rust agent to the overlay—produces identical byte representations for the same data structures.

Length-Delimited Frame Structure

The transport layer implements length-delimited framing to separate individual messages on the stream. As implemented in crates/openlogi-ipc/src/transport.rs, each payload is wrapped with a varint-encoded size prefix:

1. The payload is serialized using bincode
2. The byte length is encoded as a varint u64
3. The length prefix is written to the stream, followed by the raw payload

The receiver reads the varint length first, then allocates exactly that many bytes for the complete message. This approach prevents message boundary errors and supports efficient streaming parsing.

Append-Only Enum Encoding

Enums that cross the IPC boundary—such as InventoryHealth, PairingPhase, and MonitorEvent—follow a strict append-only rule documented in crates/openlogi-ipc/src/ipc.rs. These enums are encoded by their variant index only (not field names or discriminant values), which means:

• Forward compatibility: Adding new variants to the end of an enum is safe
• Breaking changes: Removing or reordering existing variants changes the wire format and will cause deserialization failures

For example, InventoryHealth (defined at line 89 in ipc.rs) must never have variants inserted in the middle of its definition, as this would shift the indices of subsequent variants.

Key Source Files and Implementation

The IPC wire format is defined, implemented, and verified across three critical files in the crates/openlogi-ipc package:

• src/ipc.rs: Defines the core data model, including structs and enums with documentation on the append-only requirement
• src/transport.rs: Implements the length-delimited framing logic, including the wrap function that prefixes payloads with varint length headers
• tests/wire_format.rs: Contains exhaustive tests that pin the exact binary representation of every IPC-exposed type, ensuring any format change is intentional and versioned

The design contract is further documented in AGENTS.md, which explains why bincode was selected and how enum evolution is managed across releases.

Practical Examples of the IPC Wire Format

Serializing Requests (Client Side)

When sending commands to the OpenLogi agent, clients use the standard bincode options to ensure compatibility:

use bincode::Options;
use openlogi_ipc::client;
use openlogi_ipc::ipc::ActionRingCommand;

fn send_action() -> Result<(), Box<dyn std::error::Error>> {
    // Create a request – the enum variant is encoded by its index only
    let request = ActionRingCommand::ToggleRing;

    // Serialize with the exact options used throughout the crate
    let bytes = bincode::DefaultOptions::new()
        .serialize(&request)
        .expect("serialization should succeed");

    // The client transport applies length-delimited framing automatically
    client::send(&bytes)?;
    Ok(())
}

This matches the serialization logic verified in wire_format.rs, ensuring the byte sequence ToggleRing produces is stable across versions.

Deserializing Responses (Server Side)

Server components handle incoming frames by stripping the length prefix (handled by the transport layer) and deserializing the payload:

use bincode::Options;
use openlogi_ipc::ipc::MonitorEvent;

fn handle_message(frame: &[u8]) -> Result<MonitorEvent, bincode::Error> {
    // The transport ensures `frame` contains a single, complete message
    let msg: MonitorEvent = bincode::DefaultOptions::new()
        .deserialize(frame)?;
    Ok(msg)
}

MonitorEvent is an append-only enum whose variants are encoded strictly by their index, as defined in ipc.rs.

Implementing Length-Delimited Framing

The transport layer manually constructs frames by prefixing varint-encoded lengths. This pattern from transport.rs demonstrates the exact wire format:

use tokio::io::{AsyncReadExt, AsyncWriteExt};
use bincode::Options;

async fn write_frame<W: AsyncWriteExt + Unpin>(
    writer: &mut W, 
    payload: &[u8]
) -> std::io::Result<()> {
    // Encode length as a varint (bincode's default for u64)
    let len_bytes = bincode::DefaultOptions::new()
        .serialize(&(payload.len() as u64))
        .unwrap();
    
    writer.write_all(&len_bytes).await?;
    writer.write_all(payload).await
}

The receiver parses the varint length first, then reads exactly that many bytes to obtain the complete bincode payload.

Compatibility and Versioning Strategy

The OpenLogi IPC wire format maintains backwards compatibility through two mechanical safeguards:

• Exhaustive test coverage: The wire_format.rs test suite locks down the exact byte representation of every public type. Any change to the bincode configuration or enum ordering will cause immediate test failures
• Append-only constraints: Enums exposed over IPC must only add new variants at the end. This rule is enforced by code review and verified by the test suite, ensuring that old clients can still deserialize messages from new servers (though they may ignore unrecognized variants)

Removing or reordering enum variants constitutes a breaking change that requires a coordinated migration or major version bump.

Summary

• OpenLogi's IPC wire format combines bincode v1.3 (little-endian, varint integers) with length-delimited framing using varint-encoded size prefixes
• All enums crossing the boundary are append-only and encoded by variant index only, ensuring forward compatibility
• The transport implementation lives in crates/openlogi-ipc/src/transport.rs, while data models and compatibility rules are defined in src/ipc.rs
• The wire_format.rs test suite guards against accidental wire format changes by verifying exact byte representations
• This format enables efficient, zero-copy communication between the agent, GUI, and CLI components while maintaining strict versioning guarantees

Frequently Asked Questions

What serialization library does OpenLogi use for IPC?

OpenLogi uses bincode v1.3 with DefaultOptions for all IPC serialization. This configuration produces little-endian, varint-encoded output that rejects trailing data, ensuring compact message sizes and strict deserialization requirements.

How does OpenLogi prevent message framing errors in the IPC wire format?

The transport layer implements length-delimited framing where each payload is prefixed with its size encoded as a varint u64. The receiver first parses this length header, then reads exactly that many bytes to obtain the complete message, preventing partial reads or message concatenation errors.

Why must OpenLogi's IPC enums remain append-only?

Because enums are encoded by their variant index (not name or discriminant), inserting or removing variants would shift the indices of existing variants, breaking wire compatibility with existing deployments. New variants must be added only at the end of the enum definition to maintain backwards compatibility.

Where is the IPC wire format formally specified in the source code?

The canonical specification resides in crates/openlogi-ipc/tests/wire_format.rs, which documents the exact bincode options used and verifies the byte-level representation of every IPC type. Implementation details are found in src/transport.rs (framing) and src/ipc.rs (data models), with design rationale documented in AGENTS.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}