SemanticFrame vs TerminalAnsi in herdr: Understanding Render Encoding Differences

herdr uses two distinct wire encodings—SemanticFrame transmits complete structured frame data for client-side rendering, while TerminalAnsi sends diff-based ANSI escape sequences for direct terminal output—negotiated during the initial client handshake.

The herdr terminal server architecture supports multiple client types by offering flexible render encoding options. Whether you are building a headless test client or an interactive terminal attachment, understanding the difference between SemanticFrame and TerminalAnsi render encoding in herdr allows you to optimize bandwidth usage and rendering complexity.

The RenderEncoding Enum Definition

The protocol defines these encodings in the RenderEncoding enum located in src/protocol/wire.rs. This enum is serialized during the Hello handshake message to establish how the server will transmit frame updates.

/// Render payload encoding negotiated during client handshake.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum RenderEncoding {
    /// Send full semantic FrameData values. This is the local/default mode.
    SemanticFrame,
    /// Send already‑diffed terminal ANSI byte streams.
    TerminalAnsi,
}

Source: [src/protocol/wire.rs](https://github.com/ogulcancelik/herdr/blob/master/src/protocol/wire.rs#L37-L44)

How SemanticFrame Encoding Works

When a client requests SemanticFrame encoding, the server maintains the full semantic representation of the UI state. The server compares complete FrameData structures between renders and only transmits when changes occur.

Frame Preparation Logic

The server tracks per-client state using ClientRenderState, which stores the last frame for semantic clients:

pub(crate) enum ClientRenderState {
    /// Semantic clients compare full frame data and skip identical frames.
    Semantic { last_frame: Option<FrameData> },
    /// Terminal‑ANSI clients keep a terminal diff encoder and sequence number.
    TerminalAnsi { blit_encoder: BlitEncoder, seq: u64 },
}

Source: [src/server/render_stream.rs](https://github.com/ogulcancelik/herdr/blob/master/src/server/render_stream.rs#L12-L18)

In the prepare_frame() method, the server checks the new frame against last_frame. If the frames match, the server sends nothing; otherwise, it returns a ServerMessage::Frame containing the complete FrameData:

Self::Semantic { last_frame } => {
    if last_frame.as_ref() == Some(frame) { return None; }
    Some(PreparedRender {
        message: ServerMessage::Frame(frame.clone()),
        encoded: None,
    })
}

Source: [src/server/render_stream.rs](https://github.com/ogulcancelik/herdr/blob/master/src/server/render_stream.rs#L44-L53)

How TerminalAnsi Encoding Works

TerminalAnsi encoding minimizes network traffic by computing frame diffs server-side and transmitting only the ANSI escape sequences necessary to update the terminal display. This approach is ideal for clients that pipe output directly to a physical terminal.

The BlitEncoder Implementation

The BlitEncoder struct in src/protocol/render_ansi.rs manages the differential encoding state:

pub(crate) struct BlitEncoder {
    last_frame: Option<FrameData>,
    last_visible_cursor: Option<(u16, u16)>,
    last_cursor_shape: u8,
}

Source: [src/protocol/render_ansi.rs](https://github.com/ogulcancelik/herdr/blob/master/src/protocol/render_ansi.rs#L44-L51)

The encoder provides two primary methods: encode() generates the ANSI byte stream by diffing against the previous frame, and commit() updates the baseline after successful transmission.

Frame Diffing and ANSI Generation

When processing a frame for a TerminalAnsi client, the server first checks blit_encoder.is_current(frame) to detect unchanged frames. If the frame differs, the encoder produces an EncodedBlit containing cursor moves, SGR attributes, and synchronized output sequences:

Self::TerminalAnsi { blit_encoder, seq } => {
    if blit_encoder.is_current(frame) { return None; }
    let mut encoded = blit_encoder.encode(frame, false);
    insert_graphics_before_sync_end(&mut encoded.bytes, &frame.graphics);
    Some(PreparedRender {
        message: ServerMessage::Terminal(TerminalFrame {
            seq: *seq + 1,
            width: frame.width,
            height: frame.height,
            full: encoded.full,
            bytes: encoded.bytes.clone(),
        }),
        encoded: Some(encoded),
    })
}

Source: [src/server/render_stream.rs](https://github.com/ogulcancelik/herdr/blob/master/src/server/render_stream.rs#L55-L71)

Key Differences Between SemanticFrame and TerminalAnsi

• Payload Structure: SemanticFrame transmits structured FrameData objects containing cells, colors, hyperlinks, and graphics metadata. TerminalAnsi transmits raw byte streams of ANSI escape codes that a terminal emulator can interpret directly.

• Bandwidth Efficiency: SemanticFrame sends complete frame data on every change, resulting in larger payloads for complex UIs. TerminalAnsi sends only the differences between frames after computing diffs with BlitEncoder, significantly reducing bandwidth after the initial render.

• Client Complexity: SemanticFrame clients must implement full-frame rendering logic to convert FrameData into visible output, making it suitable for GUI clients or testing frameworks. TerminalAnsi clients can forward bytes directly to stdout, requiring no intermediate parsing or rendering logic.

• Use Cases: The default herdr app client uses SemanticFrame for local rendering, while herdr terminal attach prefers TerminalAnsi to minimize latency and avoid reconstructing semantic frames from terminal output.

Practical Implementation in herdr

Client Handshake Negotiation

Clients specify their preferred encoding during the initial Hello message. The requested_encoding field determines which rendering path the server initializes:

let hello = ClientMessage::Hello {
    version: PROTOCOL_VERSION,
    cols,
    rows,
    cell_width_px,
    cell_height_px,
    // Request the diff‑based ANSI stream
    requested_encoding: RenderEncoding::TerminalAnsi,
    keybindings: ClientKeybindings::Local { keys_toml: … },
    launch_mode: ClientLaunchMode::TerminalAttach,
};

Server-Side State Initialization

Upon receiving the handshake, the server creates the appropriate ClientRenderState variant based on the requested encoding:

let render_state = ClientRenderState::new(hello.requested_encoding);
// => Semantic { … } or TerminalAnsi { blit_encoder, seq: 0 }

Source: [src/server/render_stream.rs](https://github.com/ogulcancelik/herdr/blob/master/src/server/render_stream.rs#L20-L28)

Frame Transmission Flow

Both encoding types share the same high-level transmission pattern in src/server/render_stream.rs, differing only in the message variant produced by prepare_frame():

if let Some(prep) = render_state.prepare_frame(&frame) {
    // prep.message is either ServerMessage::Frame (Semantic) 
    // or ServerMessage::Terminal (TerminalAnsi)
    client.send(prep.message);
    render_state.commit_sent_frame(frame, prep);
}

Summary

• SemanticFrame sends complete FrameData structures and relies on the client to handle rendering, suitable for headless tests and GUI implementations.
• TerminalAnsi uses BlitEncoder to compute frame diffs server-side, transmitting only ANSI escape sequences for minimal bandwidth and direct terminal compatibility.
• The encoding is negotiated once during the Hello handshake via src/protocol/wire.rs and stored in ClientRenderState (src/server/render_stream.rs).
• Both encodings skip transmission when frames are unchanged, but TerminalAnsi additionally optimizes by sending only modified regions rather than full buffers.

Frequently Asked Questions

What is the primary difference between SemanticFrame and TerminalAnsi encoding?

SemanticFrame transmits complete semantic descriptions of the UI state as structured data, requiring the client to interpret and render cells, colors, andgraphics. TerminalAnsi transmits raw ANSI escape byte streams that update the terminal display directly, offloading the rendering logic to the server via the BlitEncoder.

How does herdr's BlitEncoder optimize TerminalAnsi rendering?

The BlitEncoder in src/protocol/render_ansi.rs maintains a copy of the last transmitted frame and computes diffs using the encode() method. It generates ANSI sequences only for changed cells and cursor movements, wraps them in synchronized-output escape sequences, and suppresses transmission entirely when is_current() detects no changes.

Which herdr clients use SemanticFrame versus TerminalAnsi?

According to the herdr source code in src/server/headless.rs and client transport implementations, the default local client (herdr app) uses SemanticFrame to enable rich client-side rendering, while the herdr terminal attach command uses TerminalAnsi to stream efficient byte sequences directly to physical terminals.

Where is the render encoding negotiated in the herdr protocol?

The encoding is negotiated during the initial handshake in src/protocol/wire.rs via the Hello message's requested_encoding field. The server validates this request in src/server/client_transport.rs and initializes the corresponding ClientRenderState variant to handle all subsequent frame preparation for that connection.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}