How go-sql-driver/mysql Implements MySQL Protocol Packet Framing and Sequence Number Management

The go-sql-driver/mysql package implements the MySQL wire protocol by encoding every payload with a 4-byte header (3 bytes for length, 1 byte for sequence number) in packets.go, while managing per-connection sequence counters in the mysqlConn struct to ensure reliable bidirectional communication.

The go-sql-driver/mysql driver handles all low-level MySQL client/server communication through precise packet framing and sequence tracking. Understanding these mechanics is essential for debugging connection issues or extending the driver. The implementation centers on two core files: packets.go for wire-format encoding and connection.go for state management.

MySQL Protocol Packet Framing in Go

MySQL's wire protocol requires every logical message to be split into one or more physical packets, each prefixed with metadata that enables reassembly and ordering.

The 4-Byte Header Structure

Every packet transmitted by the driver begins with a fixed 4-byte header. The writePacket function in packets.go constructs this header by encoding a 3-byte little-endian integer representing the payload length, followed by a 1-byte sequence number.

// Conceptual representation of the header construction in writePacket
data := make([]byte, 4+len(payload))
data[0] = byte(length)
data[1] = byte(length >> 8)
data[2] = byte(length >> 16)
data[3] = mc.sequence  // Sequence number

When reading, readPacket parses these bytes to determine how many payload bytes to read next and validates the sequence identifier against the expected value.

Handling Payload Fragmentation

MySQL limits individual packets to maxPacketSize = 1<<24 - 1 (16 MiB minus 1 byte). When writePacket encounters a payload exceeding this limit, it automatically fragments the data into multiple max-size chunks.

The function loops through the payload, writing each chunk with its own header and an incremented sequence number. This fragmentation is transparent to higher-level code—commands, handshakes, and result sets all flow through this chunking logic.

Packet Reassembly Logic

On the receive path, readPacket handles defragmentation by accumulating payloads in a prevData buffer. The driver continues reading packets until it encounters one with a length less than maxPacketSize, signaling the final chunk of a sequence.

A special zero-length terminator case exists: when a split payload exactly fills the 16 MiB limit, MySQL sends a final packet with a length of zero to indicate completion. The driver detects this condition and returns the fully assembled data to the caller.

Sequence Number Management Strategy

Sequence numbers prevent desynchronization between client and server, ensuring packets are processed in the correct order even when multiple logical commands flow over the same connection.

Per-Connection Sequence Counters

The mysqlConn struct defined in connection.go maintains two counters:

• sequence – Tracks the packet sequence for uncompressed traffic
• compressSequence – A separate counter used when connection compression is enabled

These fields are initialized to zero when the connection is established and increment independently based on traffic direction.

Validating Incoming Packet Sequences

Before processing received data, readPacket extracts the sequence byte from the header (seq := data[3]) and compares it against mc.sequence. If the values diverge and compression is not active, the driver logs a warning, sets invalidSequence = true, and may return a ErrPktSync error to indicate the connection is out of sync.

Outgoing packets automatically advance the counter: writePacket embeds the current sequence value in the header, then increments mc.sequence++ immediately after a successful write.

Compression Edge Cases

When mc.compress is enabled, MySQL does not enforce sequence number checks for compressed packet streams. The driver mirrors this behavior by skipping sequence validation entirely when compression is active, preventing false synchronization errors that would otherwise occur due to MySQL's undocumented net_flush() behavior.

Resetting and Synchronizing Sequences

The driver provides two helper methods on mysqlConn:

• resetSequence – Clears both sequence and compressSequence to zero, invoked at the start of new logical command streams (such as after authentication handshakes)
• syncSequence – Called after transmitting a series of packets and before reading the response; when compression is active, this method copies the compression-specific counter back to the normal sequence counter to maintain consistency with the server's internal state

Implementation Details in packets.go and connection.go

The packet layer exposes a clean interface to higher-level protocol handlers while encapsulating the complexity of MySQL's wire format.

writePacket Implementation

Located in packets.go, writePacket accepts a payload byte slice and handles all framing concerns. It calculates the number of chunks required, writes each 4-byte header followed by the corresponding payload segment, and manages the sequence counter increment. If any write fails or if a payload exceeds maxAllowedPacket, the function returns driver.ErrBadConn or ErrPktTooLarge.

readPacket Implementation

readPacket performs the inverse operation, reading the 4-byte header to determine payload length, validating the sequence number, and appending data to an internal buffer. It returns the complete, reassembled payload only after receiving the final chunk (identified by a length less than maxPacketSize or a zero-length terminator).

Working with the Packet Layer

While most developers interact with the driver through database/sql, understanding the packet layer helps when debugging protocol errors or implementing custom commands.

Basic Connection with Compression

Enabling compression exercises the syncSequence logic and dual counter system:

import (
    "database/sql"
    _ "github.com/go-sql-driver/mysql"
)

func main() {
    dsn := "user:password@tcp(127.0.0.1:3306)/testdb?compress=true"
    db, err := sql.Open("mysql", dsn)
    if err != nil {
        panic(err)
    }
    defer db.Close()

    // The driver creates a *mysqlConn, which internally uses readPacket/writePacket
    rows, err := db.Query("SELECT id, name FROM employees")
    if err != nil {
        panic(err)
    }
    defer rows.Close()
}

Sending Low-Level Command Packets

For advanced use cases requiring raw protocol access:

func sendPing(conn *sql.Conn) error {
    raw, err := conn.Raw(func(driverConn interface{}) error {
        // Type-assert to the driver's internal connection type
        mc := driverConn.(*mysql.mysqlConn)
        
        // COM_PING command = 0x0e, no payload
        if err := mc.writeCommandPacket(0x0e); err != nil {
            return err
        }
        
        // Read the server's OK packet (validates sequence number)
        _, err := mc.readPacket()
        return err
    })
    return raw
}

The writeCommandPacket method internally calls writePacket, adding the 4-byte header and updating mc.sequence. The subsequent readPacket validates the server's response sequence against the expected value.

Summary

• Packet framing occurs in packets.go, where writePacket and readPacket handle the 4-byte header (3-byte length + 1-byte sequence) and manage payloads up to maxPacketSize (16 MiB - 1)
• Fragmentation and reassembly are automatic: large payloads split into multiple packets on send and accumulate in prevData until the final chunk arrives
• Sequence management uses two counters in mysqlConn (sequence and compressSequence) defined in connection.go, with resetSequence and syncSequence coordinating state transitions
• Validation ensures incoming packets match expected sequence numbers, except when compression is enabled (matching MySQL's behavior)
• Error handling converts protocol violations into driver.ErrBadConn, ErrPktTooLarge, or ErrPktSync for clear diagnostics

Frequently Asked Questions

What is maxPacketSize in the MySQL protocol?

maxPacketSize is a constant defined as 1<<24 - 1 (16,777,215 bytes or 16 MiB minus 1), representing the maximum payload length for a single MySQL packet. The driver references this limit in packets.go to determine when to fragment outgoing data or when to stop accumulating incoming chunks during reassembly.

How does the driver handle out-of-sequence packets?

When readPacket detects a sequence number in the header that does not match the expected mc.sequence value, and compression is not active, it logs a warning, sets the invalidSequence flag on the connection, and may return an ErrPktSync error. This indicates the client and server have lost synchronization, typically requiring connection reset.

Why does compression disable sequence validation?

MySQL's compression protocol does not enforce sequence number checks for compressed packets due to internal net_flush() behavior that can advance counters unexpectedly. The driver respects this by skipping validation when mc.compress is true, preventing false synchronization errors while still tracking sequence numbers internally via compressSequence.

Where is the packet framing logic implemented?

The core framing logic resides in packets.go (writePacket and readPacket functions), while sequence state management lives in connection.go within the mysqlConn struct (fields sequence and compressSequence, plus methods resetSequence and syncSequence).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}