How to Enforce Strict Field Ordering with MarshalVTStrict in protobuf-go-lite

Enable the marshal_strict feature flag during code generation to emit MarshalVTStrict, MarshalToVTStrict, and MarshalToSizedBufferVTStrict methods that serialize protobuf fields in ascending tag order rather than wire-format order.

The aperturerobotics/protobuf-go-lite library provides high-performance VT (Variable Tag) marshaling for Go with optional strict field ordering. Unlike standard marshaling that follows wire format conventions (emitting oneof fields first), strict ordering guarantees fields appear in the exact sequence of their declared tag numbers, enabling deterministic serialization for caching, hashing, and debugging.

What Is Strict Field Ordering?

Standard protobuf marshaling in protobuf-go-lite follows the wire format specification defined in features/marshal/marshal.go, emitting oneof fields first, then remaining fields. This produces valid protobuf bytes, but field order depends on implementation details. Strict field ordering, implemented in the MarshalVTStrict family of methods, sorts fields by their numeric tag values and serializes them in ascending order (1, 2, 3, etc.) regardless of field type.

Enabling Strict Marshal Generation

The strict marshaling methods are generated only when the marshal_strict feature is enabled. This feature works alongside the standard marshal feature.

Generator Configuration

Pass marshal_strict to the features option when invoking protoc:

protoc \
  --plugin=protoc-gen-go-lite=$(go env GOPATH)/bin/protoc-gen-go-lite \
  --go-lite_out=. \
  --go-lite_opt=features=marshal+marshal_strict \
  path/to/your.proto

The generator, located in features/marshal/marshal.go, checks the strict flag at lines 24-28 to determine whether to emit the strict method variants.

Using MarshalVTStrict Methods

Once generated, each message type receives three strict marshaling methods that mirror the standard API but enforce tag-number ordering.

Method Comparison

Method	Buffer Management	Field Ordering
MarshalVT()	Allocates new buffer	Wire format (oneofs first)
MarshalVTStrict()	Allocates new buffer	Ascending tag order
MarshalToVT(buf)	Writes to provided buffer	Wire format
MarshalToVTStrict(buf)	Writes to provided buffer	Ascending tag order
MarshalToSizedBufferVT(buf)	Writes to exact-size buffer	Wire format
MarshalToSizedBufferVTStrict(buf)	Writes to exact-size buffer	Ascending tag order

Implementation Details

The strict ordering logic resides in features/marshal/marshal.go. When generating MarshalToSizedBufferVTStrict, the code generator first sorts the message fields by their descriptor number at lines 610-613:

sort.Slice(message.Fields, func(i, j int) bool {
    return message.Fields[i].Desc.Number() < message.Fields[j].Desc.Number()
})

Because MarshalToSizedBufferVTStrict fills the buffer from the end (using i := len(dAtA) and decrementing), the generator iterates through the sorted fields in reverse order (lines 25-36):

if p.strict {
    for i := len(message.Fields) - 1; i >= 0; i-- {
        field := message.Fields[i]
        // one-of handling omitted for brevity
        p.field(false, &numGen, field)
    }
}

This reverse iteration ensures that despite filling the buffer backwards, the final serialized output presents fields in ascending tag order (1, 2, 3). When strict is false, the generator follows the legacy ordering logic (oneofs first, then fields) at lines 38-41.

Practical Example

package main

import (
    "fmt"
    "log"

    examplepb "example.com/proto/example"
)

func main() {
    p := &examplepb.Person{
        Name:   "Alice",
        Age:    30,
        Active: true,
    }

    // Standard marshal – follows wire format order
    normal, err := p.MarshalVT()
    if err != nil {
        log.Fatal(err)
    }

    // Strict marshal – fields in tag order (1, 2, 3)
    strict, err := p.MarshalVTStrict()
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("normal: %x\n", normal)
    fmt.Printf("strict: %x\n", strict)
}

When to Use Strict Field Ordering

Strict marshaling is not required for standard protobuf compatibility—all decoders accept fields in any order. However, it provides specific advantages:

• Deterministic serialization: Produces identical byte sequences across multiple marshal operations, essential for cryptographic hashing, content-addressed storage, and test assertions comparing raw bytes.
• Debugging clarity: Field order in the binary output matches the .proto file definition, making hex dumps easier to read.
• Interoperability: Some legacy systems or testing frameworks expect fields in tag-number order rather than wire-format order.

Summary

• Enable marshal_strict alongside marshal in your protoc invocation to generate strict ordering methods.
• Use MarshalVTStrict(), MarshalToVTStrict(), or MarshalToSizedBufferVTStrict() to serialize messages with fields ordered by ascending tag number.
• The implementation in features/marshal/marshal.go sorts fields by descriptor number at generation time and iterates in reverse to accommodate the buffer-filling strategy.
• Strict marshaling ensures deterministic, reproducible output ideal for hashing, caching, and debugging without breaking wire compatibility.

Frequently Asked Questions

What is the difference between MarshalVT and MarshalVTStrict?

MarshalVT follows the standard protobuf wire format, emitting oneof fields first followed by other fields. MarshalVTStrict emits all fields in strictly ascending tag number order (1, 2, 3, etc.) as declared in the .proto file, providing deterministic serialization while maintaining wire compatibility.

How do I enable strict field ordering in protobuf-go-lite?

Add marshal_strict to the features option when running protoc. For example: --go-lite_opt=features=marshal+marshal_strict. This instructs the generator in features/marshal/marshal.go to emit the MarshalVTStrict method family and enable the sorting logic that orders fields by tag number.

Does strict marshaling affect performance?

Strict marshaling incurs minimal overhead. The field sorting occurs once during code generation in features/marshal/marshal.go (lines 610-613), not at runtime. The generated code uses pre-sorted field indices, so runtime performance is comparable to standard marshaling, with only a negligible difference in the iteration pattern.

Can I use strict marshaling with existing proto files?

Yes. Strict marshaling is fully backward compatible—it produces valid protobuf wire format that any standard decoder can parse. The strict ordering only affects the serialization side; deserialization remains unchanged, and messages serialized with MarshalVTStrict can be read by any standard protobuf implementation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}