How to Handle Vertex Locks During Mesh Simplification in meshoptimizer

Pass a byte array to meshopt_simplify using the meshopt_SimplifyVertex_Lock, meshopt_SimplifyVertex_Protect, or meshopt_SimplifyVertex_Priority flags to prevent specific vertices from moving, protect attribute seams, or prioritize retention during polygon reduction.

The meshoptimizer library provides a robust mesh simplification pipeline that respects artist-authored constraints through a per-vertex locking mechanism. By supplying a vertex_lock array when calling the simplification API, you can preserve silhouette borders, prevent UV seam breaks, or ensure critical mesh details survive aggressive LOD generation. The implementation spans flag definitions in src/meshoptimizer.h and the classification logic in src/simplifier.cpp.

Vertex Lock Flag Definitions

Three bitwise flags control vertex behavior during simplification, defined in src/meshoptimizer.h (lines 86-95):

• meshopt_SimplifyVertex_Lock (1 << 0): The vertex must never move from its original position.
• meshopt_SimplifyVertex_Protect (1 << 1): The vertex may move only if the edge collapse respects attribute seams; effective only when using meshopt_SimplifyPermissive.
• meshopt_SimplifyVertex_Priority (1 << 2): The vertex receives preferential treatment during collapse ordering, increasing its chance of survival.

These flags operate on individual bytes in the lock array, allowing you to mix constraints across different vertices in the same mesh.

Classification and Lock Propagation

Before any edge collapses occur, the classifyVertices function (src/simplifier.cpp, lines 617-638) processes the vertex_lock array to determine which vertices must be preserved. The algorithm iterates through all vertices and forces the Locked classification kind for any entry containing the Lock bit (lines 640-720).

The function also propagates the locked status to all wedges that share the same original vertex. This ensures that attribute splits—where one logical vertex becomes multiple physical vertices for UV or normal discontinuities—do not accidentally unlock protected geometry.

Automatic Border Locking

The meshopt_SimplifyLockBorder option (bit 0) provides automatic silhouette preservation. When this flag is passed to the simplifier, every vertex classified as Border is upgraded to Locked status immediately after the initial classification phase (src/simplifier.cpp, lines 633-638).

Use this option when simplifying open meshes where the outer hull must remain intact, eliminating the need to manually identify and flag boundary vertices.

Permissive Mode and Seam Protection

When meshopt_SimplifyPermissive is enabled, the simplifier treats seam and border vertices as Complex rather than immediately locking them. In this mode, a vertex becomes permanently locked only if it meets specific safety criteria defined in src/simplifier.cpp (lines 744-758).

A vertex transitions to Locked in permissive mode if:

• Any of its wedges carries the Protect flag, or
• The vertex lies on a true border with no opposite edge.

This logic allows collapses across attribute seams when topology permits, while the Protect flag prevents degenerate collapses that would break UV continuity.

Final Enforcement Pass

After permissive processing completes, the simplifier performs a final validation sweep (src/simplifier.cpp, lines 822-831). This pass guarantees that any vertex with the Lock bit set—and all its associated wedges—is definitively marked as Locked, serving as a safety net against any edge cases in the complex classification logic.

Practical Implementation Example

To use vertex locks, allocate a byte array with one entry per vertex, set the appropriate flags, and pass it to the simplification function:

size_t vertexCount = ...;
const float* positions = ...;               // float3 per vertex
const unsigned int* indices = ...;
size_t targetIndices = vertexCount * 2;     // desired output size

// Allocate a lock array – one byte per vertex
std::vector<unsigned char> vertexLock(vertexCount, 0);

// Lock the outer border (e.g., vertices 0-99) so they never move
for (size_t i = 0; i < 100; ++i)
    vertexLock[i] = meshopt_SimplifyVertex_Lock;

// Protect a seam vertex (index 250) when using permissive mode
vertexLock[250] = meshopt_SimplifyVertex_Protect;

// Give high priority to a detail vertex (index 400)
vertexLock[400] = meshopt_SimplifyVertex_Priority;

// Simplify with permissive mode and border-locking
unsigned int* outIndices = new unsigned int[targetIndices];
size_t result = meshopt_simplify(
    outIndices,
    indices, indexCount,
    positions, vertexCount, sizeof(float) * 3,
    targetIndices, 0.01f,
    meshopt_SimplifyPermissive | meshopt_SimplifyLockBorder,
    nullptr);               // result_error not needed here

Key points:

• meshopt_SimplifyLockBorder prevents hull shrinkage by locking all border vertices automatically.
• meshopt_SimplifyPermissive enables collapses across seams, but the Protect flag on vertex 250 blocks collapses that would break the seam.
• The Priority flag influences the internal collapse error metric, making vertex 400 more expensive to remove.

Summary

• Flag definitions reside in src/meshoptimizer.h (lines 86-95), providing Lock, Protect, and Priority controls.
• Classification occurs in classifyVertices (src/simplifier.cpp, lines 617-720), which propagates lock flags to all wedge duplicates.
• Border protection can be automated via meshopt_SimplifyLockBorder or manual via the Lock flag.
• Permissive mode (lines 744-758) enables aggressive simplification across seams when combined with Protect for selective blocking.
• Final enforcement (lines 822-831) ensures lock integrity before the collapse loop begins.

Frequently Asked Questions

What is the difference between Lock and Protect flags?

meshopt_SimplifyVertex_Lock unconditionally prevents a vertex from moving during any collapse operation. meshopt_SimplifyVertex_Protect allows movement only if the collapse does not break an attribute seam, which requires the meshopt_SimplifyPermissive option to be set; without permissive mode, Protect has no effect.

Can I combine multiple vertex lock flags on a single vertex?

Yes, the flags are bitwise values designed for combination. For example, vertexLock[i] = meshopt_SimplifyVertex_Lock | meshopt_SimplifyVertex_Priority; ensures the vertex remains fixed while also marking it as high-priority, which is useful for critical anchor points that must never move.

How does vertex locking affect simplification performance?

The vertex_lock array is processed in linear time during the classification phase. Locked vertices reduce the pool of valid edge collapse candidates, which can improve performance by early-terminating invalid collapse tests, though the dominant cost remains the quadric error metric calculations.

When should I use meshopt_SimplifyLockBorder versus manual vertex locks?

Use meshopt_SimplifyLockBorder when you need to preserve the entire outer silhouette of an open mesh without manually detecting boundary vertices. Use manual locks when you need to protect specific interior vertices, maintain specific UV seams, or preserve arbitrary topological features that are not classified as borders.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}