How meshopt_optimizeOverdraw Balances Vertex Cache and Pixel Overdraw

meshopt_optimizeOverdraw reorders triangle indices by enforcing hard vertex-cache boundaries and then allowing a configurable threshold of cache-miss degradation to sort clusters by view-space front-facingness, simultaneously maintaining cache efficiency while minimizing pixel overdraw.

The meshoptimizer library provides meshopt_optimizeOverdraw as a specialized index buffer optimizer that reconciles the conflicting goals of vertex-cache locality and overdraw reduction. Unlike pure cache optimizers, this function implements a three-stage algorithm—defined in src/overdrawoptimizer.cpp—that explicitly trades a small, controlled amount of cache efficiency for substantially better rasterization performance through reduced fragment rewriting.

Three-Stage Optimization Process

The balancing act is achieved through a pipeline that first guarantees minimum cache performance, then strategically relaxes it to improve spatial ordering.

Enforcing Hard Cache-Miss Boundaries

The algorithm begins in generateHardBoundaries (src/overdrawoptimizer.cpp, lines 61-84) by simulating a 16-vertex post-transform cache (the default cache_size). It scans the index buffer and records every triangle where all three vertices miss the cache. These positions become hard cluster boundaries that cannot be crossed during optimization, guaranteeing a baseline level of vertex-cache friendliness regardless of subsequent reordering.

Refining with Soft Boundaries and ACMR Thresholds

Using the hard boundaries as anchors, the generateSoftBoundaries function (lines 90-124) walks each cluster to compute the Average Cache Miss Ratio (ACMR). The user-provided threshold parameter—typically set to 1.05 to permit up to 5% additional cache misses—determines where to insert soft cluster boundaries. This step strategically fragments clusters to improve spatial locality for overdraw reduction while respecting the cache-miss budget, explicitly balancing the two metrics.

Spatial Sorting to Minimize Overdraw

For each cluster, the optimizer computes a sort key in calculateSortData (lines 13-34). It derives the cluster's geometric centroid and normal, then calculates the dot product between the centroid-to-mesh-center vector and the cluster normal. This value reflects how "front-facing" the cluster is relative to the view direction. The calculateSortOrderRadix function (lines 85-115) quantizes these dot products and performs a radix sort so that clusters with higher values (more likely to be front-facing) are rendered first. This ordering reduces pixel overdraw by ensuring that triangles likely to obscure others are processed earlier in the rasterization pipeline, preventing unnecessary fragment shader invocations.

Implementation and API Usage

The public API is declared in src/meshoptimizer.h, while the core logic resides in src/overdrawoptimizer.cpp. The function accepts a threshold parameter that directly controls the ACMR-versus-overdraw trade-off.

#include "meshoptimizer.h"
#include <vector>

// Example mesh data
std::vector<unsigned int> indices   = /* triangle list */;
std::vector<float>        positions = /* XYZ float3 per vertex */;
size_t vertexCount = positions.size() / 3;

// Destination buffer (can be the same as `indices` for in‑place optimisation)
std::vector<unsigned int> optimized(indices.size());

// Threshold = 1.05 → allow up to 5 % extra cache misses for better overdraw reduction
float threshold = 1.05f;

meshopt_optimizeOverdraw(
    optimized.data(),
    indices.data(),
    indices.size(),
    positions.data(),
    vertexCount,
    sizeof(float) * 3,   // stride = 12 bytes (XYZ)
    threshold);

The same call works with raw C arrays; required parameters include the destination buffer, source index buffer, vertex positions, vertex count, stride, and the threshold controlling the cache-versus-overdraw trade-off.

Summary

• meshopt_optimizeOverdraw uses a three-stage process: hard boundaries, soft boundaries with ACMR thresholds, and spatial cluster sorting.
• Hard boundaries guarantee minimum vertex-cache efficiency by simulating a 16-vertex cache and marking complete cache misses as partition points.
• The configurable threshold allows controlled degradation of cache performance (e.g., 5% extra misses) to create additional cluster boundaries that improve spatial locality.
• Cluster sorting by front-facingness reduces pixel overdraw during rasterization by rendering likely-visible geometry first.

Frequently Asked Questions

What is the default cache size used by meshopt_optimizeOverdraw?

The optimizer simulates a 16-vertex post-transform cache by default when generating hard boundaries in generateHardBoundaries, establishing the minimum baseline for vertex-cache efficiency.

How does the threshold parameter affect optimization results?

The threshold controls the maximum allowable increase in the Average Cache Miss Ratio (ACMR). A value of 1.05 allows up to 5% more cache misses than an optimal cache-only ordering, creating additional cluster boundaries that improve spatial locality and reduce overdraw at the cost of slightly worse vertex reuse.

Why sort clusters by front-facingness?

Sorting clusters so that those with higher dot products between their normal and the view direction are rendered first ensures that triangles likely to be visible obscure subsequent triangles. This early-z optimization prevents the GPU from executing fragment shaders for pixels that will be overwritten, directly reducing pixel overdraw.

Can meshopt_optimizeOverdraw be used with any mesh topology?

The function operates on standard triangle index buffers and requires only vertex position data, making it compatible with any triangle mesh regardless of topology, though it delivers the greatest benefit on complex meshes with significant depth complexity where overdraw is a bottleneck.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}