# How to Use meshopt_generateVertexRemapCustom for Floating-Point Tolerance When Merging Vertices

> Learn how to use meshopt_generateVertexRemapCustom for floating-point tolerance when merging vertices. This guide shows you how to achieve precise vertex merging with custom equality callbacks and epsilon distance comparisons.

- Repository: [Arseny Kapoulkine/meshoptimizer](https://github.com/zeux/meshoptimizer)
- Tags: how-to-guide
- Published: 2026-07-12

---

**Use `meshopt_generateVertexRemapCustom` with a custom equality callback that compares vertex positions within an epsilon distance, allowing the function to merge vertices that differ only by floating-point noise while using exact hashing for performance.**

The `meshoptimizer` library by zeux provides low-level tools for mesh optimization, including vertex deduplication. When importing meshes from external tools, floating-point precision issues often create duplicate vertices that are not exactly bitwise equal. The `meshopt_generateVertexRemapCustom` function declared in [`src/meshoptimizer.h`](https://github.com/zeux/meshoptimizer/blob/main/src/meshoptimizer.h) solves this by allowing a custom equality callback that can merge vertices within a floating-point tolerance.

## Understanding the Custom Remap API

Unlike the standard vertex remap functions, `meshopt_generateVertexRemapCustom` exposes a two-stage hashing process implemented in [`src/indexgenerator.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/indexgenerator.cpp). The function first uses `VertexCustomHasher::hash` to group vertices by the integer representation of their three 32-bit position components. This creates fast initial buckets of potentially duplicate vertices.

After a hash collision occurs, the code falls back to `VertexCustomHasher::equal`. This method first checks for exact bitwise equality of the position components. If that fails and a callback is provided, it invokes your custom function to determine if the vertices should be merged despite small floating-point differences.

The function signature from [`src/meshoptimizer.h`](https://github.com/zeux/meshoptimizer/blob/main/src/meshoptimizer.h):

```c
size_t meshopt_generateVertexRemapCustom(
    unsigned int* destination,
    const unsigned int* indices,
    size_t index_count,
    const float* vertex_positions,
    size_t vertex_count,
    size_t vertex_positions_stride,
    int (*callback)(void*, unsigned int, unsigned int),
    void* context);

```

The callback receives a user-defined context pointer and two vertex indices, returning **non-zero** if the vertices are acceptable matches.

## Implementing a Tolerance-Based Equality Check

To merge vertices within a floating-point epsilon, you must provide a context structure and a callback function that calculates the distance between candidate vertices.

### Define a Context Structure

Create a structure to hold your tolerance threshold and position data access patterns:

```c
struct ToleranceContext {
    float epsilon;           // Distance threshold, e.g., 1e-5f
    const float* positions;  // Pointer to vertex position data
    size_t stride_floats;    // Stride in floats (vertex_positions_stride / sizeof(float))
};

```

### Write the Equality Callback

Implement the callback to compute the squared Euclidean distance between two vertex positions. Return non-zero if they are within the tolerance:

```c
int position_tolerance_callback(void* ctx, unsigned int a, unsigned int b) {
    const ToleranceContext* t = (const ToleranceContext*)ctx;
    const float* pa = t->positions + a * t->stride_floats;
    const float* pb = t->positions + b * t->stride_floats;
    
    float dx = pa[0] - pb[0];
    float dy = pa[1] - pb[1];
    float dz = pa[2] - pb[2];
    float dist_sq = dx*dx + dy*dy + dz*dz;
    
    return dist_sq <= (t->epsilon * t->epsilon); // Non-zero means "equal"
}

```

### Invoke the API

Call `meshopt_generateVertexRemapCustom` with your callback and context:

```c
// Prepare context
ToleranceContext ctx;
ctx.epsilon = 1e-5f;
ctx.positions = vertex_positions;
ctx.stride_floats = vertex_positions_stride / sizeof(float);

// Allocate remap table (one entry per original vertex)
std::vector<unsigned int> remap(vertex_count);

// Generate remap
size_t unique_count = meshopt_generateVertexRemapCustom(
    remap.data(),
    indices,           // Can be nullptr if not using an index buffer
    index_count,
    vertex_positions,
    vertex_count,
    vertex_positions_stride,
    position_tolerance_callback,
    &ctx);

```

## Applying the Vertex Remap

Once the remap table is generated, use `meshopt_remapVertexBuffer` and `meshopt_remapIndexBuffer` to create compacted buffers without duplicate vertices:

```c
// Compact vertices
std::vector<Vertex> compacted_vertices(unique_count);
meshopt_remapVertexBuffer(
    compacted_vertices.data(),
    vertices,
    vertex_count,
    sizeof(Vertex),
    remap.data());

// Remap indices to point to new vertex locations
std::vector<unsigned int> remapped_indices(index_count);
meshopt_remapIndexBuffer(
    remapped_indices.data(),
    indices,
    index_count,
    remap.data());

```

## Performance Characteristics

The implementation in [`src/indexgenerator.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/indexgenerator.cpp) optimizes performance by only invoking your callback for vertices that hash to the same bucket. Since the hash is based on the raw integer bits of the position components, the tolerance test is only performed on candidates that are already close in memory representation. This makes the floating-point tolerance check cheap even on large meshes, as the expensive distance calculation is guarded by fast hash comparisons.

## Summary

- `meshopt_generateVertexRemapCustom` allows custom equality logic through a callback mechanism defined in [`src/meshoptimizer.h`](https://github.com/zeux/meshoptimizer/blob/main/src/meshoptimizer.h).
- The function uses **two-stage hashing**: first by integer representation of positions in `VertexCustomHasher::hash`, then by your custom tolerance callback in `VertexCustomHasher::equal`.
- Implement a context structure to store your epsilon value and a callback that returns non-zero for vertices within the desired distance.
- Apply the resulting remap table with `meshopt_remapVertexBuffer` and `meshopt_remapIndexBuffer` to produce optimized mesh data.

## Frequently Asked Questions

### How does the hashing work with floating-point tolerance?

The function hashes vertices based on the exact integer representation of their three 32-bit float components in `VertexCustomHasher::hash`. Only vertices that collide in this hash bucket proceed to the equality test. If the exact bitwise comparison fails, your custom callback is invoked to determine if the floating-point difference is within acceptable tolerance. This ensures the expensive distance calculation only runs on geometrically close candidates.

### Can I use this function without an index buffer?

Yes. Pass `nullptr` for the `indices` parameter and `0` for `index_count`. The function will generate a remap table for all vertices in the `vertex_positions` array based solely on position proximity within your tolerance.

### What should the callback return for vertices that should be merged?

The callback should return **non-zero** (true) if the two vertices should be considered identical and merged together. Return **zero** (false) if they represent distinct vertices that happen to have hash collisions but are outside your tolerance distance.

### Is there a performance penalty for using a custom callback?

The callback is only invoked when hash collisions occur and exact equality checks fail. Since the hash groups vertices by their integer bit patterns, the callback is rarely called for vertices that are far apart in space. According to the implementation in [`src/indexgenerator.cpp`](https://github.com/zeux/meshoptimizer/blob/main/src/indexgenerator.cpp), this makes the tolerance-based merge nearly as fast as exact matching while handling floating-point precision issues.