Handling Large Mesh Datasets in ArmorPaint: Merging and Optimization Techniques

ArmorPaint handles large mesh datasets by automatically merging multiple mesh objects into a single consolidated GPU-friendly structure when Ray-trace Multi mode is enabled, reducing draw calls and maintaining interactive painting performance.

Managing extensive geometry in a real-time painting environment requires careful memory management and rendering optimization. In the armory3d/armorpaint repository, the engine adopts a hybrid approach that keeps individual mesh objects separate for typical workloads while switching to a merged-mesh representation for complex scenes. This strategy ensures that handling large mesh datasets in ArmorPaint remains performant without sacrificing the flexibility of per-object editing.

Mesh Data Architecture in ArmorPaint

ArmorPaint represents every model as a collection of mesh objects (mesh_object_t). Each object owns a mesh data structure that contains the vertex arrays, index array, scaling information, and a runtime handle for GPU resources.

The Core mesh_data_t Structure

The foundation of ArmorPaint's geometry handling resides in base/sources/engine.h. The mesh_data_t struct defines how vertex information is stored and prepared for GPU upload:

// engine.h – definition of mesh_data_t
typedef struct mesh_data {
    char                   *name;
    float                   scale_pos;
    float                   scale_tex;
    vertex_array_t_array_t *vertex_arrays;
    u32_array_t            *index_array;
    mesh_data_runtime_t    *_;            // ← GPU buffers, ownership flag
} mesh_data_t;

This structure maintains CPU-side vertex arrays that are later uploaded to GPU buffers through the mesh_data_runtime_t handle. The separation between CPU data and GPU runtime data allows the engine to rebuild or merge geometry without disrupting rendering resources.

Detecting When to Merge Large Datasets

The engine dynamically decides whether to merge meshes based on user configuration and scene complexity. This detection mechanism prevents unnecessary overhead for simple scenes while activating optimization paths for heavy workloads.

Ray-trace Multi Mode and Merge Triggers

The Ray-trace Multi mode toggle (config_is_raytrace_multi()) serves as the primary control for automatic mesh consolidation. When enabled, ArmorPaint automatically merges multiple mesh objects into a single larger mesh to accelerate ray-traced painting operations.

In paint/sources/util/util_mesh.c, the merge entry point evaluates this configuration:

// util_mesh.c – merge entry point
if (config_is_raytrace_multi()) {
    paint_objects = util_mesh_get_unique();   // dedupe first
}
...
util_mesh_merge(config_is_raytrace_multi() ? NULL : util_mesh_get_visible());

The util_mesh_get_unique() function deduplicates mesh data before merging, ensuring that shared geometry instances do not consume redundant memory in the consolidated buffer.

Building Consolidated Mesh Data

The util_mesh_build_merged_data() function orchestrates the construction of unified vertex buffers. This process involves accumulating geometry from multiple sources, normalizing coordinate spaces, and packing texture coordinates.

Buffer Allocation and Vertex Processing

The merging routine first calculates the total vertex and index counts across all source meshes. It then allocates temporary buffers sized for the sum of all meshes:

• Position data (va0) stored as 16-bit integers
• Normal vectors (va1)
• UV coordinates (va2)
• Optional tex-1 and color arrays

Vertex positions are rescaled to a common world scale (max_scale) during the copy operation. This normalization ensures that the merged mesh fits into a single coordinate space without precision loss.

Atlas Packing for UV Coordinates

When Ray-trace Multi mode is active, the engine packs UV coordinates into an atlas to maintain texture coherence across the merged geometry. The _util_mesh_atlas_build_slots() function creates a fixed-size atlas with ATLAS_MAX_SLOTS = 64 entries.

The atlas stride calculation (util_mesh_atlas_stride_merged) determines how texture coordinates are offset within the packed layout. If the number of unique meshes exceeds the slot limit, the system gracefully degrades to a single-tile fallback rather than failing or consuming excessive memory.

GPU Resource Creation and Memory Management

Once the consolidated mesh_data_t structure is built, util_mesh_merge() instantiates the corresponding GPU object:

mesh_data_t *raw = util_mesh_build_merged_data(paint_objects, g_context->paint_object->base->name);
mesh_data_t *md   = mesh_data_create(raw);
md->_->owns_arrays = true;

g_context->merged_object = mesh_object_create(md, paint_material);
g_context->merged_object->base->name = string("%s_merged", g_context->paint_object->base->name);

The merged object is attached to the main scene context, and the ray-trace path is marked dirty to trigger acceleration structure rebuilds. The owns_arrays flag ensures proper memory ownership semantics for the newly allocated vertex buffers.

Shared Data Handling and Deduplication

Large scenes frequently reuse identical geometry across multiple objects. ArmorPaint tracks shared data through util_mesh_data_is_shared(). When operations require unique geometry copies—such as sculpting modifications—the util_mesh_unshare_data() function performs a deep copy via util_mesh_data_duplicate().

This duplication routine creates independent copies of vertex arrays and index arrays, ensuring that destructive edits to one object do not propagate to instances sharing the same underlying data.

Memory Safety Limits for Massive Scenes

The merge logic deliberately constrains resource consumption through explicit limits. The ATLAS_MAX_STRIDE = 8 and ATLAS_MAX_SLOTS = 64 constants bound the atlas size regardless of input complexity.

If util_mesh_atlas_slots_spent returns true, indicating exhaustion of the 64 available slots, the stride clamps to 1. This fallback forces the atlas to use a single tile, protecting the engine from allocating excessively large intermediate buffers while still permitting the merge operation to complete.

In practice, ArmorPaint can handle meshes containing millions of vertices provided the combined count fits within dynamically allocated temporary buffers. The merge step dramatically reduces draw-call overhead, which typically represents the primary bottleneck for large scenes.

Practical Implementation Examples

Re-importing Large Mesh Files

When working with heavy OBJ or FBX files, you may need to force a refresh of the merged representation:

// Trigger re-import from disk
project_reimport_mesh();

// Force merged mesh recomputation
util_mesh_merge(NULL);  // NULL lets function decide based on Ray-trace Multi

The project_reimport_mesh() function is bound to UI shortcuts in util_shortcut.c, providing a direct path for reloading large datasets without restarting the application.

Manual Merging of Selected Objects

For selective optimization of specific scene elements:

// Create array of specific objects to merge
mesh_object_t_array_t *selection = any_array_create_from_raw(
    (void *[]){ g_project->_->paint_objects->buffer[0],
                g_project->_->paint_objects->buffer[5] }, 2);

// Build merged mesh for selection only
util_mesh_merge(selection);

This approach allows artists to consolidate background geometry while keeping foreground elements editable as separate objects.

Ensuring Unique Data for Sculpting Operations

Before modifying mesh topology, verify data ownership to prevent unintended side effects on instances:

mesh_object_t *obj = g_context->paint_object;
if (util_mesh_data_is_shared(obj->data)) {
    util_mesh_unshare_data(obj);   // Creates private copy
}

This guard clause in sculpting workflows prevents corruption of shared assets by cloning the underlying mesh_data_t structure when necessary.

Summary

• ArmorPaint stores geometry in mesh_data_t structures defined in base/sources/engine.h, separating CPU vertex arrays from GPU runtime handles.
• The Ray-trace Multi mode triggers automatic mesh merging via util_mesh_merge() when handling large mesh datasets that would otherwise bottleneck rendering.
• The merged mesh builder allocates temporary buffers sized to the sum of all source vertices, rescales positions to a common world space, and packs UVs into a 64-slot atlas.
• Memory safety limits (ATLAS_MAX_SLOTS = 64, ATLAS_MAX_STRIDE = 8) prevent excessive resource consumption, falling back to single-tile atlases when limits are exceeded.
• Shared data detection (util_mesh_data_is_shared()) and unsharing mechanisms protect instanced geometry during destructive editing operations.
• Artists can manually trigger merges for subsets of objects or force re-imports of large files through the utility functions exposed in paint/sources/util/util_mesh.c.

Frequently Asked Questions

How does ArmorPaint handle millions of vertices?

ArmorPaint allocates temporary buffers dynamically based on the total vertex count of meshes being merged. As implemented in util_mesh_build_merged_data(), the engine sums vertex lengths (vlen) and index lengths (ilen) across all source objects, then creates 16-bit integer arrays for positions, normals, and UVs sized to these totals. This allows the engine to process millions of vertices provided sufficient system memory exists, while the merged representation reduces draw calls from many small batches to a single GPU submission.

What is Ray-trace Multi mode?

Ray-trace Multi mode is a configuration flag (config_is_raytrace_multi()) that enables automatic mesh consolidation for ray-traced painting operations. When active, the engine deduplicates visible mesh objects using util_mesh_get_unique() and merges them into a single GPU mesh via util_mesh_merge(). This mode is essential for maintaining interactive performance when painting across multiple disconnected mesh objects, as it eliminates the per-object overhead that would otherwise accumulate in the ray-tracing acceleration structure.

How does the atlas system prevent memory overflow?

The atlas system enforces hard limits defined by ATLAS_MAX_SLOTS (64) and ATLAS_MAX_STRIDE (8). When _util_mesh_atlas_build_slots() detects that more than 64 unique texture spaces are required, the util_mesh_atlas_slots_spent flag triggers a fallback mode. In this mode, the atlas stride clamps to 1, effectively collapsing the atlas to a single tile rather than allocating unbounded memory. This protective measure ensures that merging operations complete successfully even when scene complexity exceeds the optimal atlas configuration.

Can I merge only specific objects instead of the entire scene?

Yes. While the automatic path passes NULL to util_mesh_merge() to trigger visibility-based selection, you can manually construct a mesh_object_t_array_t containing specific pointers from g_project->_->paint_objects and pass this array directly to util_mesh_merge(). This selective merging allows you to optimize rendering performance for background elements while preserving individual object editability for foreground assets that require per-object manipulation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}