What Happens When a Mesh Is Smoothed or Decimated in Modly: A Deep Dive into the Optimization Pipeline

When a mesh is smoothed or decimated in Modly, the system routes the operation through a dual-pathway pipeline using pymeshlab, applying Laplacian smoothing or quadric edge-collapse decimation while preserving UV coordinates and material data via temporary OBJ or PLY intermediate files.

Modly’s optimization layer wraps the pymeshlab library to provide robust mesh processing capabilities. According to the lightningpixel/modly source code, the implementation deliberately splits processing into two distinct pathways—one for textured meshes (OBJ-MTL) and one for geometry-only meshes (PLY)—ensuring that texture maps and UV coordinates survive the transformation whenever present.

Common Preparation Steps

Before any smoothing or decimation occurs, Modly performs a standardized preparation sequence in api/routers/optimize.py.

Input Validation and Workspace Setup

The system first resolves the input path using _resolve_input_path, which guards against path traversal attacks and raises a 404 error if the source file cannot be found. It then creates a temporary working directory via tempfile.mkdtemp() to stage intermediate files for pymeshlab processing.

Geometry Loading with Trimesh

Modly loads the source mesh using trimesh.load(), which yields either a Trimesh object or a Scene. If the result is a Scene, the individual geometries are concatenated into a single mesh to ensure uniform processing. This occurs at lines 31–38 of the optimize router.

The Mesh Smoothing Pipeline

Mesh smoothing in Modly utilizes Laplacian smoothing, an algorithm that reduces high-frequency surface noise by averaging vertex positions with their neighbors.

Texture Detection and Pathway Selection

The _has_texture function (lines 90–102) inspects the loaded geometry for texture data, checking for either a simple image attribute or PBR baseColorTexture properties. If textures exist, Modly follows the OBJ route to preserve UV coordinates; otherwise, it processes the mesh as a PLY file.

Textured Mesh Pathway (OBJ)

For meshes with textures, Modly:

1. Exports the geometry, UVs, and MTL material definitions to an OBJ file
2. Writes the associated texture to texture.png
3. Patches the MTL file to reference this known texture filename
4. Loads the OBJ into a pymeshlab.MeshSet and invokes apply_coord_laplacian_smoothing(stepsmoothnum=iterations)
5. Exports the smoothed result back to OBJ, repatches the MTL, and reloads with trimesh

This pathway ensures that UV maps remain intact after smoothing, which is critical for AI-generated meshes that often exhibit "zipper triangles" or surface noise.

Geometry-Only Pathway (PLY)

For untextured meshes, the process simplifies:

1. Export the geometry to a PLY file
2. Load into pymeshlab.MeshSet and run the same Laplacian smoothing call
3. Export as PLY and reload with trimesh

Result Handling

The smoothed Trimesh object is written to a new GLB file named *_smooth{iterations}.glb in the workspace, with the endpoint returning a URL to the newly created asset.

The Mesh Decimation Process

Decimation reduces polygon count using quadric edge-collapse, a surface simplification algorithm that minimizes geometric error while collapsing edges.

Algorithm Implementation

The texture detection logic mirrors the smoothing flow. For textured meshes, Modly invokes meshing_decimation_quadric_edge_collapse with targetfacenum=target_faces and preservetexcoord=True, guaranteeing UV coordinate preservation during simplification. For geometry-only meshes, the same algorithm runs without texture preservation flags.

Processing Steps

Textured pathway:

• Export to OBJ/MTL, patch texture references
• Run quadric edge-collapse with texture preservation enabled
• Export and reload

Geometry-only pathway:

• Export to PLY
• Run quadric edge-collapse decimation
• Export as PLY and reload

The decimated mesh saves as *_opt{target_faces}.glb, with the API returning both the GLB URL and the final face count.

Safety Limits and Error Handling

Modly enforces strict operational boundaries to prevent excessive quality loss or server overload.

Iteration and Face Count Constraints

• Smoothing iterations are clamped to 1–20 using max(1, min(20, iterations)) to prevent over-smoothing that would destroy fine geometric detail
• Decimation targets must fall between 100 and 500,000 faces, ensuring the output remains usable while preventing computational timeouts

Dependency Verification

Both operations validate that pymeshlab is importable before processing. If the native DLL is missing or blocked (common in restricted environments), the system aborts with a 503 error rather than failing mid-processing.

Code Examples

Smoothing a Mesh via the REST API

import requests

payload = {
    "path": "my_collection/model.glb",
    "iterations": 10  # Clamped to 1-20

}

resp = requests.post(
    "https://modly.example.com/optimize/smooth",
    json=payload
)

print(resp.json())

# => {"url": "/workspace/my_collection/model_smooth10.glb"}

Implementation path: router.post("/smooth") → _smooth() → pymeshlab.apply_coord_laplacian_smoothing

Decimating a Mesh via the REST API

import requests

payload = {
    "path": "my_collection/highpoly.glb",
    "target_faces": 20000  # Clamped to 100-500000

}

resp = requests.post(
    "https://modly.example.com/optimize/mesh",
    json=payload
)

print(resp.json())

# => {"url": "/workspace/my_collection/highpoly_opt20000.glb", "face_count": 19987}

Implementation path: router.post("/mesh") → _decimate() → pymeshlab.meshing_decimation_quadric_edge_collapse

Using the Standalone Processor (CLI)

Modly also provides a workflow-ready processor for command-line usage:

echo '{"input":{"filePath":"my_collection/model.glb"},"params":{"iterations":8,"mode":"laplacian"}}' \
| python -m src.areas.workflows.nodes.mesh-smoother.processor

This invokes src/areas/workflows/nodes/mesh-smoother/processor.py, which implements the same smoothing logic independently of the HTTP API.

Summary

• Modly processes all mesh optimizations through pymeshlab, with logic centralized in api/routers/optimize.py
• The system automatically selects OBJ (for textured meshes) or PLY (for geometry-only) intermediates to preserve UV coordinates and material data
• Laplacian smoothing reduces surface noise through vertex position averaging, limited to 1–20 iterations
• Quadric edge-collapse decimation reduces face counts while maintaining shape integrity, constrained between 100–500,000 faces
• Both operations validate dependencies, guard against path traversal, and output standardized GLB files

Frequently Asked Questions

Does Modly preserve textures when smoothing AI-generated meshes?

Yes. When _has_texture detects texture data in api/routers/optimize.py, Modly routes the mesh through the OBJ pathway, explicitly preserving UV coordinates and material definitions. The Laplacian smoothing algorithm runs with these coordinates intact, and the system repatches MTL files after processing to maintain texture references.

What happens if I request 1,000,000 faces during decimation?

Modly enforces a hard maximum of 500,000 faces for decimation operations. If you request 1,000,000 faces in the target_faces parameter, the system clamps the value to 500,000 before invoking meshing_decimation_quadric_edge_collapse. Similarly, requests below 100 faces are raised to 100 to prevent degenerate geometry.

Why does Modly use trimesh before calling pymeshlab?

Modly uses trimesh for initial loading and scene concatenation because it handles diverse input formats (GLB, STL, OBJ) robustly. The library converts complex scenes into unified Trimesh objects, which are then exported to temporary PLY or OBJ files for pymeshlab processing. This separation allows Modly to leverage trimesh’s format flexibility while utilizing pymeshlab’s advanced filtering algorithms.

Can I run mesh smoothing if pymeshlab is not installed?

No. Both the _smooth and _decimate functions check for pymeshlab availability at runtime. If the library or its native dependencies are missing, the endpoint returns a 503 error immediately. This check prevents partial processing failures that would leave temporary files in the workspace without producing valid output.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}