How TRELLIS.2 Handles Mesh Postprocessing: UV Unwrapping and Texture Baking with nvdiffrast

The TRELLIS.2 pipeline cleans raw voxel-extracted geometry, performs cone-angle clustering for UV parameterization, then uses nvdiffrast's CUDA-accelerated rasterizer to bake volumetric attributes into PBR textures by back-projecting through a BVH.

The Microsoft TRELLIS.2 repository implements a production-ready mesh postprocessing workflow that bridges neural reconstruction and real-time rendering. Located in o-voxel/o_voxel/postprocess.py, the pipeline transforms coarse voxel outputs into optimized, textured meshes suitable for GLB export. Understanding this implementation is essential for developers integrating differentiable rasterization into 3D generation pipelines.

The Three-Phase Postprocessing Pipeline

The to_glb function orchestrates a complete transformation from raw geometry to textured assets through three distinct phases.

Phase 1: Mesh Cleaning and Optional Remeshing

The workflow begins with aggressive geometry repair to ensure manifold, watertight surfaces suitable for UV unwrapping. According to lines 9-84 of postprocess.py, the system executes:

• Hole filling via mesh.fill_holes to close small openings
• Duplicate removal using mesh.remove_duplicate_faces
• Non-manifold repair through mesh.repair_non_manifold_edges

If remeshing is enabled, the pipeline calls cumesh.remeshing.remesh_narrow_band_dc to perform dual-contouring-based remeshing. The system then applies mesh.simplify progressively—first to a multiple of the target count, then to the exact decimation_target (e.g., 50,000 vertices)—to balance geometry fidelity with UV layout quality.

Phase 2: UV Parameterization via Cone-Angle Clustering

Once cleaned, the mesh enters UV layout generation through cumesh.CuMesh.uv_unwrap (lines 96-109). This implementation constructs per-face charts using a cone-angle clustering algorithm that minimizes angular distortion across UV islands.

The function exposes fine-grained control through parameters:

• threshold_cone_half_angle_rad: Controls chart boundary sensitivity
• refine_iterations and global_iterations: Determine smoothing passes
• smooth_strength: Adjusts distortion penalty during optimization

Setting return_vmaps=True yields UV coordinates alongside a vertex-to-chart map (vmap) that tracks chart membership for subsequent texture packing.

Phase 3: Differentiable Texture Baking with nvdiffrast

Texture baking leverages nvdiffrast (dr.RasterizeCudaContext) to perform CUDA-accelerated differentiable rasterization directly in UV space. The process spans lines 124-166 of postprocess.py and operates as follows:

1. Rasterization: dr.rasterize (lines 124-141) renders the mesh at the specified texture_size (e.g., 2048×2048), outputting per-pixel face IDs in the alpha channel and a binary mask of valid texels.

2. Position interpolation: dr.interpolate (lines 148-150) reconstructs world-space positions for each texel by interpolating the simplified mesh vertices.

3. BVH back-projection: To preserve detail lost during decimation, the pipeline queries the original high-resolution geometry using bvh.unsigned_distance (lines 154-156), mapping interpolated positions back to the source mesh.

4. Volumetric sampling: grid_sample_3d (lines 158-166) queries the attribute volume to sample base-color, metallic, roughness, and alpha channels at the back-projected coordinates.

The resulting tensors undergo post-processing (inpainting and clipping) before instantiation as a trimesh.visual.material.PBRMaterial ready for export.

Implementation Walkthrough

Running the Full Pipeline

To process a raw extraction into a textured GLB asset, use the to_glb entry point with properly structured inputs:

import torch
from o_voxel.o_voxel.postprocess import to_glb
from trellis2.representations.mesh import MeshWithVoxel

# Initialize raw geometry and voxel attributes

vertices = torch.randn(200_000, 3)  # (N, 3)

faces = torch.randint(0, 200_000, (400_000, 3))  # (M, 3)

attr_volume = torch.randn(64, 64, 64, 8)  # (D, H, W, C)

# Create coordinate grid for volumetric sampling

coords = torch.stack(torch.meshgrid(
    torch.arange(64), torch.arange(64), torch.arange(64), indexing='ij'), -1).float()

# Define attribute layout (channels 0-3: base_color, 3-4: metallic, etc.)

attr_layout = {
    'base_color': slice(0, 3),
    'metallic': slice(3, 4),
    'roughness': slice(4, 5),
    'alpha': slice(5, 6),
}
aabb = torch.tensor([[0., 0., 0.], [1., 1., 1.]])

# Execute postprocessing pipeline

textured_mesh = to_glb(
    vertices,
    faces,
    attr_volume,
    coords,
    attr_layout,
    aabb,
    voxel_size=0.01,
    decimation_target=50_000,  # Final vertex count

    texture_size=2048,
    remesh=False,  # Skip dual-contouring remeshing

    verbose=True,
)

# Export to GLB or use with PbrMeshRenderer

textured_mesh.export('output_mesh.glb')

Manual Rasterization for Custom Workflows

For developers requiring custom sampling logic, the nvdiffrast pipeline can be invoked directly after UV unwrapping:

import nvdiffrast.torch as dr
import torch

# Obtain UVs from previous processing

out_vertices, out_faces, out_uvs, out_vmaps = mesh.uv_unwrap(
    threshold_cone_half_angle_rad=0.5,
    refine_iterations=5,
    global_iterations=5,
    smooth_strength=0.5,
    return_vmaps=True
)

# Initialize CUDA rasterizer context

ctx = dr.RasterizeCudaContext()

# Map UVs [0,1] to NDC [-1,1] for nvdiffrast

uvs_ndc = torch.cat([
    out_uvs * 2 - 1,
    torch.zeros_like(out_uvs[:, :1]),
    torch.ones_like(out_uvs[:, :1])
], dim=-1).unsqueeze(0)

# Rasterize at desired texture resolution

rast, _ = dr.rasterize(ctx, uvs_ndc, out_faces, resolution=[2048, 2048])

# Interpolate positions for back-projection

pos = dr.interpolate(out_vertices.unsqueeze(0), rast, out_faces)[0][0]

# Proceed with BVH queries and grid_sample_3d as in the full pipeline...

Key Source Files

File Path	Description
o-voxel/o_voxel/postprocess.py	Implements to_glb, mesh cleaning, mesh.uv_unwrap integration, and nvdiffrast-based texture baking
trellis2/renderers/pbr_mesh_renderer.py	Consumes baked PBR materials; demonstrates nvdiffrast context initialization
trellis2/representations/mesh.py	Defines MeshWithVoxel and MeshWithPbrMaterial data structures
trellis2/utils/mesh_utils.py	Helper utilities for PLY I/O used during pipeline execution

Summary

• The TRELLIS.2 postprocessing pipeline follows a strict three-phase sequence: mesh cleaning, UV parameterization via cone-angle clustering, and differentiable texture baking with nvdiffrast.
• Cone-angle clustering in mesh.uv_unwrap minimizes distortion while exposing tunable parameters for chart generation quality.
• BVH back-projection ensures texture fidelity by mapping decimated mesh texels back to the original high-resolution geometry before volumetric sampling.
• nvdiffrast provides the CUDA-accelerated backend for dr.rasterize and dr.interpolate, enabling high-resolution texture generation (e.g., 2048×2048) in real-time workflows.
• Final output is a trimesh object with a populated PBRMaterial compatible with GLB export and the PbrMeshRenderer.

Frequently Asked Questions

What is cone-angle clustering in UV unwrapping?

Cone-angle clustering is a geometry processing technique that groups mesh faces into UV charts based on angular deviation metrics (cone angles). In TRELLIS.2, the uv_unwrap function uses this algorithm to minimize distortion across UV islands, controlled by parameters like threshold_cone_half_angle_rad and refine_iterations that determine how aggressively the optimizer merges adjacent faces.

Why does the pipeline use BVH back-projection during texture baking?

Because the mesh undergoes decimation before unwrapping, the pipeline uses bvh.unsigned_distance to map rasterized positions from the simplified geometry back to the original high-resolution mesh. This ensures that texture samples align precisely with the source voxel attributes, preserving fine details that would otherwise be lost in the reduced polygon count.

How does nvdiffrast improve texture baking performance?

nvdiffrast provides a CUDA-accelerated differentiable rasterization backend (dr.RasterizeCudaContext) that operates directly on GPU tensors. This allows the pipeline to rasterize UV coordinates and interpolate vertex attributes at high resolutions efficiently, making the baking process suitable for interactive applications and high-fidelity asset generation.

Can I customize the texture resolution and vertex count targets?

Yes. The to_glb function accepts texture_size (e.g., 1024, 2048, 4096) and decimation_target (vertex count) parameters to control output fidelity. Additionally, UV layout quality can be fine-tuned via smooth_strength, global_iterations, and threshold_cone_half_angle_rad arguments passed to the unwrap function.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}