# How TRELLIS.2 Handles Mesh Postprocessing: UV Unwrapping and Texture Baking with nvdiffrast

> Discover how TRELLIS.2 uses nvdiffrast for efficient mesh postprocessing. Learn about UV unwrapping and texture baking with CUDA acceleration for PBR textures.

- Repository: [Microsoft/TRELLIS.2](https://github.com/microsoft/TRELLIS.2)
- Tags: how-to-guide
- Published: 2026-08-03

---

**The TRELLIS.2 pipeline cleans raw voxel-extracted geometry, performs cone-angle clustering for UV parameterization, then uses nvdiffrast's CUDA-accelerated rasterizer to bake volumetric attributes into PBR textures by back-projecting through a BVH.**

The Microsoft TRELLIS.2 repository implements a production-ready **mesh postprocessing** workflow that bridges neural reconstruction and real-time rendering. Located in [`o-voxel/o_voxel/postprocess.py`](https://github.com/microsoft/TRELLIS.2/blob/main/o-voxel/o_voxel/postprocess.py), the pipeline transforms coarse voxel outputs into optimized, textured meshes suitable for GLB export. Understanding this implementation is essential for developers integrating differentiable rasterization into 3D generation pipelines.

## The Three-Phase Postprocessing Pipeline

The `to_glb` function orchestrates a complete transformation from raw geometry to textured assets through three distinct phases.

### Phase 1: Mesh Cleaning and Optional Remeshing

The workflow begins with aggressive geometry repair to ensure manifold, watertight surfaces suitable for UV unwrapping. According to lines 9-84 of [`postprocess.py`](https://github.com/microsoft/TRELLIS.2/blob/main/postprocess.py), the system executes:

- **Hole filling** via `mesh.fill_holes` to close small openings
- **Duplicate removal** using `mesh.remove_duplicate_faces`
- **Non-manifold repair** through `mesh.repair_non_manifold_edges`

If remeshing is enabled, the pipeline calls `cumesh.remeshing.remesh_narrow_band_dc` to perform dual-contouring-based remeshing. The system then applies `mesh.simplify` progressively—first to a multiple of the target count, then to the exact `decimation_target` (e.g., 50,000 vertices)—to balance geometry fidelity with UV layout quality.

### Phase 2: UV Parameterization via Cone-Angle Clustering

Once cleaned, the mesh enters UV layout generation through `cumesh.CuMesh.uv_unwrap` (lines 96-109). This implementation constructs per-face charts using a **cone-angle clustering algorithm** that minimizes angular distortion across UV islands.

The function exposes fine-grained control through parameters:
- `threshold_cone_half_angle_rad`: Controls chart boundary sensitivity
- `refine_iterations` and `global_iterations`: Determine smoothing passes
- `smooth_strength`: Adjusts distortion penalty during optimization

Setting `return_vmaps=True` yields UV coordinates alongside a vertex-to-chart map (`vmap`) that tracks chart membership for subsequent texture packing.

### Phase 3: Differentiable Texture Baking with nvdiffrast

Texture baking leverages **nvdiffrast** (`dr.RasterizeCudaContext`) to perform CUDA-accelerated differentiable rasterization directly in UV space. The process spans lines 124-166 of [`postprocess.py`](https://github.com/microsoft/TRELLIS.2/blob/main/postprocess.py) and operates as follows:

1. **Rasterization**: `dr.rasterize` (lines 124-141) renders the mesh at the specified `texture_size` (e.g., 2048×2048), outputting per-pixel face IDs in the alpha channel and a binary mask of valid texels.

2. **Position interpolation**: `dr.interpolate` (lines 148-150) reconstructs world-space positions for each texel by interpolating the simplified mesh vertices.

3. **BVH back-projection**: To preserve detail lost during decimation, the pipeline queries the original high-resolution geometry using `bvh.unsigned_distance` (lines 154-156), mapping interpolated positions back to the source mesh.

4. **Volumetric sampling**: `grid_sample_3d` (lines 158-166) queries the attribute volume to sample **base-color**, **metallic**, **roughness**, and **alpha** channels at the back-projected coordinates.

The resulting tensors undergo post-processing (inpainting and clipping) before instantiation as a `trimesh.visual.material.PBRMaterial` ready for export.

## Implementation Walkthrough

### Running the Full Pipeline

To process a raw extraction into a textured GLB asset, use the `to_glb` entry point with properly structured inputs:

```python
import torch
from o_voxel.o_voxel.postprocess import to_glb
from trellis2.representations.mesh import MeshWithVoxel

# Initialize raw geometry and voxel attributes

vertices = torch.randn(200_000, 3)  # (N, 3)

faces = torch.randint(0, 200_000, (400_000, 3))  # (M, 3)

attr_volume = torch.randn(64, 64, 64, 8)  # (D, H, W, C)

# Create coordinate grid for volumetric sampling

coords = torch.stack(torch.meshgrid(
    torch.arange(64), torch.arange(64), torch.arange(64), indexing='ij'), -1).float()

# Define attribute layout (channels 0-3: base_color, 3-4: metallic, etc.)

attr_layout = {
    'base_color': slice(0, 3),
    'metallic': slice(3, 4),
    'roughness': slice(4, 5),
    'alpha': slice(5, 6),
}
aabb = torch.tensor([[0., 0., 0.], [1., 1., 1.]])

# Execute postprocessing pipeline

textured_mesh = to_glb(
    vertices,
    faces,
    attr_volume,
    coords,
    attr_layout,
    aabb,
    voxel_size=0.01,
    decimation_target=50_000,  # Final vertex count

    texture_size=2048,
    remesh=False,  # Skip dual-contouring remeshing

    verbose=True,
)

# Export to GLB or use with PbrMeshRenderer

textured_mesh.export('output_mesh.glb')

```

### Manual Rasterization for Custom Workflows

For developers requiring custom sampling logic, the nvdiffrast pipeline can be invoked directly after UV unwrapping:

```python
import nvdiffrast.torch as dr
import torch

# Obtain UVs from previous processing

out_vertices, out_faces, out_uvs, out_vmaps = mesh.uv_unwrap(
    threshold_cone_half_angle_rad=0.5,
    refine_iterations=5,
    global_iterations=5,
    smooth_strength=0.5,
    return_vmaps=True
)

# Initialize CUDA rasterizer context

ctx = dr.RasterizeCudaContext()

# Map UVs [0,1] to NDC [-1,1] for nvdiffrast

uvs_ndc = torch.cat([
    out_uvs * 2 - 1,
    torch.zeros_like(out_uvs[:, :1]),
    torch.ones_like(out_uvs[:, :1])
], dim=-1).unsqueeze(0)

# Rasterize at desired texture resolution

rast, _ = dr.rasterize(ctx, uvs_ndc, out_faces, resolution=[2048, 2048])

# Interpolate positions for back-projection

pos = dr.interpolate(out_vertices.unsqueeze(0), rast, out_faces)[0][0]

# Proceed with BVH queries and grid_sample_3d as in the full pipeline...

```

## Key Source Files

| File Path | Description |
|-----------|-------------|
| [`o-voxel/o_voxel/postprocess.py`](https://github.com/microsoft/TRELLIS.2/blob/main/o-voxel/o_voxel/postprocess.py) | Implements `to_glb`, mesh cleaning, `mesh.uv_unwrap` integration, and nvdiffrast-based texture baking |
| [`trellis2/renderers/pbr_mesh_renderer.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/renderers/pbr_mesh_renderer.py) | Consumes baked PBR materials; demonstrates nvdiffrast context initialization |
| [`trellis2/representations/mesh.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/representations/mesh.py) | Defines `MeshWithVoxel` and `MeshWithPbrMaterial` data structures |
| [`trellis2/utils/mesh_utils.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/utils/mesh_utils.py) | Helper utilities for PLY I/O used during pipeline execution |

## Summary

- The TRELLIS.2 postprocessing pipeline follows a strict three-phase sequence: **mesh cleaning**, **UV parameterization via cone-angle clustering**, and **differentiable texture baking** with nvdiffrast.
- **Cone-angle clustering** in `mesh.uv_unwrap` minimizes distortion while exposing tunable parameters for chart generation quality.
- **BVH back-projection** ensures texture fidelity by mapping decimated mesh texels back to the original high-resolution geometry before volumetric sampling.
- **nvdiffrast** provides the CUDA-accelerated backend for `dr.rasterize` and `dr.interpolate`, enabling high-resolution texture generation (e.g., 2048×2048) in real-time workflows.
- Final output is a `trimesh` object with a populated `PBRMaterial` compatible with GLB export and the `PbrMeshRenderer`.

## Frequently Asked Questions

### What is cone-angle clustering in UV unwrapping?

Cone-angle clustering is a geometry processing technique that groups mesh faces into UV charts based on angular deviation metrics (cone angles). In TRELLIS.2, the `uv_unwrap` function uses this algorithm to minimize distortion across UV islands, controlled by parameters like `threshold_cone_half_angle_rad` and `refine_iterations` that determine how aggressively the optimizer merges adjacent faces.

### Why does the pipeline use BVH back-projection during texture baking?

Because the mesh undergoes decimation before unwrapping, the pipeline uses `bvh.unsigned_distance` to map rasterized positions from the simplified geometry back to the original high-resolution mesh. This ensures that texture samples align precisely with the source voxel attributes, preserving fine details that would otherwise be lost in the reduced polygon count.

### How does nvdiffrast improve texture baking performance?

nvdiffrast provides a CUDA-accelerated differentiable rasterization backend (`dr.RasterizeCudaContext`) that operates directly on GPU tensors. This allows the pipeline to rasterize UV coordinates and interpolate vertex attributes at high resolutions efficiently, making the baking process suitable for interactive applications and high-fidelity asset generation.

### Can I customize the texture resolution and vertex count targets?

Yes. The `to_glb` function accepts `texture_size` (e.g., 1024, 2048, 4096) and `decimation_target` (vertex count) parameters to control output fidelity. Additionally, UV layout quality can be fine-tuned via `smooth_strength`, `global_iterations`, and `threshold_cone_half_angle_rad` arguments passed to the unwrap function.