# How to Generate PBR Textures for Existing 3D Meshes Using Trellis2TexturingPipeline

> Generate PBR textures for 3D models with Trellis2TexturingPipeline. Convert meshes and images into base-color, metallic, roughness, and alpha maps efficiently.

- Repository: [Microsoft/TRELLIS.2](https://github.com/microsoft/TRELLIS.2)
- Tags: how-to-guide
- Published: 2026-08-04

---

**The `Trellis2TexturingPipeline` is a high-level wrapper that converts any input mesh and reference image into a fully textured PBR-ready model by encoding geometry into sparse latent representations, sampling texture latents via a flow-based diffusion model, and rasterizing the results into base-color, metallic, roughness, and alpha maps.**

The microsoft/TRELLIS.2 repository provides this production-ready implementation for automated texture synthesis. It bridges the gap between raw untextured geometry and production-ready assets through a modular ten-stage pipeline defined in [`trellis2/pipelines/trellis2_texturing.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_texturing.py).

## Pipeline Architecture Overview

The `Trellis2TexturingPipeline` orchestrates ten distinct stages, each implemented as a specific method with precise line references in the source code.

**1. Model Initialization**
The `from_pretrained` method loads model weights, sampler configurations, and normalization statistics from the Hugging Face repository `microsoft/TRELLIS.2-4B`.

**2. Device Allocation**
The `to` method (lines 98-105) manages device placement. In low-VRAM mode, heavy sub-modules transfer to CUDA only when needed and return to CPU immediately after computation.

**3. Mesh Preprocessing**
The `preprocess_mesh` method (lines 106-120) centers the geometry, normalizes its extent to ±0.5, and swaps Y/Z axes to ensure GLB compatibility.

**4. Image Preprocessing**
The `preprocess_image` method (lines 122-158) rescales input, removes backgrounds using a Rembg model (when alpha is absent), crops to the tightest foreground bounding box, and premultiplies alpha values.

**5. Conditioning Construction**
The `get_cond` method (lines 159-181) extracts image features via a conditioning encoder and prepares negative-conditioning tensors for classifier-free guidance.

**6. Shape Latent Encoding**
The `encode_shape_slat` method (lines 182-226) converts the mesh into a flexible dual-grid representation, builds a `SparseTensor`, and processes it through the shape-slat encoder.

**7. Texture Latent Sampling**
The `sample_tex_slat` method (lines 227-267) normalizes the shape latent, adds Gaussian noise, and samples texture representations using the configured flow model sampler.

**8. Texture Decoding**
The `decode_tex_slat` method (lines 268-286) passes sampled latents through the texture-slat decoder to produce voxel grids storing PBR attributes.

**9. Rasterization and Packing**
The `postprocess_mesh` method (lines 287-371) projects voxel grids onto mesh UVs using `nvdiffrast`, fills missing texels via OpenCV inpainting, and constructs a `PBRMaterial` object.

**10. Output Generation**
The `run` method (lines 374-408) returns a `trimesh.Trimesh` object complete with UV coordinates and a `TextureVisuals` instance ready for export.

## Step-by-Step Implementation

### Loading the Pretrained Pipeline

Instantiate the pipeline from the official repository and move it to your target device:

```python
import torch
from trellis2.pipelines import Trellis2TexturingPipeline

pipeline = Trellis2TexturingPipeline.from_pretrained(
    "microsoft/TRELLIS.2-4B",
    config_file="texturing_pipeline.json"
)

# For GPU acceleration

pipeline.cuda()

# For CPU or low-VRAM scenarios

# pipeline.to(torch.device("cpu"))

```

### Preprocessing Input Assets

Load existing geometry and reference imagery using standard libraries. The pipeline accepts any format supported by trimesh (PLY, OBJ, GLB):

```python
import trimesh
from PIL import Image

mesh_path = "assets/example_texturing/the_forgotten_knight.ply"
image_path = "assets/example_texturing/image.webp"

mesh = trimesh.load(mesh_path)
image = Image.open(image_path)

```

### Executing the Texturing Pipeline

Invoke the `run` method with control parameters for reproducibility and quality:

```python
textured_mesh = pipeline.run(
    mesh,
    image,
    seed=123,
    tex_slat_sampler_params={"num_steps": 50},
    preprocess_image=True,
    resolution=1024,
    texture_size=2048
)

```

Key parameters include:
- **seed**: Ensures reproducible stochastic sampling
- **tex_slat_sampler_params**: Configures diffusion steps and sampling behavior
- **resolution**: Controls internal voxel grid size (512 or 1024)
- **texture_size**: Sets final output texture dimensions (e.g., 2048×2048)

### Exporting Textured Meshes

Save the result with embedded WebP-compressed textures:

```python
output_path = "textured_output.glb"
textured_mesh.export(output_path, extension_webp=True)

```

## Complete Implementation Example

The following end-to-end script mirrors the official [`example_texturing.py`](https://github.com/microsoft/TRELLIS.2/blob/main/example_texturing.py) while demonstrating all configurable options:

```python
import os
import torch
import trimesh
from PIL import Image
from trellis2.pipelines import Trellis2TexturingPipeline

# -------------------------------------------------

# 1️⃣ Load the pretrained pipeline

# -------------------------------------------------

pipeline = Trellis2TexturingPipeline.from_pretrained(
    "microsoft/TRELLIS.2-4B",
    config_file="texturing_pipeline.json"
)

# Choose device (GPU for speed, CPU for low‑VRAM)

pipeline.cuda()  # or pipeline.to(torch.device("cpu"))

# -------------------------------------------------

# 2️⃣ Load your mesh and reference image

# -------------------------------------------------

mesh_path = "assets/example_texturing/the_forgotten_knight.ply"
image_path = "assets/example_texturing/image.webp"

mesh = trimesh.load(mesh_path)
image = Image.open(image_path)

# -------------------------------------------------

# 3️⃣ Run the pipeline

# -------------------------------------------------

textured_mesh = pipeline.run(
    mesh,
    image,
    seed=123,
    tex_slat_sampler_params={"num_steps": 50},
    preprocess_image=True,
    resolution=1024,
    texture_size=2048
)

# -------------------------------------------------

# 4️⃣ Export the textured mesh

# -------------------------------------------------

output_path = "textured_output.glb"
textured_mesh.export(output_path, extension_webp=True)
print(f"Saved textured mesh to {output_path}")

```

## Technical Deep Dive

### Dual-Grid and Sparse Tensor Representation

The pipeline converts input meshes using `o_voxel.convert.mesh_to_flexible_dual_grid` to create sparse representations where each voxel stores dual vertices capturing geometric detail. These feed into `SparseTensor` objects defined in `trellis2/modules/sparse`, enabling efficient sparse convolutions throughout the encoding stages.

### Flow-Based Sampling and Conditioning

Texture generation employs a normalizing-flow model (`tex_slat_flow_model_*`) integrated with the `samplers.Sampler` class. The `get_cond` method concatenates image and shape-latent conditioning vectors, then performs stochastic denoising to produce coherent PBR patterns.

### Rasterization and Inpainting

The final projection uses `nvdiffrast` for UV rasterization, with `flex_gemm.ops.grid_sample.grid_sample_3d` handling trilinear interpolation from 3D voxel space to 2D texture coordinates. OpenCV inpainting repairs missing texels before the `postprocess_mesh` method packs channels into the final `PBRMaterial`.

## Memory Optimization Strategies

When `low_vram=True` (the default), the pipeline keeps heavy modules such as flow models and image encoders on CPU, transferring them to GPU only during active computation. This selective device migration occurs automatically within each pipeline stage, keeping VRAM consumption manageable while processing high-resolution textures.

## Summary

- **Trellis2TexturingPipeline** provides end-to-end PBR texture generation via [`trellis2/pipelines/trellis2_texturing.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_texturing.py) with ten modular stages
- **SparseTensor** representations and dual-grid conversions enable efficient processing of complex geometries at high resolution
- The `run` method orchestrates preprocessing, encoding, sampling, and rasterization through a single interface
- Low-VRAM mode automatically manages device memory by cycling weights between CPU and GPU during processing
- Output meshes include UV coordinates and `TextureVisuals` compatible with GLB, OBJ, and other standard formats

## Frequently Asked Questions

### What input mesh formats does Trellis2TexturingPipeline support?

The pipeline accepts any format supported by the trimesh library, including PLY, OBJ, GLB, and STL. The `preprocess_mesh` method automatically handles coordinate system conversion (Y/Z axis swapping) and normalization to ensure compatibility with internal voxelization requirements.

### How does the pipeline handle images without transparent backgrounds?

The `preprocess_image` method automatically detects missing alpha channels and invokes a Rembg model to remove backgrounds. It then crops to the tightest foreground bounding box and premultiplies alpha values, preparing the image for conditioning regardless of the original background state.

### What is the difference between resolution and texture_size parameters?

The **resolution** parameter (512 or 1024) controls the internal voxel grid size used during shape encoding in `encode_shape_slat` and texture sampling in `sample_tex_slat`. The **texture_size** parameter (e.g., 2048) determines the final output texture dimensions rasterized onto mesh UVs during `postprocess_mesh`. Higher resolution values capture more geometric detail but require additional VRAM during processing.

### Can I run Trellis2TexturingPipeline on CPU-only systems?

Yes. While GPU acceleration via `pipeline.cuda()` is recommended for performance, the pipeline supports CPU inference through `pipeline.to(torch.device("cpu"))`. The low-VRAM mode further reduces memory requirements by keeping inactive model weights on CPU, though processing times will increase significantly without CUDA acceleration.