How Does PBR Texture Decoding Work in TRELLIS.2: Base Color, Metallic, Roughness, and Alpha

TRELLIS.2 decodes PBR texture channels—base color, metallic, roughness, and alpha—through a layout-based indexing system in PbrMeshRenderer that samples GPU textures, applies material factors, clamps values, and optionally applies sRGB gamma correction.

TRELLIS.2 is Microsoft's open-source framework for 3D asset generation, and its physically-based rendering (PBR) pipeline handles four critical material channels. Understanding how this PBR texture decoding functions is essential for anyone working with material generation, texture editing, or custom rendering pipelines in the repository.

The Four PBR Channels and Layout Mapping

Every mesh with PBR materials in TRELLIS.2 carries a layout dictionary that maps logical channel names to tensor indices. This decouples semantic meaning from raw tensor ordering.

In trellis2/representations/mesh/base.py, the MeshWithPbrMaterial class defines this layout. The PbrMeshRenderer then uses these indices to extract channels from the rendered image tensor:

# Inside PbrMeshRenderer.render() - trellis2/renderers/pbr_mesh_renderer.py

gb_basecolor = img[0, ..., mesh.layout['base_color']]      # line 338

gb_metallic   = img[0, ..., mesh.layout['metallic']]       # line 339

gb_roughness = img[0, ..., mesh.layout['roughness']]       # line 340

gb_alpha     = img[0, ..., mesh.layout['alpha']]           # line 341

This layout-driven approach allows flexible channel ordering without hardcoded assumptions throughout the codebase.

Texture Sampling and Material Factor Application

The renderer applies a consistent pattern across all four channels: sample the GPU texture if present, then multiply by a scalar material factor.

Metallic Channel Processing

# trellis2/renderers/pbr_mesh_renderer.py lines 380-390

if mat.metallic_texture is not None:
    m = dr.texture(
        mat.metallic_texture.image.unsqueeze(0), 
        texcoord,
        filter_mode='linear-mipmap-linear' if mat.metallic_texture.filter_mode == TextureFilterMode.LINEAR else 'nearest',
        boundary_mode='clamp' if mat.metallic_texture.wrap_mode == TextureWrapMode.CLAMP_TO_EDGE else 'wrap'
    )
    gb_metallic += m * mat.metallic_factor * mat_mask
else:
    gb_metallic += mat.metallic_factor * mat_mask  # line 388

The same pattern applies to base color (lines 337-350), roughness (lines 392-402), and alpha (lines 424-430). The dr.texture call uses NVDiffRast for differentiable rasterization, enabling gradient flow through texture sampling.

Factor Multiplication Logic

Each channel supports an optional texture map plus a mandatory scalar factor:

Channel	Texture Attribute	Factor Attribute
base_color	base_color_texture	base_color_factor
metallic	metallic_texture	metallic_factor
roughness	roughness_texture	roughness_factor
alpha	alpha_texture	alpha_mode (blending control)

When no texture is present, the renderer falls back to uniform values defined by the factors alone.

Post-Processing: Clamping, Gamma, and Output Packing

After accumulation across all materials and lights, channels undergo final processing before returning to the pipeline:

# trellis2/renderers/pbr_mesh_renderer.py lines 430-436

out_dict.base_color = torch.clamp(gb_basecolor, 0.0, 1.0) ** 2.2   # sRGB gamma

out_dict.metallic   = torch.clamp(gb_metallic,   0.0, 1.0)
out_dict.roughness  = torch.clamp(gb_roughness,  0.0, 1.0)
out_dict.alpha      = torch.clamp(gb_alpha,      0.0, 1.0)

Note the gamma correction (²·² exponent) applied specifically to base color—converting from linear to sRGB space—while metallic, roughness, and alpha remain linear. The output is an EasyDict consumed by downstream components.

Dataset Pipeline: Building Combined Textures

The texturing pipeline in trellis2/pipelines/trellis2_texturing.py handles the inverse operation: packing raw attribute tensors into standard texture formats for storage.

Metallic-Roughness-Alpha Packing

# trellis2/pipelines/trellis2_texturing.py lines 337-338

metallic = np.clip(attrs[..., self.pbr_attr_layout['metallic']].cpu().numpy()*255, 0, 255).astype(np.uint8)
roughness = np.clip(attrs[..., self.pbr_attr_layout['roughness']].cpu().numpy()*255, 0, 255).astype(np.uint8)

# Combine: R=unused, G=roughness, B=metallic (matches glTF 2.0 convention)

metallicRoughnessTexture = Image.fromarray(
    np.concatenate([np.zeros_like(metallic), roughness, metallic], axis=-1)
)

This produces a 3-channel PNG compatible with the glTF 2.0 metallic-roughness standard. The alpha channel, when present, is typically stored as a separate texture or in the base color's alpha channel depending on the export configuration.

Complete Rendering Example

from trellis2.renderers.pbr_mesh_renderer import PbrMeshRenderer

# Initialize renderer with CUDA acceleration

renderer = PbrMeshRenderer(rendering_options={}, device='cuda')

# Render a mesh with PBR materials

render_output = renderer.render(mesh)  # mesh: MeshWithPbrMaterial

# Access decoded channels

base_color = render_output.base_color   # [H, W, 3] float32, sRGB

metallic   = render_output.metallic     # [H, W, 1] float32, linear

roughness  = render_output.roughness    # [H, W, 1] float32, linear

alpha      = render_output.alpha        # [H, W, 1] float32, linear

Creating Combined Textures from Raw Attributes

import numpy as np
from PIL import Image

# attrs: tensor of shape [N, C, H, W] from model output

metallic = np.clip(attrs[..., 3].cpu().numpy() * 255, 0, 255).astype(np.uint8)
roughness = np.clip(attrs[..., 4].cpu().numpy() * 255, 0, 255).astype(np.uint8)

# Pack into standard metallic-roughness format

combined = np.concatenate([
    np.zeros_like(metallic),  # reserved channel

    roughness,                # green channel

    metallic                  # blue channel

], axis=-1)

texture_image = Image.fromarray(combined)
texture_image.save('metallic_roughness.png')

Key Implementation Files

File	Purpose
trellis2/renderers/pbr_mesh_renderer.py	Core PBR texture decoding logic, sampling, and factor application
trellis2/representations/mesh/base.py	MeshWithPbrMaterial class and layout definitions
trellis2/pipelines/trellis2_texturing.py	Texture creation and channel packing for export
trellis2/utils/render_utils.py	Tensor-to-image conversion helpers

Summary

• Layout-based indexing decouples semantic channel names from tensor positions via mesh.layout dictionaries
• Differentiable texture sampling uses NVDiffRast's dr.texture with configurable filtering and wrapping
• Material factors provide per-channel scalar multipliers that apply regardless of texture presence
• Gamma correction applies only to base color (sRGB output), while metallic, roughness, and alpha remain linear
• Standard packing follows glTF 2.0 conventions in the dataset pipeline for interoperability

Frequently Asked Questions

How does TRELLIS.2 handle missing PBR textures?

When a texture is absent, the renderer falls back to the material's scalar factor alone. For example, if metallic_texture is None, gb_metallic receives only mat.metallic_factor * mat_mask without texture sampling.

Why does base color get gamma-corrected while other channels don't?

Base color represents diffuse albedo, which human perception expects in sRGB space. Metallic, roughness, and alpha are physically meaningful linear parameters used directly in BRDF calculations. The ²·² exponent conversion in line 430 prepares base color for display while preserving physical correctness for the other channels.

What texture format does TRELLIS.2 use for metallic-roughness export?

The pipeline creates a 3-channel PNG with green channel storing roughness and blue channel storing metallic, matching the glTF 2.0 metallic-roughness texture specification. The red channel is zeroed as unused.

Can I modify the PBR channel layout in my own meshes?

Yes. The MeshWithPbrMaterial accepts custom layout dictionaries, but you must ensure consistency between the layout used during texture creation in trellis2_texturing.py and the layout expected by PbrMeshRenderer during inference.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}