How to Use rembg Background Removal in the TRELLIS.2 Image Preprocessing Pipeline

The TRELLIS.2 framework from Microsoft enables automatic background removal by configuring a rembg_model parameter when initializing the pipeline, which instantiates the specified rembg class and applies it during the _preprocess_image stage.

The microsoft/TRELLIS.2 repository integrates the rembg library directly into its image-to-3D and texturing pipelines, allowing you to clean input images without external preprocessing scripts. By leveraging the rembg submodule imported in trellis2/pipelines/__init__.py, the framework dynamically loads background removal models and applies them transparently during the preprocessing phase. This integration supports models like U2Net and U2NetPortrait, automatically handling device placement and batch processing.

How rembg Integration Works in TRELLIS.2

Configuration via the rembg_model Parameter

The pipeline accepts a nested dictionary under the key rembg_model containing the class name and initialization arguments. In trellis2/pipelines/trellis2_image_to_3d.py, the constructor parses this configuration to determine which background removal model to load. The expected structure includes a name field specifying the class (e.g., U2Net) and an args dictionary containing initialization parameters like pretrained weights.

Dynamic Class Instantiation

Around line 86 of trellis2/pipelines/trellis2_image_to_3d.py, the pipeline retrieves the specified class from the rembg submodule using getattr(rembg, args['rembg_model']['name']). It then instantiates the model by unpacking the configuration arguments with **args['rembg_model']['args']. This dynamic approach allows you to swap between U2Net, U2NetPortrait, or MobileNetV2 variants without modifying source code.

Device Management and Application

The pipeline handles GPU allocation automatically. Lines 103-105 move the instantiated model to the configured device using .to(device) when CUDA is available. During execution, the _preprocess_image method invokes self.rembg_model(input) to strip backgrounds before tensors proceed to the 3D reconstruction modules (lines 141-147). After processing each batch, the model can be moved back to CPU to free VRAM, ensuring efficient memory management during large dataset processing.

Practical Implementation Examples

Example 1: U2Net Background Removal in 3D Reconstruction

from trellis2.pipelines import trellis2_image_to_3d

config = {
    "input_dir": "data/images",
    "output_dir": "output/3d",
    "device": "cuda",
    "rembg_model": {
        "name": "U2Net",
        "args": {"pretrained": "u2net"}
    }
}

pipeline = trellis2_image_to_3d.Trellis2ImageTo3D(**config)
pipeline.run()

Example 2: Portrait-Specific Removal in Texturing

from trellis2.pipelines import trellis2_texturing

config = {
    "input_dir": "data/textures",
    "output_dir": "output/textured",
    "device": "cuda",
    "rembg_model": {
        "name": "U2NetPortrait",
        "args": {"pretrained": "u2net_portrait"}
    }
}

pipeline = trellis2_texturing.Trellis2Texturing(**config)
pipeline.run()

Key Source Files and Functions

• trellis2/pipelines/__init__.py: Imports the rembg submodule via from . import rembg, exposing the background removal classes to the pipeline package.
• trellis2/pipelines/trellis2_image_to_3d.py: Contains the instantiation logic (getattr at line 86), device handling (lines 103-105), and the _preprocess_image method (lines 141-147) that applies the model.
• trellis2/pipelines/trellis2_texturing.py: Implements identical rembg integration for texture synthesis pipelines, using the same configuration schema and preprocessing workflow.

Summary

• Configure rembg_model with name and args keys to enable automatic background removal in TRELLIS.2 pipelines.
• The framework uses getattr(rembg, name) to dynamically load model classes from the rembg submodule.
• Models are automatically moved to the specified compute device and applied within _preprocess_image before 3D generation.
• Supported architectures include U2Net, U2NetPortrait, and MobileNetV2, configurable without code modifications.
• GPU memory is managed efficiently by moving models back to CPU after batch processing when configured to do so.

Frequently Asked Questions

What rembg models are supported by TRELLIS.2?

TRELLIS.2 supports any model class available in the rembg submodule, including U2Net, U2NetPortrait, and MobileNetV2. You specify the exact class name in the rembg_model.name configuration field, and the pipeline instantiates it dynamically using getattr(rembg, config['rembg_model']['name']).

Does background removal execute on GPU or CPU?

The pipeline automatically moves the rembg model to the device specified in your configuration (typically "cuda" or "cpu") using .to(device) before processing. After each batch completes, the framework can relocate the model back to CPU to free GPU memory, making it suitable for processing large image collections.

Can I use rembg with the texturing pipeline as well?

Yes, the identical rembg_model configuration schema applies to both the image-to-3D pipeline (trellis2_image_to_3d.py) and the texturing pipeline. Both classes implement the same preprocessing logic to strip backgrounds before applying textures or generating 3D meshes.

Where in the code is the background removal actually executed?

The removal occurs in the _preprocess_image method of trellis2/pipelines/trellis2_image_to_3d.py, specifically where the pipeline calls self.rembg_model(input) on the loaded image tensor. This execution happens after image loading and tensor conversion but before the data reaches the 3D reconstruction or texturing networks.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}