# How Background Removal Preprocessing Works in TRELLIS.2

> Discover how TRELLIS.2 background removal preprocessing isolates foreground objects using BiRefNet segmentation, resizing, alpha masks, and cropping for precise 3D generation.

- Repository: [Microsoft/TRELLIS.2](https://github.com/microsoft/TRELLIS.2)
- Tags: how-to-guide
- Published: 2026-08-03

---

**Background removal preprocessing in TRELLIS.2 automatically isolates foreground objects using the BiRefNet segmentation model, resizing inputs to 1024px, generating alpha masks, and cropping to centered bounding boxes before 3D generation.**

The `preprocess_image` method in the microsoft/TRELLIS.2 repository handles this critical preprocessing step before feeding images to the 3D generation models. Implemented identically in both `Trellis2ImageTo3DPipeline` and `Trellis2TexturingPipeline`, this pipeline relies on a configurable background-removal model that defaults to the BiRefNet implementation.

## The Preprocessing Pipeline Architecture

Background removal preprocessing occurs in the `preprocess_image` method found in [`trellis2/pipelines/trellis2_image_to_3d.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_image_to_3d.py) and [`trellis2/pipelines/trellis2_texturing.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_texturing.py). Both pipelines instantiate a `rembg_model` attribute that defaults to the BiRefNet model loaded from Hugging Face, though the architecture supports swapping in alternative segmentation implementations.

The process operates on `PIL.Image.Image` inputs and returns processed RGB images with transparent or black backgrounds where the original background pixels were removed. This ensures that 3D generation models receive clean foreground objects without distracting background elements.

## Six-Step Background Removal Process

The `preprocess_image` method executes a deterministic sequence to extract foreground objects from input photographs.

### 1. Alpha Channel Detection

The pipeline first checks if the input image already contains a usable mask. In [`trellis2/pipelines/trellis2_image_to_3d.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_image_to_3d.py) (lines 31-33), the code scans the alpha channel:

```python
alpha = np.array(input)[:, :, 3]
has_alpha = np.any(alpha != 255)

```

If `has_alpha` evaluates to `True`—meaning any pixel has an alpha value not equal to 255—the image is treated as pre-masked and background removal is skipped entirely.

### 2. Efficiency Resizing

For performance optimization, the image is downscaled to a maximum dimension of 1024 pixels while preserving aspect ratio using `Image.Resampling.LANCZOS` (lines 38-40):

```python
scale = min(1, 1024 / max_size)
new_size = (int(input.width * scale), int(input.height * scale))
input = input.resize(new_size, Image.Resampling.LANCZOS)

```

This resizing occurs before segmentation to reduce VRAM usage during model inference.

### 3. BiRefNet Segmentation

When no alpha channel exists, the RGB image is passed to the `rembg_model` running on the pipeline's device (CPU or CUDA). The call at lines 44-48 executes:

```python
output = self.rembg_model(input)

```

The model returns an RGBA image where the newly generated alpha channel encodes the foreground mask, with white representing the object and black representing the background.

### 4. Foreground Bounding Box Extraction

The method extracts the foreground region by identifying pixels where alpha exceeds 80% intensity (lines 50-55):

```python
bbox = np.argwhere(alpha > 0.8 * 255)
bbox = [bbox.min(axis=0)[1], bbox.min(axis=0)[0], 
        bbox.max(axis=0)[1] + 1, bbox.max(axis=0)[0] + 1]
output = output.crop(bbox)

```

This creates a centered square crop containing only the detected object, removing excess transparent padding.

### 5. RGB Premultiplication

To ensure proper blending in downstream 3D models, the cropped RGBA image undergoes premultiplication (lines 56-60):

```python
output = np.array(output).astype(np.float32) / 255
output = output[:, :, :3] * output[:, :, 3:4]
output = (output * 255).astype(np.uint8)
output = Image.fromarray(output, mode='RGB')

```

This mathematical operation multiplies the RGB channels by the alpha mask, converting transparent regions to black and ensuring the foreground object blends correctly with 3D rendering backgrounds.

### 6. Output Generation

The final `Image.Image` object containing only the foreground object is returned at line 61, ready for conditioning the 3D generation networks.

## The BiRefNet Model Implementation

The default background removal model resides in [`trellis2/pipelines/rembg/BiRefNet.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/rembg/BiRefNet.py) and implements a complete segmentation workflow.

### Model Loading and Configuration

The `BiRefNet` class loads a pretrained `AutoModelForImageSegmentation` from Hugging Face (lines 9-13):

```python
from transformers import AutoModelForImageSegmentation
self.model = AutoModelForImageSegmentation.from_pretrained(
    "ZhengPeng7/BiRefNet", trust_remote_code=True
)

```

### Inference Pipeline

The model processes inputs through a standardized transformation pipeline (lines 14-19) including `Resize` to 1024×1024, `ToTensor`, and `Normalize` operations. During the forward pass (lines 35-40), the model generates a prediction tensor that is converted to a PIL mask via `transforms.ToPILImage()(pred)`.

### Alpha Channel Integration

The final step inserts the generated mask into the original image's alpha channel (lines 40-41):

```python
image.putalpha(mask)

```

This produces the RGBA output that the preprocessing pipeline expects for subsequent bounding box extraction and cropping operations.

## Memory Optimization Strategies

In low-VRAM environments, the pipeline implements dynamic device management to minimize GPU memory usage. The `rembg_model` temporarily moves to the inference device (`self.rembg_model.to(self.device)`) for segmentation and returns to CPU afterward. This prevents the BiRefNet parameters from permanently occupying GPU memory during the full 3D generation workflow, allowing the system to run on hardware with limited VRAM.

## Practical Code Examples

### Using the Image-to-3D Pipeline

```python
from trellis2.pipelines.trellis2_image_to_3d import Trellis2ImageTo3DPipeline
from PIL import Image

# Load pretrained pipeline with BiRefNet background removal

pipeline = Trellis2ImageTo3DPipeline.from_pretrained(
    path="microsoft/trellis2-pretrained",
    config_file="pipeline.json"
)

# Optional: Move models to GPU

pipeline.to("cuda")

# Load RGB photograph

input_img = Image.open("photo.jpg")

# Background removal preprocessing happens automatically

foreground = pipeline.preprocess_image(input_img)
foreground.save("photo_fg.png")

```

### Direct BiRefNet Invocation

```python
from trellis2.pipelines.rembg.BiRefNet import BiRefNet
from PIL import Image

# Initialize background removal model

rembg = BiRefNet()
rembg.to("cuda")

# Process image directly

img = Image.open("portrait.jpg")
rgba_output = rembg(img)  # Returns RGBA with segmentation mask

rgba_output.save("portrait_fg.png")

```

## Summary

- **TRELLIS.2** implements background removal preprocessing in [`trellis2/pipelines/trellis2_image_to_3d.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_image_to_3d.py) and [`trellis2/pipelines/trellis2_texturing.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_texturing.py) via the `preprocess_image` method.
- The pipeline defaults to **BiRefNet** segmentation but supports configurable replacement models through the `rembg_model` interface.
- Preprocessing includes alpha channel detection, 1024px resizing, foreground bounding box extraction (alpha > 0.8 threshold), and RGB premultiplication.
- The system optimizes for low-VRAM scenarios by temporarily moving the segmentation model to GPU only during inference.
- All operations preserve aspect ratio and center-crop results to provide consistent inputs for 3D generation models.

## Frequently Asked Questions

### What segmentation model does TRELLIS.2 use for background removal?

TRELLIS.2 uses **BiRefNet** (Bilateral Reference Network) as the default background removal model, loaded from the `ZhengPeng7/BiRefNet` Hugging Face repository. The model is instantiated in [`trellis2/pipelines/rembg/BiRefNet.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/rembg/BiRefNet.py) and resizes inputs to 1024×1024 resolution before generating foreground masks.

### Does TRELLIS.2 skip background removal if the image already has transparency?

Yes. The `preprocess_image` method detects existing alpha channels by checking if any pixel value differs from 255 in the fourth channel. According to lines 31-33 of [`trellis2/pipelines/trellis2_image_to_3d.py`](https://github.com/microsoft/TRELLIS.2/blob/main/trellis2/pipelines/trellis2_image_to_3d.py), if `np.any(alpha != 255)` evaluates to `True`, the pipeline treats the image as pre-masked and bypasses the BiRefNet segmentation step entirely.

### How does TRELLIS.2 handle limited GPU memory during preprocessing?

The pipeline implements temporary device migration for low-VRAM systems. The `rembg_model` moves to the active compute device (CUDA) immediately before inference and returns to CPU afterward, preventing the segmentation model from permanently occupying GPU memory during the full 3D generation workflow.

### Can I replace the default BiRefNet model with custom background removal?

Yes. The pipeline accepts a configurable `rembg_model` parameter during initialization. Any model implementing the `__call__(image: PIL.Image.Image) -> PIL.Image.Image` interface that returns RGBA images with valid alpha channels can replace the default BiRefNet implementation in both `Trellis2ImageTo3DPipeline` and `Trellis2TexturingPipeline`.