How to Integrate TRELLIS.2 with Custom 3D Formats Using the O-Voxel API

TRELLIS.2 ships with the O-Voxel sub-package that converts arbitrary mesh data into sparse voxel representations stored in the compact .vxz format, enabling seamless integration of proprietary 3D formats into the TRELLIS.2 training and rendering pipeline.

The microsoft/TRELLIS.2 repository includes a complete voxelization toolkit under the o-voxel/ directory that abstracts complex spatial encoding and compression. By leveraging the O-Voxel API, you can transform any vertex-face mesh data into the native sparse voxel grid format without modifying core library code. This guide demonstrates the complete workflow from custom format ingestion to compressed voxel output.

Overview of the O-Voxel Pipeline

The O-Voxel API provides a four-stage pipeline for integrating external 3D data. First, you load your custom format into PyTorch tensors. Second, you voxelize the mesh using the flexible dual-grid algorithm. Third, you optionally attach custom per-voxel attributes. Finally, you serialize the data to the .vxz compression format.

Key components reside in specific source files:

• o-voxel/o_voxel/convert/flexible_dual_grid.py contains the mesh_to_flexible_dual_grid function for voxelization
• o-voxel/o_voxel/serialize.py handles Morton/Hilbert encoding via encode_seq and decode_seq
• o-voxel/o_voxel/io/vxz.py implements the write_vxz and read_vxz functions for I/O
• o-voxel/o_voxel/rasterize.py provides the VoxelRenderer for visualization

Step 1: Load Custom Format into Tensors

To begin the integration, parse your proprietary 3D format into a torch.Tensor of vertex positions and a torch.LongTensor of face indices. The following example demonstrates parsing a simple CSV-based mesh format:

import torch
import csv
from pathlib import Path

def load_custom_mesh(path: Path):
    verts, faces = [], []
    with open(path, newline="") as f:
        reader = csv.reader(f)
        for row in reader:
            if row[0] == "v":          # vertex line: v,x,y,z

                verts.append([float(v) for v in row[1:]])
            elif row[0] == "f":        # face line: f,i0,i1,i2 (0‑based)

                faces.append([int(i) for i in row[1:]])
    vertices = torch.tensor(verts, dtype=torch.float32)
    faces    = torch.tensor(faces,  dtype=torch.long)
    return vertices, faces

Step 2: Voxelize with Flexible Dual Grid

Once loaded, convert the mesh to a sparse voxel representation using mesh_to_flexible_dual_grid from o_voxel.convert. This function solves the dual-vertex QEF (Quadratic Error Function) and generates per-voxel intersection data.

from o_voxel.convert import mesh_to_flexible_dual_grid
from o_voxel import serialize, io

# Load mesh

vertices, faces = load_custom_mesh(Path("assets/custom_mesh.xyz"))

# Voxelization – choose a resolution (grid size) that matches your application

RES = 256                                   # e.g. 256³ voxels

voxel_indices, dual_vertices, intersected = mesh_to_flexible_dual_grid(
    vertices,
    faces,
    grid_size=RES,
    aabb=[[-0.5, -0.5, -0.5], [0.5, 0.5, 0.5]],   # optional bounding box

    face_weight=1.0,
    boundary_weight=0.2,
    regularization_weight=1e-2,
    timing=True
)

# Sort to keep geometry & material streams aligned (required by the I/O layer)

vid = serialize.encode_seq(voxel_indices)
mapping = torch.argsort(vid)
voxel_indices = voxel_indices[mapping]
dual_vertices = dual_vertices[mapping]
intersected = intersected[mapping]

The serialize.encode_seq function applies Morton or Hilbert encoding to generate sortable voxel IDs, ensuring spatial coherence in the compressed output.

Step 3: Attach Custom Per-Voxel Attributes

You can enrich the voxel data with custom attributes such as segmentation masks or scalar fields. These must be provided as torch.uint8 tensors matching the voxel count.

# Suppose you have a numpy array `scalar_field` with shape (N,)

import numpy as np
scalar_field = np.random.rand(voxel_indices.shape[0]).astype(np.float32)

# Convert to uint8 (O‑Voxel expects uint8 attributes)

scalar_uint8 = (scalar_field * 255).astype(np.uint8)
scalar_tensor = torch.from_numpy(scalar_uint8)

# Build the attribute dictionary – the key becomes the attribute name in the .vxz file

attributes = {
    "dual_vertices": dual_vertices,          # already prepared by the pipeline

    "intersected":    intersected,
    "scalar":        scalar_tensor,
}

Step 4: Write to Compressed VXZ Format

Serialize the processed data using io.write_vxz from o-voxel/o_voxel/io/vxz.py. This creates a compact .vxz file using zstd compression and chunking for efficient storage.

output_path = Path("output/custom_mesh.vxz")
io.write_vxz(
    output_path,
    coord=voxel_indices,
    attr=attributes,
    chunk_size=256,               # must be a power of two

    compression="zstd",          # fast and high‑ratio compression

    attr_interleave="as_is",     # keep each attribute separate

    num_threads=-1               # use all CPU cores

)
print(f"Saved O‑Voxel file to {output_path}")

Step 5: Load and Render in TRELLIS.2

After conversion, load the .vxz file back into TRELLIS.2 for rendering or training. The VoxelRenderer class rasterizes the sparse voxels for visualization.

from o_voxel.io import read_vxz
from o_voxel.rasterize import VoxelRenderer

coords, data = read_vxz(output_path)

# Example: render the custom scalar field as a grayscale image

scalar = data["scalar"].float() / 255.0   # back‑to‑float

renderer = VoxelRenderer(rendering_options={"resolution": 512, "ssaa": 2})
rendered = renderer.render(
    position=(coords / RES - 0.5).cuda(),
    attrs=scalar.unsqueeze(1).cuda(),      # (N, 1) channel

    voxel_size=1.0 / RES,
    extrinsics=torch.eye(4).cuda(),
    intrinsics=torch.tensor([[500.0, 0, 256], [0, 500.0, 256], [0, 0, 1]]).cuda()
)

# `rendered.attr` now holds a (1, H, W) image you can save with torchvision or PIL

Summary

• O-Voxel API Location: The integration toolkit resides in microsoft/TRELLIS.2 under the o-voxel/ directory, providing mesh-to-voxel conversion without core library modifications.
• Core Function: Use mesh_to_flexible_dual_grid in o-voxel/o_voxel/convert/flexible_dual_grid.py to convert vertex-face tensors into sparse voxel grids.
• Encoding: Apply serialize.encode_seq to generate Morton/Hilbert codes for spatial sorting before compression.
• Storage: Write finalized data using io.write_vxz to produce compressed .vxz files compatible with TRELLIS.2 training pipelines.
• Visualization: Render results immediately using VoxelRenderer from o-voxel/o_voxel/rasterize.py to validate the integration.

Frequently Asked Questions

What file formats does the O-Voxel API support for input?

The O-Voxel API accepts any format you can parse into PyTorch tensors. The core functions mesh_to_flexible_dual_grid require only a torch.Tensor of vertices and a torch.LongTensor of face indices. You must implement your own loader for proprietary formats, as demonstrated in the CSV example above.

How does the .vxz compression format work?

The .vxz format stores sparse voxel coordinates and attributes in zstd-compressed chunks. According to o-voxel/o_voxel/io/vxz.py, the writer organizes data into power-of-two sized chunks (typically 256) and supports attribute interleaving options. The format uses Morton or Hilbert encoding for spatial coherence, enabling efficient random access and small file sizes.

Can I add multiple custom attributes to a single voxel file?

Yes. The attributes dictionary passed to io.write_vxz accepts multiple torch.uint8 tensors. Each key in the dictionary becomes a named attribute accessible when loading the file via read_vxz. Ensure all attribute tensors match the voxel count and use uint8 dtype for compatibility.

Is the voxelization process differentiable?

The mesh_to_flexible_dual_grid function performs QEF solving and dual-grid generation, which are primarily geometric operations optimized for throughput rather than automatic differentiation. For differentiable voxelization requirements, you may need to wrap specific operations or use the output grids as inputs to differentiable neural network components within TRELLIS.2.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}