How Frustum Culling Improves Rendering Performance in LingBot-Map's Point Cloud Pipeline

Frustum culling discards invisible geometry before downstream processing by using a GPU-accelerated two-kernel system that projects world-space points against camera bounds and atomically selects only the nearest visible point per pixel, reducing memory traffic and compute load by up to 99%.

LingBot-Map processes massive point clouds streamed from long video sequences, where unfiltered data can reach tens of millions of points. The pipeline implements frustum culling directly on the GPU to eliminate points outside the camera view frustum before voxelization, shading, or disk operations. This technique ensures that only visible geometry consumes computational resources, enabling the rendering of hundreds of thousands of frames on consumer hardware without memory exhaustion.

The GPU-Accelerated Two-Kernel Architecture

The implementation in preprocess/points_visibility/frustum_cull.cu uses two distinct CUDA kernels to filter points in parallel without CPU intervention. This design eliminates the need for a separate CPU-side visibility pass and keeps data transfers to a minimum.

Projection Kernel (frustum_project_kernel)

The first kernel transforms each world-space point using the camera extrinsics (R_cw, t_cw) and tests against the near/far depth range and image bounds (W × H). For every point that passes the frustum test, the kernel packs the depth (as an ordered integer) and the original point index into a 64-bit value. It then performs an atomicMin operation on a per-pixel "combined map," ensuring that only the closest point per pixel survives. This executes for all N points in parallel across N threads, making the operation cost scale linearly with point count rather than pixel overlap density.

Unpack Kernel (unpack_kernel)

The second kernel reads the combined map and produces two output tensors: a depth_map containing the nearest depth values and a winner_map storing the original point indices. Pixels with no projected points receive zero depth and a -1 index. This separation allows downstream modules to reference exact 3D coordinates via the winner_map without reprocessing the entire original point cloud.

Three Major Performance Benefits

By running culling entirely on the GPU before any downstream processing, LingBot-Map achieves significant optimizations:

• Reduced Memory Traffic: Only visible points are retained in the depth and winner maps. The GPU never transfers the full raw point set to later stages, saving substantial VRAM bandwidth.

• Lower Compute Load: Subsequent operations such as voxel-Morton hashing or rasterization operate on dramatically smaller datasets—often less than 1% of the original cloud for typical camera poses.

• Constant-Time Per-Pixel Work: The atomicMin operation ensures each pixel's processing remains O(1) regardless of how many points project onto it. Kernel cost scales linearly with point count, not with per-pixel overlap density.

Implementation and API Usage

The CUDA code is compiled into the Python extension frustum_cull_ext via demo_render/render_cuda_ext/setup.py and exposed through demo_render/render_cuda_ext/render_cuda_ext/_api.py.

The following Python snippet demonstrates how to invoke the culling routine:

import torch
from render_cuda_ext import frustum_cull_ext

def frustum_cull(points_world, R_cw, t_cw, intrinsics, img_wh, near=0.1, far=100.0):
    """
    points_world : (N, 3) float32 CUDA tensor
    R_cw         : (3, 3) float32 CUDA tensor (world → camera rotation)
    t_cw         : (3,)   float32 CUDA tensor (world → camera translation)
    intrinsics   : (4,)   float32 tensor  (fx, fy, cx, cy)
    img_wh       : (2,)   int32 tensor    (W, H)
    """
    fx, fy, cx, cy = intrinsics.tolist()
    W, H = img_wh.tolist()
    depth_map, winner_map = frustum_cull_ext.frustum_cull(
        points_world, R_cw, t_cw, fx, fy, cx, cy, W, H, near, far
    )
    return depth_map, winner_map

# ----------------------------------------------------------------------

# Example usage inside the rendering pipeline:

# ----------------------------------------------------------------------

# Load a point cloud (N × 3) already on the GPU

pts = torch.randn(2_000_000, 3, device="cuda", dtype=torch.float32)

# Camera pose (world → cam)

R = torch.eye(3, device="cuda", dtype=torch.float32)   # identity rotation

t = torch.tensor([0.0, 0.0, -5.0], device="cuda")      # translate back 5 m

# Intrinsics for a 640×480 image

intr = torch.tensor([525.0, 525.0, 320.0, 240.0], device="cuda")
wh   = torch.tensor([640, 480], device="cuda", dtype=torch.int32)

depth, winner = frustum_cull(pts, R, t, intr, wh, near=0.1, far=10.0)

# `depth` now holds a (H, W) float32 map of the nearest visible depth.

# `winner` holds the index into `pts` for each pixel that saw a hit.

# These tensors are passed to the voxel-morton encoder / visualiser.

The depth tensor contains the (H, W) depth image of the nearest visible surface, while winner stores indices back into the original point cloud for downstream color assignment or 3D coordinate retrieval.

Summary

• Frustum culling in LingBot-Map uses a two-kernel GPU approach (frustum_project_kernel and unpack_kernel) defined in preprocess/points_visibility/frustum_cull.cu to filter points without CPU bottlenecks.
• The atomicMin operation on packed 64-bit depth-index values ensures only the nearest point per pixel survives, reducing data volume by up to 99%.
• The implementation is exposed via the frustum_cull_ext Python extension built by demo_render/render_cuda_ext/setup.py and wrapped in demo_render/render_cuda_ext/render_cuda_ext/_api.py.
• This optimization enables processing of hundreds of thousands of frames on single consumer GPUs while maintaining interactive viewer performance at approximately 20 FPS.

Frequently Asked Questions

What is frustum culling in the context of point cloud rendering?

Frustum culling is a visibility determination technique that discards 3D points falling outside the camera's view frustum before rendering. In LingBot-Map, this prevents invisible geometry from consuming GPU memory and computational resources during downstream processing stages like voxelization or shading, which is critical when handling tens of millions of points from long video sequences.

How does the atomicMin operation improve performance in the frustum culling pipeline?

The atomicMin operation allows thousands of threads to compete for each pixel simultaneously, automatically selecting the closest depth value without requiring explicit sorting or thread synchronization. This ensures constant-time O(1) per-pixel work regardless of point density, preventing performance degradation when many points project to the same screen location.

Why is the winner_map necessary if we already have the depth_map?

The winner_map stores the original point indices that survived the depth competition, allowing downstream modules to retrieve exact 3D world coordinates, colors, or other attributes from the source point cloud. Without this back-reference, the pipeline would lose connection to the original data after projection, making it impossible to perform voxel-based operations or color mapping on the visible subset.

Can the frustum culling implementation run on CPU-only systems?

No, the implementation requires CUDA-capable GPUs as defined in preprocess/points_visibility/frustum_cull.cu. The performance benefits depend on GPU parallelism and hardware atomic operations; a CPU fallback would require complete reimplementation and would likely bottleneck when processing the millions of points typical in LingBot-Map's video sequences.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}