How SuperSplat Implements Shader-Based Selection Operations for Real-Time Intersection Testing

SuperSplat performs selection entirely on the GPU using a specialized fragment shader that tests millions of splats in parallel against mask, rectangle, sphere, or box selectors, returning results via render target readback.

The playcanvas/supersplat repository implements picking operations through GPU-accelerated intersection testing rather than traditional CPU-side loops. By encoding selection logic into the fragment shader at src/shaders/intersection-shader.ts, the application evaluates intersection criteria against scene splats in parallel, scaling efficiently to millions of points without blocking the render thread.

The GPU-Driven Selection Architecture

Selection operations in SuperSplat are implemented as full-screen render passes. When a user activates a selection tool, the system constructs a fragment shader that iterates over splat geometry and writes selection masks to a framebuffer. The CPU later reads this buffer to update the internal selection state managed by src/selection.ts.

The high-level API exposed in src/picker.ts orchestrates this process. It configures uniform buffers, triggers the render pass via src/utils/simple-render-pass.ts, and interprets the resulting Uint8Array to populate the selection.changed event pipeline.

Shader Input Data Preparation

Before executing the intersection pass, the CPU uploads comprehensive scene data as shader uniforms. These buffers enable the fragment shader to reconstruct each splat's world position and screen-space projection:

• transformA – A texture containing raw splat center positions (x, y, z).
• splatTransform and transformPalette – Optional per-splat model transforms stored as 3×4 matrices.
• splat_params – Dimensions of the splat-center texture and total splat count.
• matrix_model and matrix_viewProjection – Scene transformation matrices for coordinate conversion.
• output_params – Dimensions of the output selection mask texture.
• mode – Integer selector type (0 = mask, 1 = rectangle, 2 = sphere, 3 = box).
• Geometry-specific parameters (mask_params, rect_params, sphere_params, box_params, aabb_params).

The Intersection Shader Pipeline

Inside main() in src/shaders/intersection-shader.ts, the shader processes splats in batches of four per fragment. The pipeline follows these stages:

1. Index Calculation – The shader computes the output texel index (outputId) and iterates over four splats using for (uint i = 0u; i < 4u; i++).

2. Splat Position Reconstruction – Each splat index maps to a UV coordinate (splatUV) in the transformA texture. The raw center is sampled and, if present, multiplied by the per-splat transform from transformPalette.

3. Coordinate Transformation – The (possibly transformed) center is multiplied by matrix_model and matrix_viewProjection to produce clip-space coordinates. After perspective division, the shader obtains normalized device coordinates (ndc).

4. Frustum Culling – The shader discards splats outside the view frustum using the test if (!any(greaterThan(abs(ndc), vec3(1.0)))).

5. Intersection Testing – Based on the mode uniform, the shader executes specific geometry tests.

Intersection Test Implementations

The shader implements four distinct selection modes, each writing 1.0 (selected) or 0.0 (rejected) into the output color vector component clr[i]:

Mask Mode (mode = 0)

The shader maps ndc coordinates to mask texture space (maskUV) and samples the alpha channel. Values < 1.0 reject the splat; otherwise, it is selected.

Rectangle Mode (mode = 1)

The shader performs a simple bounds check against rect_params. If ndc.xy lies between rect_params.xy (min) and rect_params.zw (max), the splat is selected.

Sphere Mode (mode = 2)

Using world-space coordinates, the shader calculates Euclidean distance between the splat center and sphere_params.xyz. If length(world - sphere_params.xyz) < sphere_params.w, the splat intersects the sphere.

Box Mode (mode = 3)

The shader expresses the splat center relative to box_params.xyz (origin). It then verifies that the absolute offset on each axis is less than or equal to the half-extents stored in aabb_params. Successful checks on all three axes indicate containment within the axis-aligned bounding box.

Result Extraction and CPU Processing

After the fragment shader writes to gl_FragColor, the GPU render target contains a four-component mask where each channel represents the selection state of one splat. The CPU reads this buffer as a Uint8Array and interprets any non-zero component as a selected splat ID. This data flows to src/selection.ts, which fires selection.changed events to update the application state.

Practical Usage Examples

Configure and execute selection passes using the picker API:

// Select splats inside a sphere (center = [0, 0, 0], radius = 5)
picker.setSelectionMode(2);                     // mode 2 = sphere
picker.setSphereParams([0, 0, 0, 5]);           // vec4(center, radius)
picker.runSelectionPass();                      // triggers intersection-shader.ts
// selectedSplatIds now contains intersecting splat indices

// Select splats inside an axis-aligned box
picker.setSelectionMode(3);                     // mode 3 = box
picker.setBoxParams([1, 2, 3]);                 // box origin (x, y, z)
picker.setAABBParams([0.5, 0.5, 0.5]);          // half-extents (x, y, z)
picker.runSelectionPass();

// Mask-based selection using an uploaded texture
picker.setSelectionMode(0);
picker.setMaskTexture(maskTexture);
picker.setMaskParams([maskWidth, maskHeight]);
picker.runSelectionPass();

Summary

• GPU-Parallel Processing – Selection logic resides in src/shaders/intersection-shader.ts, processing four splats per fragment in parallel.
• Four Selection Modes – Implements mask, rectangle, sphere, and axis-aligned box intersection tests via uniform-configured shader branches.
• Coordinate Pipeline – Transforms splat centers through model, view-projection, and NDC spaces before testing, with integrated frustum culling.
• Zero-Delay Feedback – Results written to gl_FragColor are read back as Uint8Array and interpreted by src/picker.ts to update src/selection.ts.

Frequently Asked Questions

Why does SuperSplat perform selection on the GPU rather than the CPU?

Processing selection on the GPU leverages parallel execution units to test millions of splats simultaneously. According to the playcanvas/supersplat source code, this avoids expensive CPU-side loops that would block the main thread when handling large point clouds, maintaining interactive frame rates during selection operations.

How does the shader handle per-splat transformations?

The shader checks for optional per-splat transforms using the splatTransform uniform. When present, it samples a 3×4 matrix from transformPalette based on the splat index and applies it to the raw center position sampled from transformA, ensuring selection tests respect individual splat orientations and scales.

What coordinate spaces are used for intersection testing?

The pipeline transforms splat centers from local space to world space using matrix_model, then to clip space via matrix_viewProjection. After perspective division, tests operate in normalized device coordinates (NDC) for screen-aligned selectors (mask, rectangle) or world space for geometric selectors (sphere, box).

How does the system handle splats outside the camera view?

The shader implements frustum culling by testing if NDC components exceed [-1, 1]. The conditional if (!any(greaterThan(abs(ndc), vec3(1.0)))) ensures that off-screen splats are rejected before expensive intersection tests execute, optimizing GPU utilization for visible geometry only.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}