How SH (Spherical Harmonics) Colors Are Decoded and Processed in Supersplat

Supersplat decodes quantized spherical harmonic coefficients stored in GPU textures using custom bit-unpacking in shaders, evaluates them against the view direction to compute lighting contributions, and supports CPU-side rotation when splats are transformed.

The PlayCanvas Supersplat viewer stores per-splat lighting information as compressed spherical harmonic (SH) coefficients to enable view-dependent color rendering. Understanding how these SH colors are decoded and processed reveals the technical pipeline behind realistic Gaussian splat illumination. This article examines the complete flow from packed texture storage through shader evaluation to CPU-side coefficient rotation.

How SH Coefficients Are Packed for GPU Storage

Supersplat stores the lighting contribution of each splat as quantized spherical-harmonic coefficients in GPU textures. The packing scheme uses four 11-bit signed integers plus a 32-bit floating-point scale value per texel. Depending on the number of bands configured (SH_BANDS), the system stores three coefficients (for band 1) or up to fifteen coefficients (for band 3) across multiple textures named splatSH_1to3, splatSH_4to7, and so forth.

The texture creation and format setup occurs in src/splat.ts, where the engine initializes these data sources with PIXELFORMAT settings capable of holding the packed 11-10-11 bit layout. This compact representation allows efficient storage while preserving sufficient precision for lighting calculations.

Fetching and Unpacking SH Data in Shaders

The decoding process begins in the fragment and vertex shaders defined in src/shaders/splat-overlay-shader.ts. The readSHData function (with multiple overloads for different band counts) orchestrates the retrieval process.

First, the shader calls unpack111011s to convert the packed bits back to a vec3 in the range [-1, 1]. This helper operates on the raw texture data sampled from the splatSH_* textures. For example, lines 47-51 of splat-overlay-shader.ts handle the bit manipulation required to extract the three coefficient vectors.

Next, fetchScale extracts the floating-point scale factor using uintBitsToFloat(t.x) and retrieves the three coefficient vectors a, b, and c (lines 52-58). For higher SH bands (when SH_BANDS equals 2 or 3), additional coefficients are fetched using fetchSH and fetchSH1, which rely on the same unpack111011s routine.

Evaluating SH Lighting for the View Direction

After unpacking, the shaders in src/shaders/splat-shader.ts compute the model-space view direction to evaluate the lighting contribution. During the forward pass, the shader calculates dir as the normalized view direction transformed by the splat's model-view matrix.

The function evalSH(sh, dir)—included from the gsplatEvalSHVS shader chunk—evaluates the SH series up to order 3 and returns a color contribution. This result is then multiplied by the previously fetched scale value and added to the Gaussian's base color:

// In src/shaders/splat-shader.ts (FORWARD_PASS block)
#if SH_BANDS > 0
    // view direction in model space
    vec3 dir = normalize(center.view * mat3(center.modelView));

    // read packed SH coefficients and scale
    vec3 sh[SH_COEFFS];
    float scale;
    readSHData(sh, scale);

    // evaluate the SH series and add its contribution
    color.xyz += evalSH(sh, dir) * scale;
#endif

Additional post-processing—including tint, brightness, saturation, and tonemapping—occurs after this SH term is applied.

Rotating SH Coefficients on the CPU

When a splat’s orientation changes, the stored SH coefficients must rotate to remain aligned with the splat’s local space. Supersplat handles this transformation on the CPU using the SHRotation utility class in src/sh-utils.ts.

The implementation pre-computes rotation matrices for up to band 3 based on Andrew Willmott’s sh-lib. The constructor builds rotation factors for each band, storing them for efficient reuse. When rotation is required, the apply method (lines 151-188) performs dot-product operations (dp) to transform the source coefficients into the destination array:

import { Mat3 } from 'playcanvas';
import { SHRotation } from './sh-utils';

// `rotMat` is a 3×3 rotation matrix derived from the splat's orientation
const rotMat = new Mat3();
// … set rotMat from a quaternion or Euler angles …

// create the rotation helper (pre‑computes the band‑wise rotation matrices)
const shRot = new SHRotation(rotMat);

// `srcCoeffs` is a Float32Array with 3, 8 or 15 SH coefficients
const srcCoeffs = new Float32Array([...]);

// Destination array can be the same as `srcCoeffs` (in‑place)
shRot.apply(srcCoeffs);   // after this, `srcCoeffs` holds the rotated SH values

This rotation executes once per splat when its transform palette updates, ensuring the GPU receives already-rotated coefficients for rendering.

Complete Decoding Pipeline Example

The following GLSL snippet demonstrates the full shader-side decoding flow used in the overlay shaders:

// From src/shaders/splat-overlay-shader.ts
vec3 sh[SH_COEFFS];
float scale;

// Fetch and unpack based on SH_BANDS count
#if SH_BANDS == 1
    fetchScale(sh, scale);
#elif SH_BANDS == 2
    fetchScale(sh, scale);
    fetchSH(sh, 3);
#elif SH_BANDS == 3
    fetchScale(sh, scale);
    fetchSH(sh, 3);
    fetchSH1(sh, 8);
#endif

// Evaluate for current view direction
vec3 modelViewDir = normalize(center.view * mat3(center.modelView));
vec3 shColor = evalSH(sh, modelViewDir) * scale;

Summary

• Storage Format: SH coefficients are quantized to 11-bit signed integers with a 32-bit float scale, stored across multiple splatSH_* textures.
• Shader Decoding: The readSHData function in splat-overlay-shader.ts uses unpack111011s to convert packed bits back to floating-point vectors in the [-1, 1] range.
• Lighting Evaluation: The evalSH function evaluates spherical harmonics against the model-space view direction, with results scaled and added to base color in splat-shader.ts.
• CPU Rotation: The SHRotation class in sh-utils.ts pre-computes rotation matrices and applies them via dot-product operations when splats transform.

Frequently Asked Questions

What bit format does Supersplat use for SH coefficient packing?

Supersplat packs each SH coefficient using 11-bit signed integers stored alongside a 32-bit floating-point scale value. The unpack111011s function in src/shaders/splat-overlay-shader.ts decodes these packed bits back into vec3 values ranging from -1 to 1, while fetchScale extracts the floating-point multiplier using uintBitsToFloat.

How does the shader handle different SH band counts?

The shader uses conditional compilation based on the SH_BANDS define (set to 1, 2, or 3). The readSHData function has multiple overloads generated in splat-overlay-shader.ts (lines 72-88) that fetch the appropriate number of coefficients—3 for band 1, 8 for band 2, or 15 for band 3—ensuring only necessary texture reads execute.

When are SH coefficients rotated on the CPU versus GPU?

Rotation occurs exclusively on the CPU using the SHRotation class in src/sh-utils.ts. When a splat’s orientation changes, the apply method rotates the coefficients once per splat before they reach the GPU. The shaders always receive pre-rotated coefficients, eliminating the need for per-pixel rotation math in the fragment shader.

Where does the final SH color contribution combine with the splat base color?

In src/shaders/splat-shader.ts, the forward pass computes the SH evaluation result and adds it directly to the base color: color.xyz += evalSH(sh, dir) * scale; (lines 21-23). This occurs before post-processing effects like tonemapping and saturation adjustments, allowing the spherical harmonic lighting to influence the final splat appearance naturally.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}