How Supersplat Implements WebGL Render Passes in `render.ts`

Supersplat orchestrates WebGL render passes through the registerRenderEvents function in src/render.ts, registering three command events—render.offscreen, render.image, and render.video—that leverage PlayCanvas's off-screen camera mode, pixel readback, and the WebCodecs API for high-performance capture and encoding.

Supersplat is a Gaussian splat editor built on the PlayCanvas engine within the playcanvas/supersplat repository. Its rendering pipeline isolates capture operations from the main viewport using sophisticated WebGL render passes that handle everything from raw pixel extraction to hardware-accelerated video encoding.

Off-Screen Rendering Architecture

The foundation of Supersplat's render pass system is off-screen rendering, which isolates capture operations from the on-screen display loop. When a render command is invoked, the system initializes a separate framebuffer using scene.camera.startOffscreenMode(width, height) defined in src/scene.ts. This method creates a detached render target that allows the pipeline to toggle overlays, gizmos, and clear colors independently without affecting the visible canvas.

After rendering completes, the system calls scene.camera.endOffscreenMode() to restore the default camera configuration. This architecture ensures that screenshot and video capture never interfere with the interactive viewport while maintaining full access to the scene's rendering context.

The Three Core Render Commands

The registerRenderEvents function in src/render.ts exposes three high-level commands that UI components can invoke via the event system. Each command represents a distinct render pass optimized for specific output formats.

render.offscreen: Raw Pixel Extraction

The render.offscreen command captures raw pixel data from the current view without compression. The implementation follows this sequence:

• Buffer Creation: Invokes scene.camera.startOffscreenMode to allocate an off-screen framebuffer.
• Forced Rendering: Sets scene.forceRender = true and awaits the postrender event to ensure the frame is fully processed.
• Pixel Readback: Copies the main render target to a work target using scene.dataProcessor.copyRt(mainTarget, workTarget), then reads the color buffer into a Uint8Array via workTarget.colorBuffer.read.
• Coordinate Correction: Vertically flips the image buffer to convert from WebGL's bottom-left origin to the top-left origin expected by image formats.

This command returns the raw RGBA pixel buffer, making it ideal for further processing or custom encoding pipelines.

render.image: PNG Screenshot Generation

The render.image command builds upon the off-screen foundation to export compressed PNG screenshots. Located in the same src/render.ts file, this pass:

• Configures Transparency: Optionally disables the clear color when transparentBg is enabled, allowing alpha channel preservation.
• Compresses Output: Feeds the flipped pixel buffer into PngCompressor.compress from src/png-compressor.ts, which performs lossless compression.
• Triggers Download: Generates a filename based on the current selection state and initiates a download via downloadFile.

The command accepts parameters for resolution, transparency, and debug overlay visibility, providing a complete screenshot utility that runs entirely client-side.

render.video: Hardware-Accelerated Video Encoding

The most complex render pass, render.video, encodes frame sequences to MP4, WebM, MOV, or MKV containers using the WebCodecs API and the mediabunny library. This implementation:

• Initializes Encoder: Creates a VideoEncoder instance and Output container using lookup tables FORMAT_CONFIG and CODEC_CONFIG (lines 13–27) to select appropriate codec and container formats based on user preferences.
• Locks Render Loop: Sets scene.lockedRenderMode = true to prevent interference from the standard render cycle during batch processing.
• Processes Frames Sequentially: For each frame:
  • Advances the timeline via events.invoke('plysequence.setFrameAsync') to load PLY sequence data.
  • Sorts splats when the camera moves using instance.sort to maintain visual correctness.
  • Renders the frame, reads pixels, flips vertically, and creates a VideoFrame object fed to the encoder.
• Handles Back-Pressure: Monitors encoder.encodeQueueSize and recreates the encoder if codec reclamation occurs (lines 30–42).
• Persists Tab State: Uses navigator.locks.request when available to keep the tab alive during long encoding sessions, preventing background throttling.

After processing all frames, the encoder flushes, finalizes the container, and triggers a download if a writable file stream wasn't provided.

Pixel Processing Pipeline

All three render passes share a common pixel processing stage implemented in src/render.ts. After the off-screen render completes, the system executes scene.dataProcessor.copyRt to transfer the rendered image from the main render target to a temporary work target. This copy operation is necessary because the main target may be bound to the canvas or other pipeline stages.

The readback operation workTarget.colorBuffer.read transfers GPU memory to CPU-accessible Uint8Array. Because WebGL coordinates originate at the bottom-left while PNG and video codecs expect top-left orientation, the code performs an in-place vertical flip of the buffer before passing it to compression or encoding functions.

Auxiliary Render Pass Integration

While src/render.ts handles high-level capture passes, Supersplat implements secondary rendering effects through SimpleRenderPass located in src/utils/simple-render-pass.ts. This lightweight wrapper is utilized by modules such as src/underlay.ts, src/outline.ts, and src/picker.ts to create specialized buffers for selection highlighting, outline effects, and picking operations without duplicating the off-screen logic.

These auxiliary passes demonstrate the modular architecture: core capture logic resides in render.ts, while effects and tools build reusable render pass objects that integrate with PlayCanvas's compositor.

Practical Implementation Examples

To capture a transparent PNG screenshot with debug overlays enabled:

await events.invoke('render.image', {
  width: 1920,
  height: 1080,
  transparentBg: true,
  showDebug: true
});

To record a 10-second MP4 video at 30fps using H.264 encoding:

const fileHandle = await window.showSaveFilePicker({
  suggestedName: 'capture.mp4',
  types: [{ accept: { 'video/mp4': ['.mp4'] } }]
});
const writable = await fileHandle.createWritable();

await events.invoke('render.video', {
  startFrame: 0,
  endFrame: 300,
  frameRate: 30,
  width: 1280,
  height: 720,
  bitrate: 5_000_000,
  transparentBg: false,
  showDebug: false,
  format: 'mp4',
  codec: 'h264'
}, writable);

await writable.close();

Summary

• Supersplat implements WebGL render passes in src/render.ts through the registerRenderEvents function, exposing render.offscreen, render.image, and render.video commands.
• Off-screen rendering uses scene.camera.startOffscreenMode to isolate capture from the viewport, with pixel readback via copyRt and vertical flipping for coordinate system alignment.
• PNG export leverages PngCompressor from src/png-compressor.ts for lossless compression with optional transparency.
• Video encoding utilizes the WebCodecs API and mediabunny library, supporting multiple containers (MP4, WebM, MOV, MKV) and codecs (H.264, etc.) with back-pressure handling and Web Locks integration.
• Auxiliary effects use SimpleRenderPass from src/utils/simple-render-pass.ts for modular secondary rendering operations.

Frequently Asked Questions

How does Supersplat handle coordinate system differences when reading WebGL buffers?

Supersplat performs an explicit vertical flip of the pixel buffer after reading from workTarget.colorBuffer.read because WebGL defines the origin (0,0) at the bottom-left corner, while PNG and video codecs expect the top-left origin. This transformation occurs before compression or encoding to ensure the output image displays correctly.

What is the purpose of scene.lockedRenderMode in video rendering?

The scene.lockedRenderMode = true flag prevents the standard PlayCanvas render loop from interfering during batch video encoding. This lock ensures that frame-by-frame rendering occurs deterministically without asynchronous updates or viewport changes that could corrupt the video sequence, particularly when advancing PLY sequence frames or sorting splats per camera movement.

Can Supersplat export video with transparency?

Yes, the render.video command supports transparent backgrounds by disabling the clear color when transparentBg is set to true, similar to the PNG capture mode. However, transparency support depends on the selected codec and container format—codecs like VP9 in WebM support alpha channels, while H.264 in MP4 does not.

Where does the PNG compression logic reside outside of render.ts?

The PNG compression implementation is located in src/png-compressor.ts and exported as the PngCompressor class. The render.image command imports this module and calls PngCompressor.compress to convert raw RGBA buffers into compressed ArrayBuffer data, keeping the compression algorithm separate from the render pass orchestration logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}