# How Supersplat Implements WebGL Render Passes in `render.ts`

> Learn how Supersplat implements WebGL render passes in render.ts using PlayCanvas off-screen cameras and WebCodecs API for efficient capture and encoding.

- Repository: [PlayCanvas/supersplat](https://github.com/playcanvas/supersplat)
- Tags: how-to-guide
- Published: 2026-05-10

---

**Supersplat orchestrates WebGL render passes through the `registerRenderEvents` function in [`src/render.ts`](https://github.com/playcanvas/supersplat/blob/main/src/render.ts), registering three command events—`render.offscreen`, `render.image`, and `render.video`—that leverage PlayCanvas's off-screen camera mode, pixel readback, and the WebCodecs API for high-performance capture and encoding.**

Supersplat is a Gaussian splat editor built on the PlayCanvas engine within the `playcanvas/supersplat` repository. Its rendering pipeline isolates capture operations from the main viewport using sophisticated **WebGL render passes** that handle everything from raw pixel extraction to hardware-accelerated video encoding.

## Off-Screen Rendering Architecture

The foundation of Supersplat's render pass system is **off-screen rendering**, which isolates capture operations from the on-screen display loop. When a render command is invoked, the system initializes a separate framebuffer using `scene.camera.startOffscreenMode(width, height)` defined in [`src/scene.ts`](https://github.com/playcanvas/supersplat/blob/main/src/scene.ts). This method creates a detached render target that allows the pipeline to toggle overlays, gizmos, and clear colors independently without affecting the visible canvas.

After rendering completes, the system calls `scene.camera.endOffscreenMode()` to restore the default camera configuration. This architecture ensures that screenshot and video capture never interfere with the interactive viewport while maintaining full access to the scene's rendering context.

## The Three Core Render Commands

The `registerRenderEvents` function in [`src/render.ts`](https://github.com/playcanvas/supersplat/blob/main/src/render.ts) exposes three high-level commands that UI components can invoke via the event system. Each command represents a distinct render pass optimized for specific output formats.

### render.offscreen: Raw Pixel Extraction

The `render.offscreen` command captures raw pixel data from the current view without compression. The implementation follows this sequence:

- **Buffer Creation**: Invokes `scene.camera.startOffscreenMode` to allocate an off-screen framebuffer.
- **Forced Rendering**: Sets `scene.forceRender = true` and awaits the `postrender` event to ensure the frame is fully processed.
- **Pixel Readback**: Copies the main render target to a work target using `scene.dataProcessor.copyRt(mainTarget, workTarget)`, then reads the color buffer into a `Uint8Array` via `workTarget.colorBuffer.read`.
- **Coordinate Correction**: Vertically flips the image buffer to convert from WebGL's bottom-left origin to the top-left origin expected by image formats.

This command returns the raw RGBA pixel buffer, making it ideal for further processing or custom encoding pipelines.

### render.image: PNG Screenshot Generation

The `render.image` command builds upon the off-screen foundation to export compressed PNG screenshots. Located in the same [`src/render.ts`](https://github.com/playcanvas/supersplat/blob/main/src/render.ts) file, this pass:

- **Configures Transparency**: Optionally disables the clear color when `transparentBg` is enabled, allowing alpha channel preservation.
- **Compresses Output**: Feeds the flipped pixel buffer into `PngCompressor.compress` from [`src/png-compressor.ts`](https://github.com/playcanvas/supersplat/blob/main/src/png-compressor.ts), which performs lossless compression.
- **Triggers Download**: Generates a filename based on the current selection state and initiates a download via `downloadFile`.

The command accepts parameters for resolution, transparency, and debug overlay visibility, providing a complete screenshot utility that runs entirely client-side.

### render.video: Hardware-Accelerated Video Encoding

The most complex render pass, `render.video`, encodes frame sequences to MP4, WebM, MOV, or MKV containers using the **WebCodecs API** and the **mediabunny** library. This implementation:

- **Initializes Encoder**: Creates a `VideoEncoder` instance and `Output` container using lookup tables `FORMAT_CONFIG` and `CODEC_CONFIG` (lines 13–27) to select appropriate codec and container formats based on user preferences.
- **Locks Render Loop**: Sets `scene.lockedRenderMode = true` to prevent interference from the standard render cycle during batch processing.
- **Processes Frames Sequentially**: For each frame:
  - Advances the timeline via `events.invoke('plysequence.setFrameAsync')` to load PLY sequence data.
  - Sorts splats when the camera moves using `instance.sort` to maintain visual correctness.
  - Renders the frame, reads pixels, flips vertically, and creates a `VideoFrame` object fed to the encoder.
- **Handles Back-Pressure**: Monitors `encoder.encodeQueueSize` and recreates the encoder if codec reclamation occurs (lines 30–42).
- **Persists Tab State**: Uses `navigator.locks.request` when available to keep the tab alive during long encoding sessions, preventing background throttling.

After processing all frames, the encoder flushes, finalizes the container, and triggers a download if a writable file stream wasn't provided.

## Pixel Processing Pipeline

All three render passes share a common pixel processing stage implemented in [`src/render.ts`](https://github.com/playcanvas/supersplat/blob/main/src/render.ts). After the off-screen render completes, the system executes `scene.dataProcessor.copyRt` to transfer the rendered image from the main render target to a temporary work target. This copy operation is necessary because the main target may be bound to the canvas or other pipeline stages.

The readback operation `workTarget.colorBuffer.read` transfers GPU memory to CPU-accessible `Uint8Array`. Because WebGL coordinates originate at the bottom-left while PNG and video codecs expect top-left orientation, the code performs an in-place vertical flip of the buffer before passing it to compression or encoding functions.

## Auxiliary Render Pass Integration

While [`src/render.ts`](https://github.com/playcanvas/supersplat/blob/main/src/render.ts) handles high-level capture passes, Supersplat implements secondary rendering effects through **`SimpleRenderPass`** located in [`src/utils/simple-render-pass.ts`](https://github.com/playcanvas/supersplat/blob/main/src/utils/simple-render-pass.ts). This lightweight wrapper is utilized by modules such as [`src/underlay.ts`](https://github.com/playcanvas/supersplat/blob/main/src/underlay.ts), [`src/outline.ts`](https://github.com/playcanvas/supersplat/blob/main/src/outline.ts), and [`src/picker.ts`](https://github.com/playcanvas/supersplat/blob/main/src/picker.ts) to create specialized buffers for selection highlighting, outline effects, and picking operations without duplicating the off-screen logic.

These auxiliary passes demonstrate the modular architecture: core capture logic resides in [`render.ts`](https://github.com/playcanvas/supersplat/blob/main/render.ts), while effects and tools build reusable render pass objects that integrate with PlayCanvas's compositor.

## Practical Implementation Examples

To capture a transparent PNG screenshot with debug overlays enabled:

```typescript
await events.invoke('render.image', {
  width: 1920,
  height: 1080,
  transparentBg: true,
  showDebug: true
});

```

To record a 10-second MP4 video at 30fps using H.264 encoding:

```typescript
const fileHandle = await window.showSaveFilePicker({
  suggestedName: 'capture.mp4',
  types: [{ accept: { 'video/mp4': ['.mp4'] } }]
});
const writable = await fileHandle.createWritable();

await events.invoke('render.video', {
  startFrame: 0,
  endFrame: 300,
  frameRate: 30,
  width: 1280,
  height: 720,
  bitrate: 5_000_000,
  transparentBg: false,
  showDebug: false,
  format: 'mp4',
  codec: 'h264'
}, writable);

await writable.close();

```

## Summary

- **Supersplat** implements WebGL render passes in [`src/render.ts`](https://github.com/playcanvas/supersplat/blob/main/src/render.ts) through the `registerRenderEvents` function, exposing `render.offscreen`, `render.image`, and `render.video` commands.
- **Off-screen rendering** uses `scene.camera.startOffscreenMode` to isolate capture from the viewport, with pixel readback via `copyRt` and vertical flipping for coordinate system alignment.
- **PNG export** leverages `PngCompressor` from [`src/png-compressor.ts`](https://github.com/playcanvas/supersplat/blob/main/src/png-compressor.ts) for lossless compression with optional transparency.
- **Video encoding** utilizes the WebCodecs API and mediabunny library, supporting multiple containers (MP4, WebM, MOV, MKV) and codecs (H.264, etc.) with back-pressure handling and Web Locks integration.
- **Auxiliary effects** use `SimpleRenderPass` from [`src/utils/simple-render-pass.ts`](https://github.com/playcanvas/supersplat/blob/main/src/utils/simple-render-pass.ts) for modular secondary rendering operations.

## Frequently Asked Questions

### How does Supersplat handle coordinate system differences when reading WebGL buffers?

Supersplat performs an explicit vertical flip of the pixel buffer after reading from `workTarget.colorBuffer.read` because WebGL defines the origin (0,0) at the bottom-left corner, while PNG and video codecs expect the top-left origin. This transformation occurs before compression or encoding to ensure the output image displays correctly.

### What is the purpose of `scene.lockedRenderMode` in video rendering?

The `scene.lockedRenderMode = true` flag prevents the standard PlayCanvas render loop from interfering during batch video encoding. This lock ensures that frame-by-frame rendering occurs deterministically without asynchronous updates or viewport changes that could corrupt the video sequence, particularly when advancing PLY sequence frames or sorting splats per camera movement.

### Can Supersplat export video with transparency?

Yes, the `render.video` command supports transparent backgrounds by disabling the clear color when `transparentBg` is set to `true`, similar to the PNG capture mode. However, transparency support depends on the selected codec and container format—codecs like VP9 in WebM support alpha channels, while H.264 in MP4 does not.

### Where does the PNG compression logic reside outside of [`render.ts`](https://github.com/playcanvas/supersplat/blob/main/render.ts)?

The PNG compression implementation is located in [`src/png-compressor.ts`](https://github.com/playcanvas/supersplat/blob/main/src/png-compressor.ts) and exported as the `PngCompressor` class. The `render.image` command imports this module and calls `PngCompressor.compress` to convert raw RGBA buffers into compressed `ArrayBuffer` data, keeping the compression algorithm separate from the render pass orchestration logic.