# How Hyperframes Handles HDR Video Capture and Encoding (HDR10, PQ)

> Learn how Hyperframes handles HDR video capture and encoding including HDR10 and PQ. Discover end-to-end HDR10 video production with Puppeteer, FFmpeg, and HEVC.

- Repository: [HeyGen/hyperframes](https://github.com/heygen-com/hyperframes)
- Tags: deep-dive
- Published: 2026-05-17

---

**Hyperframes implements end-to-end HDR10 video production by combining Puppeteer-controlled canvas capture in the BT.2020 color space with FFmpeg HEVC encoding using SMPTE 2084 (PQ) transfer characteristics and 10-bit color depth.**

Hyperframes, the open-source video generation engine from HeyGen, enables high-dynamic-range (HDR) video capture and encoding through a specialized HDR-IO pipeline. The system captures raw radiance data as 16-bit PNG frames from a headless browser and transcodes them into standards-compliant HDR10 MP4 files using hardware-accelerated HEVC encoding.

## The HDR-IO Pipeline Architecture

The HDR workflow in Hyperframes operates as a four-stage pipeline managed through the `@hyperframes/producer` and `@hyperframes/engine` packages:

1. **Composition Configuration**: The `HDRIOBlock` mutates the rendering context to enable HDR mode
2. **Frame Capture**: Puppeteer renders content to a BT.2020 canvas and exports 16-bit PNGs
3. **Video Encoding**: FFmpeg processes raw frames into HEVC with HDR10 metadata
4. **Playback**: The native player streams the HDR-enabled MP4 via standard `<video>` elements

## Configuring HDR Mode with HDRIOBlock

To initiate HDR capture, compositions include the `hdr-io` block defined in [`packages/producer/src/blocks/hdr-io.ts`](https://github.com/heygen-com/hyperframes/blob/main/packages/producer/src/blocks/hdr-io.ts). This block instantiates an `HDRIOBlock` class that mutates the `RenderContext` via `ctx.setHDRMode()`, accepting either `pq` (SMPTE 2084) or `hlg` (Hybrid Log-Gamma) as the **EOTF** (Electro-Optical Transfer Function).

The block also injects optional **SMPTE-2086** master-display metadata into the context. In [`packages/engine/src/context.ts`](https://github.com/heygen-com/hyperframes/blob/main/packages/engine/src/context.ts), the `RenderContext` interface exposes `hdrMode` and `masterDisplay` properties that downstream processes reference during capture and encoding.

```json
{
  "blocks": [
    {
      "type": "hdr-io",
      "mode": "pq",
      "masterDisplay": {
        "redX": 34000,
        "redY": 16000,
        "greenX": 13250,
        "greenY": 34500,
        "blueX": 7500,
        "blueY": 3000,
        "whiteX": 15635,
        "whiteY": 16450,
        "maxLuminance": 1000,
        "minLuminance": 0
      }
    }
  ]
}

```

## Capturing 16-Bit PNG Frames from Headless Chrome

The capture mechanism in [`packages/engine/src/capture.ts`](https://github.com/heygen-com/hyperframes/blob/main/packages/engine/src/capture.ts) creates a Puppeteer-controlled page with a high-bit-depth canvas context. Rather than using standard 8-bit RGB, the pipeline initializes the canvas with `colorSpace: 'display-p3'` (representing BT.2020 primaries) and requests 16-bit per channel output to preserve HDR luminance values without tone-mapping.

The `captureFrames` function screenshots the canvas element into raw PNG buffers. These intermediates retain the full dynamic range of the source content, avoiding the crushing of highlights or loss of shadow detail that occurs in SDR conversion. Files are written as 16-bit PNGs to disk before entering the encoding queue.

```javascript
// Canvas initialization for HDR capture
const canvas = await page.evaluateHandle(() => {
  const c = document.createElement('canvas');
  c.width = 1920;
  c.height = 1080;
  const ctx = c.getContext('2d', {
    colorSpace: 'display-p3',
    pixelFormat: 'float16'  // Ensures 16-bit depth retention
  });
  return c;
});

```

## Encoding HDR10 with FFmpeg

The `ffmpegEncode` function in [`packages/engine/src/ffmpeg.ts`](https://github.com/heygen-com/hyperframes/blob/main/packages/engine/src/ffmpeg.ts) processes the 16-bit PNG sequence into a standards-compliant HDR10 MP4. The implementation spawns FFmpeg with `libx265` and injects precise color metadata required for HDR10 playback compatibility.

Key encoding parameters include:
- **`-pix_fmt yuv420p10le`**: Forces 10-bit 4:2:0 chroma subsampling
- **`-color_primaries bt2020`**: Declares BT.2020 color gamut
- **`-color_trc smpte2084`**: Sets PQ (SMPTE 2084) transfer curve
- **`-colorspace bt2020nc`**: Specifies BT.2020 non-constant luminance matrix
- **`-x265-params`**: Configures full-range quantization and 10-bit internal precision
- **`-metadata:s:v:0 master-display=...`**: Embeds SMPTE-2086 display primaries and luminance data

```bash
ffmpeg -framerate 30 -i pipe:0 \
  -c:v libx265 \
  -pix_fmt yuv420p10le \
  -color_primaries bt2020 \
  -color_trc smpte2084 \
  -colorspace bt2020nc \
  -x265-params "colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:range=full" \
  -metadata:s:v:0 "master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,0)" \
  output.mp4

```

## Native HDR Playback in the Player

Playback requires no special handling beyond serving the generated MP4. The `@hyperframes/player` package in [`packages/player/src/video.ts`](https://github.com/heygen-com/hyperframes/blob/main/packages/player/src/video.ts) renders the video through a standard HTML5 `<video>` element. Modern browsers automatically detect the HDR10 metadata flags and switch the display into HDR mode when available hardware supports the BT.2020 gamut and PQ curve.

## Summary

- **`HDRIOBlock`** configures the render context with PQ or HLG modes and optional SMPTE-2086 metadata
- **Puppeteer captures** 16-bit PNG frames in the BT.2020 color space without tone-mapping
- **FFmpeg encodes** to HEVC using 10-bit precision and embedded HDR10 mastering display metadata
- **Native browser playback** automatically triggers HDR mode on compatible displays when video tags carry the correct color characteristics

## Frequently Asked Questions

### What HDR formats does Hyperframes currently support?

Hyperframes currently supports **HDR10** using the **PQ** (Perceptual Quantizer, SMPTE 2084) transfer function. You configure this via the `mode: 'pq'` property in the `HDRIOBlock`. Support for **HLG** (Hybrid Log-Gamma) is architecturally supported through the same block interface but requires explicit future implementation in the encoding pipeline.

### Why does the pipeline use 16-bit PNG intermediates instead of video-native formats?

The 16-bit PNG intermediates preserve **raw radiance data** without tone-mapping between the browser capture and final encode. This prevents clipping of highlight details above the SDR threshold (100 nits) and ensures the **BT.2020** color gamut remains intact throughout the compositing process. Using lossless PNGs also allows multiple rendering passes without generational quality loss.

### How can I verify that my output video contains correct HDR10 metadata?

Run `ffprobe` on the generated MP4 and inspect the video stream metadata. Confirm that `color_primaries` equals `bt2020`, `color_trc` equals `smpte2084`, and `colorspace` equals `bt2020nc`. Additionally, verify the presence of the `master-display` metadata tag, which carries the SMPTE-2086 mastering display color volume information required for accurate HDR tone-mapping on consumer displays.

### Can I override the default HDR color space or master display parameters?

Yes. The `ExportConfig` interface supports optional overrides including `forceColorPrimaries`, `forceColorTransfer`, and `overrideMasterDisplay`. When provided, the `ffmpegEncode` function in [`packages/engine/src/ffmpeg.ts`](https://github.com/heygen-com/hyperframes/blob/main/packages/engine/src/ffmpeg.ts) prefers these user-supplied values over the defaults derived from the `RenderContext`, allowing custom HDR grading workflows or compatibility adjustments for specific delivery targets.