How FFmpeg Conversions Are Handled in the Browser in the p2r3/convert Project

The p2r3/convert project runs FFmpeg conversions entirely client-side using WebAssembly (WASM), loading the FFmpeg core into the browser's memory and executing commands on a virtual filesystem without server-side processing.

The convert web application leverages the @ffmpeg/ffmpeg WebAssembly build to perform video, audio, and image conversions directly within the browser sandbox. By utilizing an in-memory virtual filesystem and a robust error-recovery wrapper, the project eliminates server-side processing while maintaining format compatibility through dynamic discovery of FFmpeg's capabilities.

Loading the FFmpeg WASM Core

When the application initializes, the FFmpegHandler class in src/handlers/FFmpeg.ts creates a new FFmpeg instance and loads the core script from the local path /convert/wasm/ffmpeg-core.js.

// src/handlers/FFmpeg.ts – initialization routine
this.#ffmpeg = new FFmpeg();
await this.loadFFmpeg();  // Loads ffmpeg-core.js from /convert/wasm/

The loadFFmpeg method (lines 33-38) downloads the WASM binary once per page load. After successful initialization, the handler sets ready = true, signaling that the instance can accept conversion requests. This singleton-like pattern ensures that the relatively expensive WASM load occurs only when necessary, while subsequent operations can reuse the initialized environment.

Discovering Supported Formats

Before presenting options to the user, the handler executes ffmpeg -formats to introspect the WASM build's capabilities. During the init() routine (lines 83-112), the output is parsed to populate the supportedFormats array with FileFormat objects containing MIME types, extensions, and codec information.

The discovered formats are cached globally in window.supportedFormatCache by src/main.ts via the buildOptionList() function. This cache prevents redundant format discovery across multiple conversion sessions and provides the UI with immediate access to valid input/output combinations.

Executing Browser-Based Conversions

The actual conversion workflow is orchestrated through the doConvert() method in FFmpegHandler. When a user triggers a conversion via the UI, src/main.ts resolves the conversion path and invokes this method, which implements a six-stage pipeline entirely within the browser.

Preparing the Virtual Filesystem

First, doConvert() ensures a fresh FFmpeg instance by calling await this.reloadFFmpeg() (line 124). This prevents memory leaks and state pollution from previous runs. The method then writes each input file into the virtual filesystem using ffmpeg.writeFile, generating a list.txt file for the FFmpeg concat demuxer when processing multiple inputs (lines 131-140).

// Writing inputs to the virtual filesystem
for (const file of inputFiles) {
  await this.#ffmpeg!.writeFile(file.name, file.bytes);
}
// Generate concat list for multiple inputs
await this.#ffmpeg!.writeFile("list.txt", concatListContent);

Building and Running the FFmpeg Command

The handler constructs a command array based on the target output format and optional user-supplied arguments (lines 145-151). For example, MP4 outputs may automatically include -pix_fmt yuv420p for compatibility. The command executes via ffmpeg.exec wrapped in the execSafe utility (lines 58-81), which adds timeout protection and handles the infamous "out of bounds" memory errors common in WASM environments.

// src/handlers/FFmpeg.ts – execSafe implementation
async execSafe(args: string[], timeout = -1, attempts = 1): Promise<void> {
  if (!this.#ffmpeg) throw "Handler not initialized.";
  try {
    if (timeout === -1) await this.#ffmpeg.exec(args);
    else await Promise.race([
      this.#ffmpeg.exec(args, timeout),
      new Promise((_, reject) => setTimeout(reject, timeout))
    ]);
  } catch (e) {
    // Auto-restart on memory errors and retry
    if (!e || (typeof e === "string" && e.includes("out of bounds") && attempts > 1)) {
      await this.reloadFFmpeg();
      return this.execSafe(args, timeout, attempts - 1);
    }
    throw e;
  }
}

Error Handling and Auto-Recovery

If FFmpeg exits with an error, the handler inspects stdout for common issues such as "dimension not divisible by X" (lines 162-176). When detected, the conversion automatically retries with corrective filters like -vf or -s flags to adjust dimensions, requiring no user intervention.

Extracting the Output

Upon successful completion, the output file is read from the virtual filesystem using ffmpeg.readFile, converted to a Uint8Array, and temporary files are deleted (lines 180-207). The method returns an array of {bytes, name} objects to the caller.

Integration with the UI Layer

The UI layer in src/main.ts bridges user interactions with the FFmpeg handler. When the convert button is clicked, the application resolves the conversion path—potentially traversing multiple handlers via src/TraversionGraph.ts—and ultimately calls FFmpegHandler.doConvert().

// src/main.ts – simplified conversion trigger
ui.convertButton.onclick = async () => {
  const inputOption = allOptions[Number(inputButton.getAttribute("format-index"))];
  const outputOption = allOptions[Number(outputButton.getAttribute("format-index"))];
  
  const files = await Promise.all(selectedFiles.map(async f => ({
    name: f.name,
    bytes: new Uint8Array(await f.arrayBuffer())
  })));
  
  const result = await window.tryConvertByTraversing(files, inputOption, outputOption);
  // Trigger download of result.files
};

The tryConvertByTraversing function determines whether FFmpeg can handle the direct conversion or if intermediate steps are required, making FFmpeg one node in a larger conversion graph while keeping all processing client-side.

Summary

• Pure Client-Side Execution: The p2r3/convert project uses @ffmpeg/ffmpeg WASM builds to run FFmpeg entirely in the browser without server resources.
• Virtual Filesystem: Files are written to an in-memory virtual filesystem via ffmpeg.writeFile, with outputs retrieved via ffmpeg.readFile.
• Robust Error Handling: The execSafe wrapper in src/handlers/FFmpeg.ts automatically restarts FFmpeg on memory errors and retries failed conversions with corrective filters.
• Dynamic Format Discovery: Supported formats are discovered at runtime using ffmpeg -formats and cached in window.supportedFormatCache for UI consumption.
• Fresh Instances: Each conversion starts with reloadFFmpeg() to prevent state pollution, ensuring reliable execution for each job.

Frequently Asked Questions

What WebAssembly build does the convert project use for FFmpeg?

The project uses the official @ffmpeg/ffmpeg npm package with a custom core script loaded from /convert/wasm/ffmpeg-core.js. This WASM build runs entirely within the browser's sandboxed environment, executing FFmpeg commands on a virtual in-memory filesystem without sending data to external servers.

How does the convert project handle FFmpeg memory errors in the browser?

The execSafe method in src/handlers/FFmpeg.ts (lines 58-81) implements automatic recovery for "out of bounds" memory errors common in WASM environments. When such an error occurs, the handler automatically restarts the FFmpeg instance via reloadFFmpeg() and retries the conversion with the remaining attempt count, transparently handling memory limitations without user intervention.

Can the convert project handle multiple input files in a single conversion?

Yes. The doConvert() method generates a list.txt file using FFmpeg's concat demuxer protocol when processing multiple inputs. Each input file is written to the virtual filesystem individually, then referenced in the concatenation list before executing the conversion command, allowing batch processing entirely client-side.

Where does the convert project store converted files before download?

Converted files exist only in the browser's memory within FFmpeg's virtual filesystem. After ffmpeg.exec completes successfully, doConvert() reads the output file using ffmpeg.readFile, returns the bytes as a Uint8Array, and immediately cleans up temporary files. The resulting data is then converted to a Blob for the download trigger, ensuring no persistent server storage is used.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}