How the Lighthouse GitHub Actions CI/CD Pipeline Extracts Assets Without Storing ROMs

The Lighthouse CI/CD pipeline extracts game assets from ROMs during the GitHub Actions workflow without ever persisting the original ROM files in build artifacts, using a custom torch utility that reads ROMs ephemerally and produces a distributable .o2r archive.

The HarbourMasters/Lighthouse project solves a critical legal and technical challenge: building a game port that requires copyrighted assets while ensuring those assets—and especially the original ROMs—never appear in public artifacts. This article examines exactly how the .github/workflows/main.yml pipeline achieves this separation, with complete technical details sourced from the repository's implementation.

The Core Challenge: Assets Without ROM Exposure

Game engine reimplementations like Lighthouse need textures, shaders, level data, and other proprietary assets to function. Distributing ROMs violates copyright law and Nintendo's terms of service. The Lighthouse solution treats ROMs as ephemeral build inputs, extracting only the necessary data and discarding the original files before any artifacts are created.

Step 1: Building the torch Extraction Tool

The workflow begins on an Ubuntu runner by compiling the custom torch utility. This binary is purpose-built to parse ROM formats and extract specific asset types without loading the entire file into persistent storage.

- name: Generate lighthouse.o2r
  run: |
    export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH"
    make -C Torch type=release -j3

The make -C Torch type=release -j3 command (lines 41–44 in .github/workflows/main.yml) builds Torch/cmake-build-release/torch from the Torch submodule source. The torch tool understands the specific ROM format used by the target game and can locate asset tables, compressed textures, and serialized level data within the binary.

Step 2: Preparing Non-ROM Assets

Before ROM extraction, the workflow stages assets that ship with the port but originate from the open-source repository itself.

cp -r ./libultraship/src/fast/shaders ./port

This command (line 44) copies GPU shader source files from libultraship/src/fast/shaders into a temporary port/ directory. These shaders are part of the final asset bundle but are developed and licensed separately from the proprietary game content.

Step 3: Ephemeral ROM Processing with torch pack

The critical security boundary occurs at the torch pack invocation:

Torch/cmake-build-release/torch pack port lighthouse.o2r o2r -u ${{ steps.extract_version.outputs.version }}

This single command (line 45) performs three operations atomically:

• Reads the ROM from the runner's ephemeral workspace without loading it into environment variables or logs
• Extracts only required assets (textures, models, audio samples, level geometry) based on internal manifest definitions
• Writes lighthouse.o2r containing extracted assets plus the ./port directory contents
• Discards ROM data—the original file is never opened for reading again and is excluded from the output archive

The .o2r format (a custom archive format used by the Shipwright/libultraship ecosystem) contains only the decompressed, reformatted assets needed at runtime—not ROM headers, executable code, or copyrighted BIOS data.

Step 4: Artifact Distribution Without ROM Contamination

The workflow uploads only the extracted asset archive:

- uses: actions/upload-artifact@v4
  with:
    name: lighthouse.o2r
    path: lighthouse.o2r
    retention-days: 1

With retention-days: 1 (lines 46–50), even this limited asset bundle has minimal persistence. Downstream platform builds download lighthouse.o2r via actions/download-artifact@v4 and bundle it with compiled executables:

- name: Download lighthouse.o2r
  uses: actions/download-artifact@v4
  with:
    name: lighthouse.o2r
    path: ./build/x64

The ROM itself existed only in the Ubuntu runner's temporary filesystem during the torch pack execution window. No workflow step copies, caches, or logs the ROM filename or contents.

Technical Safeguards in the Pipeline Design

Several architectural decisions reinforce the extract assets without storing ROMs principle:

• Single-purpose tool: torch is compiled fresh per-run, ensuring extraction logic matches the current codebase
• Ephemeral workspace: GitHub Actions runners are destroyed after job completion, eliminating residual filesystem data
• Explicit artifact enumeration: Only lighthouse.o2r appears in upload-artifact paths—no wildcards that could capture unexpected files
• No cross-job ROM passing: The ROM is never passed as an artifact; only processed outputs flow downstream

Comparison with Alternative Approaches

Manual asset extraction requires users to own and process ROMs locally, complicating installation.

Server-side asset hosting would require Lighthouse to distribute copyrighted material directly.

The ephemeral extraction pipeline used in HarbourMasters/Lighthouse keeps the project legally compliant while providing a seamless build process: users supply their own ROMs, the infrastructure processes them transiently, and releases contain only legally redistributable components.

Summary

• The torch utility in the Torch/ submodule performs ROM-specific asset extraction without preserving the original file.
• The .github/workflows/main.yml pipeline builds torch on-demand, runs torch pack to generate lighthouse.o2r, and uploads only that archive.
• ROM data never enters GitHub Actions artifacts, caches, or logs—remaining strictly in the runner's ephemeral filesystem.
• Downstream platform jobs receive only the processed .o2r asset bundle, ensuring distributed builds contain no copyrighted ROM content.

Frequently Asked Questions

What is the .o2r file format used by Lighthouse?

The .o2r format is a custom archive format developed for the Shipwright/libultraship engine ecosystem. It packages extracted game assets—textures, audio, level data, and shaders—into a single file that the engine can mount at runtime. Unlike raw ROM dumps, .o2r files contain only the specific resources the reimplementation requires, stripped of proprietary executable code and headers.

How does torch know which assets to extract from the ROM?

The torch tool contains hardcoded knowledge of the target game's ROM layout, including filesystem offsets, compression algorithms, and asset table locations. This mapping is maintained in the Torch submodule source code and compiled into the binary during the CI workflow. When invoked with pack, torch traverses these known structures and extracts only the file types the Lighthouse engine expects.

Why build torch in CI instead of using a precompiled binary?

Building torch from source ensures the extraction logic stays synchronized with the current codebase commit. As the Torch submodule receives updates—supporting new asset types or fixing decompression bugs—the CI pipeline automatically incorporates those changes. This eliminates version skew between the extraction tool and the engine that consumes its output.

Can the ROM be recovered from a lighthouse.o2r artifact?

No. The .o2r archive contains decompressed, often converted asset formats that discard the original ROM's binary structure. Headers, padding, executable code, and copyright-protected system libraries are excluded by design. Reconstructing the original ROM from extracted assets would require solving an intentionally lossy compression and format conversion process.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}