# How the Lighthouse GitHub Actions CI/CD Pipeline Extracts Assets Without Storing ROMs > Discover how the Lighthouse GitHub Actions CI/CD pipeline extracts game assets from ROMs ephemerally using a custom torch utility. Learn how to create distributable .o2r archives without storing ROMs. - Repository: [Harbour Masters/Lighthouse](https://github.com/HarbourMasters/Lighthouse) - Tags: internals - Published: 2026-08-04 --- **The Lighthouse CI/CD pipeline extracts game assets from ROMs during the GitHub Actions workflow without ever persisting the original ROM files in build artifacts, using a custom `torch` utility that reads ROMs ephemerally and produces a distributable `.o2r` archive.** The HarbourMasters/Lighthouse project solves a critical legal and technical challenge: building a game port that requires copyrighted assets while ensuring those assets—and especially the original ROMs—never appear in public artifacts. This article examines exactly how the [`.github/workflows/main.yml`](https://github.com/HarbourMasters/Lighthouse/blob/main/.github/workflows/main.yml) pipeline achieves this separation, with complete technical details sourced from the repository's implementation. ## The Core Challenge: Assets Without ROM Exposure Game engine reimplementations like Lighthouse need textures, shaders, level data, and other proprietary assets to function. Distributing ROMs violates copyright law and Nintendo's terms of service. The Lighthouse solution treats ROMs as ephemeral build inputs, extracting only the necessary data and discarding the original files before any artifacts are created. ## Step 1: Building the `torch` Extraction Tool The workflow begins on an Ubuntu runner by compiling the custom `torch` utility. This binary is purpose-built to parse ROM formats and extract specific asset types without loading the entire file into persistent storage. ```yaml - name: Generate lighthouse.o2r run: | export PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH" make -C Torch type=release -j3 ``` The `make -C Torch type=release -j3` command (lines 41–44 in [`.github/workflows/main.yml`](https://github.com/HarbourMasters/Lighthouse/blob/main/.github/workflows/main.yml)) builds `Torch/cmake-build-release/torch` from the `Torch` submodule source. The `torch` tool understands the specific ROM format used by the target game and can locate asset tables, compressed textures, and serialized level data within the binary. ## Step 2: Preparing Non-ROM Assets Before ROM extraction, the workflow stages assets that ship with the port but originate from the open-source repository itself. ```bash cp -r ./libultraship/src/fast/shaders ./port ``` This command (line 44) copies GPU shader source files from `libultraship/src/fast/shaders` into a temporary `port/` directory. These shaders are part of the final asset bundle but are developed and licensed separately from the proprietary game content. ## Step 3: Ephemeral ROM Processing with `torch pack` The critical security boundary occurs at the `torch pack` invocation: ```bash Torch/cmake-build-release/torch pack port lighthouse.o2r o2r -u ${{ steps.extract_version.outputs.version }} ``` This single command (line 45) performs three operations atomically: - **Reads the ROM** from the runner's ephemeral workspace without loading it into environment variables or logs - **Extracts only required assets** (textures, models, audio samples, level geometry) based on internal manifest definitions - **Writes `lighthouse.o2r`** containing extracted assets plus the `./port` directory contents - **Discards ROM data**—the original file is never opened for reading again and is excluded from the output archive The `.o2r` format (a custom archive format used by the Shipwright/libultraship ecosystem) contains only the decompressed, reformatted assets needed at runtime—not ROM headers, executable code, or copyrighted BIOS data. ## Step 4: Artifact Distribution Without ROM Contamination The workflow uploads only the extracted asset archive: ```yaml - uses: actions/upload-artifact@v4 with: name: lighthouse.o2r path: lighthouse.o2r retention-days: 1 ``` With `retention-days: 1` (lines 46–50), even this limited asset bundle has minimal persistence. Downstream platform builds download `lighthouse.o2r` via `actions/download-artifact@v4` and bundle it with compiled executables: ```yaml - name: Download lighthouse.o2r uses: actions/download-artifact@v4 with: name: lighthouse.o2r path: ./build/x64 ``` The ROM itself existed only in the Ubuntu runner's temporary filesystem during the `torch pack` execution window. No workflow step copies, caches, or logs the ROM filename or contents. ## Technical Safeguards in the Pipeline Design Several architectural decisions reinforce the **extract assets without storing ROMs** principle: - **Single-purpose tool**: `torch` is compiled fresh per-run, ensuring extraction logic matches the current codebase - **Ephemeral workspace**: GitHub Actions runners are destroyed after job completion, eliminating residual filesystem data - **Explicit artifact enumeration**: Only `lighthouse.o2r` appears in `upload-artifact` paths—no wildcards that could capture unexpected files - **No cross-job ROM passing**: The ROM is never passed as an artifact; only processed outputs flow downstream ## Comparison with Alternative Approaches **Manual asset extraction** requires users to own and process ROMs locally, complicating installation. **Server-side asset hosting** would require Lighthouse to distribute copyrighted material directly. The **ephemeral extraction pipeline** used in HarbourMasters/Lighthouse keeps the project legally compliant while providing a seamless build process: users supply their own ROMs, the infrastructure processes them transiently, and releases contain only legally redistributable components. ## Summary - The `torch` utility in the `Torch/` submodule performs ROM-specific asset extraction without preserving the original file. - The [`.github/workflows/main.yml`](https://github.com/HarbourMasters/Lighthouse/blob/main/.github/workflows/main.yml) pipeline builds `torch` on-demand, runs `torch pack` to generate `lighthouse.o2r`, and uploads only that archive. - ROM data never enters GitHub Actions artifacts, caches, or logs—remaining strictly in the runner's ephemeral filesystem. - Downstream platform jobs receive only the processed `.o2r` asset bundle, ensuring distributed builds contain no copyrighted ROM content. ## Frequently Asked Questions ### What is the `.o2r` file format used by Lighthouse? The `.o2r` format is a custom archive format developed for the Shipwright/libultraship engine ecosystem. It packages extracted game assets—textures, audio, level data, and shaders—into a single file that the engine can mount at runtime. Unlike raw ROM dumps, `.o2r` files contain only the specific resources the reimplementation requires, stripped of proprietary executable code and headers. ### How does `torch` know which assets to extract from the ROM? The `torch` tool contains hardcoded knowledge of the target game's ROM layout, including filesystem offsets, compression algorithms, and asset table locations. This mapping is maintained in the `Torch` submodule source code and compiled into the binary during the CI workflow. When invoked with `pack`, `torch` traverses these known structures and extracts only the file types the Lighthouse engine expects. ### Why build `torch` in CI instead of using a precompiled binary? Building `torch` from source ensures the extraction logic stays synchronized with the current codebase commit. As the `Torch` submodule receives updates—supporting new asset types or fixing decompression bugs—the CI pipeline automatically incorporates those changes. This eliminates version skew between the extraction tool and the engine that consumes its output. ### Can the ROM be recovered from a `lighthouse.o2r` artifact? No. The `.o2r` archive contains decompressed, often converted asset formats that discard the original ROM's binary structure. Headers, padding, executable code, and copyright-protected system libraries are excluded by design. Reconstructing the original ROM from extracted assets would require solving an intentionally lossy compression and format conversion process.