The Role of Lock Files in twist.nix for Reproducible Builds

twist.nix uses three lock files—flake.lock, archive.lock, and metadata.json—to pin exact Git revisions, tarball checksums, and package metadata, ensuring that Emacs package builds produce bit-identical outputs across any machine and at any time.

The twist.nix framework transforms Emacs package management into a fully declarative Nix workflow. By materializing dependency resolution into version-controlled lock files, the system eliminates network variability and "latest-commit" drift, making the role of lock files in twist.nix for reproducible builds foundational to its architecture.

The Three Lock Files That Enable Reproducibility

flake.lock: Pinning Git Sources

The flake.lock file ensures that every package built from source (the src attribute) uses an identical Git checkout. According to the twist.nix source code, this file is generated by the helper in pkgs/emacs/lock/flake-lock.nix, which rewrites the generic Nix flake lock to preserve original origin fields while adding the exact locked revision.

This guarantees that the same commit of each upstream repository is used every time, eliminating drift caused by new commits to ELPA packages.

archive.lock: Pinning ELPA Tarballs

For packages obtained from ELPA archives (GNU ELPA, MELPA, etc.), twist.nix generates archive.lock to record exact tarball versions and checksums. The logic in pkgs/emacs/lock/default.nix extracts archive, version, packageRequires, and inventory from the resolved packageInputs (lines 20–28, 51–58, 70–88) and writes a JSON file using jq.

Even when fetching from a tarball, this lock file records the exact narHash and version, ensuring the same binary artifact is reused on any machine.

metadata.json: Caching Package Metadata

The optional metadata.json file stores additional metadata about each package, including narHash, author, meta, and dependency information. Produced by the same default.nix module when persistMetadata = true (lines 31–40, 52–55), this file allows downstream tools like twist.el to reconstruct the exact package graph without re-evaluating the full Nix expression, avoiding hidden import-from-derivation (IFD) sources.

How Lock Files Are Generated and Wired Together

Package Discovery and Storage

In pkgs/emacs/default.nix, the system builds a concrete package set via enumerateConcretePackageSet and stores the paths to lock files in the variables flakeLockFile, archiveLockFile, and metadataJsonFile (lines 68–73). This establishes the connection between the abstract package set and its concrete, serialized representations.

Lock Generation Process

The generateLockFiles attribute, imported from pkgs/emacs/lock, orchestrates the creation of lock files. When invoked with flags flakeNix = true and archiveLock = true (and optionally metadataJson = persistMetadata), it creates a derivation whose writerScript materializes the three files into a user-provided directory (lockDir). The writer script is defined in pkgs/emacs/lock/write-lock-1.nix and invoked via pkgs/emacs/lock/default.nix (lines 22–27).

CLI Exposure via Nix Apps

Both generateLockDir and the makeApps helpers expose the lock-writer as a Nix app (type = "app"), enabling users to materialize lock files through standard Nix commands. This design integrates seamlessly with standard nix run workflows.

Working with Lock Files in Practice

Generating a Lock Directory

To create lock files for your Emacs configuration:

# Create a directory for the lock files

mkdir -p lockfiles

# Run the lock-writer app produced by makeApps

nix run .#makeApps.lock lockDirName=lockfiles

This executes the app defined in pkgs/emacs/default.nix (lines 44–48), producing:

• lockfiles/flake.lock
• lockfiles/archive.lock
• lockfiles/metadata.json (only if persistMetadata = true)

Consuming Lock Files in a Downstream Flake

Supply the generated files to emacsWithLock to enforce deterministic builds:

{
  inputs = {
    twist = {
      url = "github:emacs-twist/twist.nix";
      inputs.lock = ./lockfiles/flake.lock;
    };
  };
  
  outputs = { self, twist, ... }:
    let
      twistPkg = twist.packages.x86_64-linux.emacsWithLock {
        lockDir = ./lockfiles;
      };
    in {
      packages.x86_64-linux.default = twistPkg.emacsWrapper;
    };
}

Because flake.lock and archive.lock are supplied, evaluation never reaches out to the network for newer versions, ensuring fully deterministic builds.

Selective Updates

To update only the archive lock (for example, after a new ELPA release) while preserving Git revisions:

nix run .#makeApps.update lockDirName=lockfiles

This executes the update app defined in pkgs/emacs/default.nix (lines 60–70), regenerating only archive.lock and optionally metadata.json while leaving flake.lock untouched.

Summary

• Three lock files—flake.lock, archive.lock, and metadata.json—work together to eliminate variability in Emacs package sources.
• flake.lock pins exact Git revisions for source-based packages via pkgs/emacs/lock/flake-lock.nix.
• archive.lock records exact tarball versions and narHash values from ELPA archives via pkgs/emacs/lock/default.nix.
• metadata.json optionally caches package metadata to avoid IFD and enable hot-reloading in external tools.
• The generateLockFiles attribute and makeApps expose lock management as standard Nix applications, accessible via nix run.

Frequently Asked Questions

What happens if I don't use lock files in twist.nix?

Without lock files, twist.nix must resolve package versions and fetch sources during every evaluation, leading to non-reproducible builds that vary depending on the state of upstream Git repositories and ELPA archives at build time. The build may succeed on one machine and fail on another due to commit drift or deleted tags.

How do I update only the archive lock without changing Git revisions?

Use the update app exposed by makeApps.update. This executes the logic in pkgs/emacs/default.nix (lines 60–70) to regenerate archive.lock and optionally metadata.json while preserving the existing flake.lock, allowing you to pull in new ELPA releases without affecting packages built from specific Git commits.

What is the purpose of metadata.json?

metadata.json stores supplementary package information including narHash, authorship details, and dependency graphs when persistMetadata = true. This allows downstream tools like twist.el to reconstruct the package graph for hot-reloading without triggering a full Nix evaluation, significantly improving responsiveness while maintaining accuracy.

Can I use twist.nix without Nix flakes?

While twist.nix is designed around the flake system for lock file management, the core lock generation logic in pkgs/emacs/lock operates on standard Nix derivations. However, flake.lock specifically requires a flake-based workflow; without flakes, you would need to manually pin source URLs and hashes in your Nix expressions to achieve equivalent reproducibility.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}