How twist.nix Manages ELPA Packages: Inventory Parsing and Build Logic

Twist.nix converts ELPA (Emacs Lisp Package Archive) collections into reproducible Nix derivations by parsing the archive-contents file, distinguishing between core and external packages, and generating locked source trees with filtered file sets.

Twist.nix, the Nix-based Emacs package manager from the emacs-twist/twist.nix repository, treats ELPA archives as structured inventories rather than static repositories. The system transforms the Lisp-style archive-contents metadata into declarative Nix attribute sets that define how each package is fetched, filtered, and built. This approach enables fully reproducible Emacs configurations while maintaining compatibility with the traditional ELPA ecosystem.

ELPA Inventory Architecture

Twist.nix implements ELPA support through a pipeline that converts archive metadata into buildable Nix derivations. The architecture separates package discovery from package construction, allowing granular control over sources and contents.

Parsing Archive Contents with parseElpaPackages

The entry point for ELPA processing is lib.parseElpaPackages, exposed through pkgs/build-support/default.nix. This function reads the archive-contents file—a Lisp-style alist that describes the entire ELPA package set—and converts it into a structured Nix attribute set. Each entry in the parsed data includes version strings, dependencies, URLs, and optional build instructions like :make fields.

According to the twist.nix source code, this parser handles the Lisp-to-Nix translation that makes the ELPA metadata accessible to the Nix expression language.

Core vs. External Package Handling

In pkgs/emacs/data/inventory/elpa.nix, the system bifurcates packages into two distinct categories based on the presence of a core attribute.

Core packages (lines 24–44) are those shipped with the Emacs source tree itself. When args ? core-src is provided, these packages are built from the supplied Emacs source tree; otherwise, they are omitted from the package set to avoid conflicts with the host Emacs installation.

External packages (lines 74–138) comprise the majority of ELPA entries. The makeExternal function processes each entry to construct a derivation that includes:

• Source resolution: Uses fetchTree to obtain the exact revision from the flake lock file, falling back to impure fetching if no lock entry exists (lines 86–95).
• File filtering: Applies ignored-files patterns to exclude tests, documentation drafts, or build artifacts from the final package.
• Pre-build steps: Injects a preBuild phase when the ELPA entry defines a :make field, ensuring packages requiring compilation steps are properly built.

Derivation Generation and Source Locking

For each external package, twist.nix constructs an attribute set containing src, origin metadata via lib.flakeRefAttrsFromElpaAttrs, and a curated files list. The filesInDir and isElisp predicates identify relevant Lisp files, documentation, and texinfo sources while respecting the ignore patterns defined in the ELPA metadata.

The final output of elpa.nix merges these components into corePackages // externalPackages (lines 167–169), producing a complete attribute set that the higher-level inventory dispatcher consumes.

Key Implementation Files

The ELPA management system spans several interconnected files in the twist.nix repository:

• pkgs/emacs/data/inventory/elpa.nix – Core ELPA logic implementing makeExternal, core package handling, and the merge logic that produces the final package set.
• pkgs/build-support/default.nix – Re-exports parseElpaPackages and other parsers, making them available to inventory loaders.
• pkgs/emacs/data/inventory/default.nix – Dispatches to the appropriate inventory loader based on the registry type field ("elpa" or "melpa").
• pkgs/emacs/default.nix – Top-level entry point that reads user init files, enumerates the concrete package set via enumerateConcretePackageSet, and exposes the final elispPackages.
• pkgs/emacs/data/package.nix – Consumes the attributes generated by elpa.nix to build individual package derivations.

Configuring ELPA Registries

To include ELPA packages in a twist.nix configuration, define registries with type = "elpa" pointing to directories containing archive-contents files.

{
  lockDir = ./lock;
  
  registries = {
    gnu-elpa = {
      type = "elpa";
      path = inputs.gnu-elpa.outPath + "/elpa-packages";
    };
    nongnu = {
      type = "elpa";
      path = inputs.nongnu.outPath + "/elpa-packages";
    };
  };
  
  initFiles = [ ./init.el ];
}

The inventory loader (elpa.nix) parses the archive-contents at these paths, converting each entry into a derivation available as part of the elispPackages set.

Pinning ELPA Versions for Reproducibility

Twist.nix achieves reproducibility through lock files generated by twist generateLockDir. The resulting archive.lock stores exact Git revisions for each ELPA package:

{
  "packages": {
    "elpa-mirror/gnu": {
      "url": "github:elpa-mirrors/elpa",
      "rev": "a1b2c3d4e5f6…"
    }
  }
}

When building, elpa.nix detects the lock entry (hasAttr ename flakeLockData) and fetches the source using fetchTree with the pinned revision, ensuring deterministic builds across different environments and time.

Customizing Package Contents

You can exclude specific files from an ELPA package by adding ignored-files entries to the package metadata. The makeExternal function (lines 74–80) constructs a predicate from glob patterns and filters the file list accordingly:

# Example archive-contents entry

("my-package" . [("1.0" . ((:url . "https://example.com/pkg.tar")
                            (:ignored-files . ("tests/*" "scripts/*"))))])

During derivation creation, files matching tests/* or scripts/* are excluded from the package's files attribute, keeping the closure size minimal and avoiding unnecessary build-time dependencies.

Summary

• Twist.nix manages ELPA packages by treating archive-contents as an inventory parsed by lib.parseElpaPackages in pkgs/build-support/default.nix.
• The system separates core packages (built from Emacs source) from external packages (fetched from upstream) in pkgs/emacs/data/inventory/elpa.nix.
• External packages are processed by makeExternal (lines 74–138), which handles source fetching via locked fetchTree calls, file filtering via ignored-files, and optional preBuild steps for :make directives.
• Registry configuration uses type = "elpa" to trigger the inventory dispatcher in pkgs/emacs/data/inventory/default.nix.
• Reproducibility is enforced through archive.lock files that pin exact Git revisions for every ELPA package.

Frequently Asked Questions

How does twist.nix handle ELPA packages that require compilation or Makefiles?

When an ELPA entry contains a :make field in its metadata, the makeExternal function in pkgs/emacs/data/inventory/elpa.nix automatically adds a preBuild step to the derivation. This ensures that Makefiles are executed during the Nix build phase before the package is installed, handling any required byte-compilation or resource generation.

What is the difference between core and external ELPA packages in twist.nix?

Core packages are those included in the Emacs source distribution itself, identified by a core attribute in the ELPA metadata. These are built from the Emacs source tree when core-src is provided. External packages are fetched from their own repositories via fetchTree using revisions pinned in the flake lock file, making them independent of the Emacs release cycle.

Can I exclude specific files like tests or documentation from ELPA packages?

Yes. Twist.nix respects the ignored-files field in ELPA metadata entries. When makeExternal processes a package, it builds a predicate from the glob patterns listed in ignored-files (e.g., "tests/*" or "doc/old/*") and filters the file list before adding files to the derivation's files attribute, effectively excluding them from the final build.

Where does twist.nix store the exact versions of ELPA packages?

Exact versions are stored in the archive.lock file within your configured lockDir. This JSON file maps package names to specific Git revisions (rev) and URLs. The elpa.nix inventory loader checks for these entries using hasAttr ename flakeLockData and passes them to fetchTree, ensuring that every build uses the identical source code regardless of when the build occurs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}