How twist.nix Tracks Built-In Emacs Libraries: A Complete Technical Guide

Twist.nix tracks built-in Emacs libraries by scanning the Emacs source tree to generate a reproducible list of built-in .el files, parsing them into a structured Nix list, and integrating them into the dependency resolution pipeline to handle conflicts with external packages.

Twist.nix is a Nix-based build system for Emacs configurations that must distinguish between libraries shipped with Emacs itself and those provided by external packages. Understanding how twist.nix tracks built-in Emacs libraries is essential for debugging dependency conflicts, optimizing closure sizes, and ensuring deterministic builds across different Emacs versions.

The Three-Stage Discovery Process

Twist.nix discovers and tracks built-in libraries through a pipeline that ensures source-truth and reproducibility.

Stage 1: Generating the Raw Library List from Emacs Source

In pkgs/emacs/builtins.nix, twist.nix defines a Nix derivation that walks the Emacs source tree's lisp/ directory. It uses rg (ripgrep) and sed to extract every .el file containing the standard GNU Emacs header "This file is part of GNU Emacs." The output is a plain-text file with one library name per line, stored in the Nix store as builtinLibraryList.

This approach ensures the list reflects the actual upstream source rather than a hardcoded manifest that could become stale across Emacs versions.

Stage 2: Parsing the List into Structured Data

In pkgs/emacs/default.nix, the derivation from Stage 1 is built and imported:

builtinLibraryList = self.callPackage ./builtins.nix {};

The resulting file is read, split on newlines, and filtered to produce the Nix list builtinLibraries. This transformation uses helper functions from lib/default.nix—such as lib.pipe, split, and filter—to turn the raw string into a structured list like ["abbrev" "apropos" "bookmark" ...].

Stage 3: Integrating Built-Ins into Dependency Resolution

The builtinLibraries list feeds into three critical dependency calculation mechanisms defined in pkgs/emacs/default.nix:

Visible Built-In Libraries: The visibleBuiltinLibraries attribute (lines 122-124) represents built-in libraries that are not explicitly requested by the user in their configuration. These are available for implicit dependencies but not explicitly installed.

Masked Built-Ins: When a user provides an external package that conflicts with a built-in library, maskedBuiltins (lines 87-89) identifies these overlaps. For example, if you request json-mode as an external package, it will mask the built-in json-mode, ensuring the external version takes precedence.

Transitive Dependency Filtering: The libraries are subtracted from the explicit package set when computing allDependencies, preventing built-ins from being redundantly fetched as external dependencies and ensuring minimal closure sizes.

Practical Examples: Working with Built-In Library Tracking

Inspecting the Raw Built-In List

To view the raw list of built-in libraries generated from the Emacs source:

let
  twist = import (builtins.fetchGit {
    url   = "https://github.com/emacs-twist/twist.nix";
    rev   = "master";
  }) {};
in
  builtins.readFile twist.emacs.builtinLibraryList

This returns a newline-separated string of library names (e.g., abbrev\napropos\n...).

Retrieving the Parsed Library List

To access the structured Nix list of built-ins:

let
  twist = import (builtins.fetchGit {
    url = "https://github.com/emacs-twist/twist.nix";
    rev = "master";
  }) {};
in
  twist.emacs.builtinLibraries

Result: ["abbrev" "apropos" "bookmark" ...]

Identifying Masked Built-Ins

To see which built-ins are being overridden by external packages:

let
  twist = import (builtins.fetchGit {
    url = "https://github.com/emacs-twist/twist.nix";
    rev = "master";
  }) {
    # Example: Request a newer json-mode that shadows the built-in

    userConfig.elispPackages = [ "json-mode" ];
  };
in
  twist.emacs.maskedBuiltins

This returns ["json-mode"], indicating that Twist.nix will use the external version rather than the built-in one.

Key Files in the Built-In Library Tracking System

Understanding the architecture requires familiarity with these specific files:

• pkgs/emacs/builtins.nix – The derivation that scans the Emacs source tree (lisp/ directory) using rg and sed to generate the raw list of built-in libraries.

• pkgs/emacs/default.nix – Imports the derivation, parses the raw list into builtinLibraries, and implements the dependency resolution logic including visibleBuiltinLibraries and maskedBuiltins.

• lib/default.nix – Provides utility functions (lib.pipe, split, filter) used to transform the raw text output into structured Nix lists.

• pkgs/emacs/tools/check-versions.nix – Consumes builtinLibraries (via inherit lib builtinLibraries) to verify version constraints, demonstrating how the built-in list propagates to other tooling in the ecosystem.

These files work together to ensure that Twist.nix always has an up-to-date, reproducible view of the libraries that are part of Emacs itself.

Summary

Twist.nix tracks built-in Emacs libraries through a three-stage pipeline that ensures reproducibility and accurate dependency resolution:

• Source extraction: The system scans the Emacs lisp/ directory in pkgs/emacs/builtins.nix to generate a raw list of built-in libraries directly from upstream source code.
• Structured parsing: pkgs/emacs/default.nix transforms this raw text into the builtinLibraries Nix list, making it available for programmatic dependency calculations.
• Conflict resolution: The system identifies maskedBuiltins where external packages override built-ins and filters built-ins from transitive dependencies to prevent redundant package fetching.

This architecture guarantees that your Emacs configuration always uses the correct library versions while maintaining the deterministic guarantees of the Nix ecosystem.

Frequently Asked Questions

How does twist.nix distinguish between built-in and external Emacs libraries?

Twist.nix distinguishes built-in from external libraries by scanning the Emacs source tree's lisp/ directory for files containing the standard GNU Emacs header. In pkgs/emacs/builtins.nix, the system uses rg (ripgrep) and sed to extract these files, generating a definitive list of libraries shipped with Emacs. Any package not appearing in this builtinLibraries list is treated as an external dependency that must be fetched separately.

What happens when an external package conflicts with a built-in library?

When you request an external package that provides the same library as a built-in, twist.nix resolves the conflict through the maskedBuiltins mechanism defined in pkgs/emacs/default.nix. The system computes the intersection between your requested packages and the builtinLibraries list. If a match is found—such as requesting an external json-mode when json-mode is built-in—that library is added to maskedBuiltins. The build pipeline then uses the external version instead of the built-in one, ensuring you get the specific version you requested.

Can I inspect which built-in libraries are available in my twist.nix configuration?

Yes, you can inspect the built-in library list at multiple stages of the twist.nix pipeline. To view the raw text file generated from the Emacs source, access twist.emacs.builtinLibraryList in a Nix REPL and read it with builtins.readFile. For the parsed Nix list, evaluate twist.emacs.builtinLibraries. To see which built-ins are being masked by your external packages, evaluate twist.emacs.maskedBuiltins after providing your userConfig.elispPackages. These attributes provide full transparency into how twist.nix categorizes libraries for your specific configuration.

Why does twist.nix scan the Emacs source tree instead of using a hardcoded list?

Twist.nix scans the Emacs source tree in pkgs/emacs/builtins.nix rather than maintaining a hardcoded list to ensure source-truth and reproducibility across Emacs versions. Because different Emacs releases ship with different sets of built-in libraries, generating the list dynamically from the actual lisp/ directory guarantees that the dependency graph accurately reflects whatever Emacs version you are building against. This approach also eliminates manual maintenance burden and ensures that any upstream changes to Emacs's built-in library set are automatically captured when you update your Emacs input.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}