How twist.nix Parses Emacs init Files with use-package Declarations

Twist.nix extracts Emacs package dependencies from use-package declarations by parsing init files into three Nix collections—elispPackages, elispPackagePins, and systemPackages—using a pure Nix function that operates directly on the Lisp AST.

The emacs-twist/twist.nix project enables reproducible Emacs configurations by treating use-package declarations in your init.el as first-class Nix build inputs. Instead of manually maintaining separate package lists, twist.nix automatically parses your init files at evaluation time to discover which ELPA/MELPA packages and system tools your configuration requires.

The Three Collections Extracted from use-package

When twist.nix parses your Emacs init files, it aggregates every use-package form into three distinct attribute sets used by the Nix build pipeline:

• elispPackages — The set of Emacs Lisp packages that must be pulled from ELPA, MELPA, or other archive sources. This includes both direct packages (where :ensure t) and indirect packages (where :ensure specifies a different package name).

• elispPackagePins — Optional version pinning information extracted from the :pin keyword, mapping package names to specific commit hashes or version strings.

• systemPackages — External system dependencies declared via the :ensure-system-package keyword, converted into a list of strings for Nix to provide via nixpkgs.

How the Parsing Pipeline Works

The parsing process is implemented entirely in pure Nix and runs during evaluation. The pipeline follows a strict sequence from raw text to structured dependency data.

Reading and Converting the init File

The entry point is the initReader function defined in pkgs/emacs/default.nix (lines 13–16). It reads the raw file content and passes it to the parser:

initReader ? file: initParser (builtins.readFile file)

By default, twist.nix uses the built-in parseUsePackages helper as the initParser (line 14 of the same file). If you do not provide a custom parser, this default handles the heavy lifting.

Converting Lisp to a Nix AST

The parseUsePackages function in pkgs/build-support/elisp/parseUsePackages.nix receives the raw string and converts it into a traversable data structure. It calls the fromElisp library (bundled via the elisp-helpers input) to transform the Lisp syntax into Nix lists:

blocks = fromElisp.fromElisp string;

This occurs at lines 6–7 of parseUsePackages.nix, producing a list of forms (expressions) from your init file.

Detecting and Filtering use-package Forms

The parser identifies valid use-package declarations using the isUsePackageForm predicate (lines 17–22):

isUsePackageForm = xs:
  isList xs && length xs > 0 && head xs == "use-package" && isEnabled xs;

A form qualifies only if it is a list starting with the string "use-package" and is not disabled (the isEnabled helper checks for the :disabled keyword).

Extracting Package Names and Metadata

Once valid forms are isolated, the parser extracts three categories of data:

Direct and indirect packages (lines 38–40): Direct packages have :ensure t (or any truthy value), while indirect packages occur when :ensure points to a specific package name string.

directPackages = map enameFromUsePackage (filter isEnsured usePackageForms);
indirectPackages = filter isString (map ensuredPackageName usePackageForms);

Pinned versions (lines 42–49): If a form contains :pin, the parser extracts the value into elispPackagePins using findPin.

System packages (lines 51–52): The :ensure-system-package keyword is normalized to a list of strings via toSystemPackages and collected into systemPackages.

Aggregating Results Across Multiple Files

In pkgs/emacs/default.nix (lines 75–88), initReader is mapped over the list of provided init files, and lib.zipAttrs merges the per-file results:

userConfig = lib.pipe self.initFiles [
  (map initReader)
  lib.zipAttrs
  (lib.mapAttrs (name: values:
    if name == "elispPackages" then concatLists values
    else if name == "elispPackagePins" then lib.foldl' (acc: x: acc // x) {} values
    else if name == "systemPackages" then concatLists values
    else throw "${name} is an unknown attribute"))
];

The resulting userConfig attribute feeds directly into twist.nix’s dependency resolution, lock-file generation, and the emacsWrapper derivation.

Key Source Files and Functions

Understanding the implementation requires familiarity with these specific modules:

• pkgs/build-support/elisp/parseUsePackages.nix — The core parser containing isUsePackageForm, enameFromUsePackage, and the extraction logic for :ensure, :pin, and :ensure-system-package.

• pkgs/emacs/default.nix — Defines initReader, sets the default initParser, and implements the aggregation logic that produces the final userConfig.

• lib/default.nix — Re-exports the parser as lib.parseUsePackages (lines 21–27), making it available for external consumption or custom parser development.

• pkgs/build-support/elisp/testUsePackage.nix — Contains the test suite verifying parser behavior for various use-package edge cases, including disabled packages and complex :ensure values.

Practical Usage Examples

Directly Invoking the Parser on a String

You can test the parser interactively in a Nix REPL or expression:

let
  lib = import <nixpkgs> { };
  parseUse = lib.parseUsePackages { lib = lib; };
  init = ''
    (use-package magit :ensure t)
    (use-package projectile :ensure projectile :pin "1234abcd")
    (use-package flycheck :ensure-system-package "ripgrep")
  '';
in
  parseUse {} init

Result:

{
  elispPackages = [ "magit" "projectile" ];
  elispPackagePins = { projectile = "1234abcd"; };
  systemPackages = [ "ripgrep" ];
}

Using the Default Configuration

To parse a real init.el within the twist.nix framework:

let
  twist = import (builtins.fetchGit {
    url = "https://github.com/emacs-twist/twist.nix";
    rev = "master";
  }) { };
  myInit = ./my-init.el;
  cfg = twist.emacs {
    initFiles = [ myInit ];
  };
in
  cfg.userConfig

The userConfig attribute contains the aggregated elispPackages, elispPackagePins, and systemPackages derived from your init file.

Extending the Parser for Custom Keywords

You can override initParser to handle additional use-package keywords not supported by the default implementation:

let
  lib = import <nixpkgs> { };
  myParser = { lib }: initStr:
    let
      base = lib.parseUsePackages { inherit lib; } {} initStr;
    in base // {
      # Example: collect all :bind keys

      binds = lib.concatMap (form:
        if lib.elem ":bind" form then [ (lib.elemAt form 2) ] else []
      ) (fromElisp.fromElisp initStr);
    };
in
  twist.emacs {
    initFiles = [ ./init.el ];
    initParser = myParser { inherit lib; };
  }

This leverages the initParser parameter defined in pkgs/emacs/default.nix (line 14), allowing you to extend parsing logic without modifying core twist.nix code.

Summary

• Twist.nix parses Emacs init files at Nix evaluation time using a pure function that converts Lisp syntax into a Nix-compatible AST.
• The parser produces three collections: elispPackages for Lisp libraries, elispPackagePins for version constraints, and systemPackages for external binaries.
• Detection relies on the isUsePackageForm predicate in parseUsePackages.nix, which filters out disabled declarations and extracts metadata from :ensure, :pin, and :ensure-system-package.
• Results are aggregated via lib.zipAttrs in emacs/default.nix and consumed by the lock-file generator and emacsWrapper.
• Users can override the default initParser to support custom use-package keywords or alternative configuration formats.

Frequently Asked Questions

How does twist.nix handle disabled use-package declarations?

The parser explicitly filters out disabled forms. In pkgs/build-support/elisp/parseUsePackages.nix, the isUsePackageForm function checks isEnabled xs, which returns false if the form contains the :disabled keyword. These forms are excluded from all three output collections (elispPackages, elispPackagePins, and systemPackages), ensuring that disabled packages do not become Nix dependencies.

What is the difference between direct and indirect packages in the parser?

Direct packages are those where the :ensure keyword has a truthy value (typically t), meaning the package name matches the use-package declaration itself. Indirect packages occur when :ensure specifies a different string value, indicating that the configuration requires a package other than the one being configured. The parser handles both cases separately in lines 38–40 of parseUsePackages.nix, concatenating them into the final elispPackages list.

Can I parse init files that use alternative package management syntax?

Yes, by providing a custom initParser function to the emacs module. The default initReader in pkgs/emacs/default.nix accepts an initParser argument (line 14) that you can override with any function following the signature string -> { elispPackages, elispPackagePins, systemPackages }. This allows you to support straight.el, elpaca, or bespoke declaration formats while still integrating with twist.nix's build pipeline.

Where does the Lisp-to-Nix conversion happen?

The conversion from raw Emacs Lisp text to a Nix data structure occurs in pkgs/build-support/elisp/parseUsePackages.nix via the fromElisp library (lines 6–7). This library, provided by the elisp-helpers input, parses the string into a list of lists (Lisp forms) that native Nix functions can traverse and filter without external build dependencies.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}