How the E2E Test Harness Manages Temporary Daemons in no-mistakes

The no-mistakes E2E test harness uses an inventory-based manager in internal/e2edaemon to spawn isolated daemon processes with sandboxed $NM_HOME directories, enforcing concurrency limits and guaranteeing cleanup through graceful shutdown hooks and a background reaper process.

The no-mistakes project requires a real daemon instance to validate its end-to-end workflows, but testing against a developer's local daemon risks data corruption and resource conflicts. The E2E test harness solves this by orchestrating temporary, sandboxed daemons that are tightly coupled to test lifecycles through a dedicated inventory management system.

Inventory-Based Ownership and Concurrency Control

The internal/e2edaemon package maintains a central inventory that tracks every temporary daemon spawned during test execution. This inventory serves as the source of truth for resource allocation and cleanup.

The Inventory Directory Structure

The harness stores daemon metadata in $NM_HOME/e2e/daemon-inventory, where each spawned daemon receives a JSON file containing:

• A unique slot ID
• The running process PID
• The absolute path to the temporary $NM_HOME directory

This file-based inventory enables atomic reservation of daemon slots and prevents race conditions when multiple tests run concurrently.

Concurrency Limits with NM_E2E_DAEMON_MAX

To prevent host overload, the harness enforces a concurrency cap via the NM_E2E_DAEMON_MAX environment variable (defaulting to 2). When a test calls e2edaemon.Acquire, the function atomically checks the inventory against this limit and either reserves a slot or returns a clear error if capacity is reached.

According to the package documentation in internal/e2edaemon/doc.go, this design ensures the test suite never exhausts system resources, even during parallel test execution.

Sandboxed Daemon Launch Process

Each test receives an isolated daemon instance through a combination of temporary directories and controlled process spawning.

Isolated NM_HOME Directories

The harness creates a fresh sandbox for every test using Go's t.TempDir():

func (h *Harness) setup() {
    h.NMHome = h.T.TempDir()
    // NM_HOME is set to this temporary path
}

This temporary directory becomes the daemon's workspace, housing its database, log files, and Unix socket. By isolating $NM_HOME, the test daemon cannot access the user's real ~/.no-mistakes configuration or data.

Process Spawning and PID Tracking

The harness launches the daemon via e2edaemon.Acquire(h.NMHome, h.NMBin, timeout), which:

1. Writes the inventory entry to reserve a slot
2. Executes the binary using exec.CommandContext with the test's environment variables
3. Records the child PID through Ownership.SyncPID

As implemented in internal/e2e/harness.go (lines 166-173), the daemon inherits the test process's environment while maintaining isolation from the host's default configuration.

Automatic Cleanup and Lifecycle Management

The harness guarantees daemon termination through multiple complementary mechanisms that handle both normal test completion and unexpected crashes.

Graceful Shutdown on Test Exit

The harness registers a cleanup function via t.Cleanup(h.Close) that executes three critical steps:

1. Terminate the daemon: Sends a graceful shutdown command (nm daemon stop) and waits for the PID to exit
2. Release the inventory slot: Calls e2edaemon.Release to delete the JSON entry and free the concurrency slot
3. Remove temporary files: The t.TempDir() cleanup automatically deletes the sandboxed $NM_HOME

This sequence ensures no resources leak between test runs, even when tests fail or panic.

The Reaper Process for Crash Recovery

When the test process crashes or receives a SIGKILL, the graceful cleanup may not execute. To handle this, the scripts/e2e.sh script invokes reapmain.go at exit, which scans the inventory directory for entries with non-existent PIDs and safely removes stale records.

As noted in scripts/e2e.sh (lines 48-55), this reaper prevents abandoned daemon processes from accumulating across CI runs.

Test Integration and Visibility

The harness exposes helpers that enable tests to verify correct daemon lifecycle behavior.

Harness Helper Methods

Tests interact with daemon lifecycle state through methods defined in internal/e2e/harness.go:

• daemonStopDir(): Verifies that nm daemon stop writes its PID file into the temporary $NM_HOME (asserted in internal/e2e/harness_test.go lines 105-107)
• h.daemonOwn.SyncPID(pid): Synchronizes the live process ID into the inventory for consistency checks

These utilities are exercised across the test suite, including internal/e2e/daemon_run_test.go and internal/cli/daemon_lifecycle_test.go, ensuring the temporary daemon behaves identically to production deployments.

Summary

• The inventory-based architecture in internal/e2edaemon tracks all temporary daemons via JSON files in $NM_HOME/e2e/daemon-inventory
• Concurrency control via NM_E2E_DAEMON_MAX limits parallel daemons to 2 by default
• Sandboxed environments using t.TempDir() ensure complete isolation from user data
• Automatic cleanup combines t.Cleanup hooks for graceful shutdown with a reaper process in scripts/e2e.sh for crash recovery
• PID synchronization through Ownership.SyncPID maintains accurate process tracking throughout the test lifecycle

Frequently Asked Questions

How does the E2E test harness prevent test daemons from interfering with my local no-mistakes installation?

The harness generates a unique temporary directory via t.TempDir() for each test and sets this as NM_HOME. This sandboxed directory contains all daemon data, including the database and Unix socket, ensuring complete isolation from the default ~/.no-mistakes location used by local development instances.

What happens if a test crashes while a temporary daemon is running?

If a test process crashes or is killed, the standard t.Cleanup hooks may not execute. The harness mitigates this through a reaper mechanism in scripts/e2e.sh that invokes reapmain.go to scan the inventory directory and remove entries for processes that no longer exist, preventing daemon accumulation across CI runs.

How many temporary daemons can run simultaneously during E2E testing?

The harness enforces a default concurrency limit of 2 simultaneous daemons, controlled by the NM_E2E_DAEMON_MAX environment variable. The e2edaemon.Acquire function checks this cap against the current inventory before spawning new processes, returning an error if the limit is reached.

Can I configure the timeout for daemon startup in the E2E harness?

Yes, the e2edaemon.Acquire function accepts a timeout parameter (as seen in the signature e2edaemon.Acquire(h.NMHome, h.NMBin, timeout)) that controls how long the harness waits for the daemon to initialize before failing the test setup.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}