How the Omarchy Test Suite Works: CLI, Shell, and Acceptance Testing

Omarchy's automated testing infrastructure is divided into three independent layers—CLI validation, shell plugin verification, and graphical acceptance testing—that execute via ./test/cli, ./test/shell, and disposable VMs through the omarchy-iso repository.

The basecamp/omarchy repository maintains a robust, multi-layered Omarchy test suite designed to validate everything from command routing to full desktop interactions. This testing pyramid separates concerns into fast headless checks and comprehensive graphical validation, ensuring reliable deployments across both CI environments and local development machines.

CLI Test Suite

The CLI test suite validates the command router, metadata linting, and theme pipelines located under bin/omarchy-*. It ensures that every binary exposes proper metadata headers and that the routing logic correctly handles help output and required arguments.

How CLI Tests Validate the Router

According to the source code in docs/testing.md, the ./test/cli script executes against a fake $HOME and a stubbed $PATH to verify routing behavior. The runner confirms that the omarchy command correctly resolves routes, loads metadata, and enforces required arguments as implemented in docs/cli-router.md. Additionally, it validates that every binary includes a proper # omarchy:summary= header and that omarchy commands --check passes metadata linting.

Running the CLI Tests

Execute the CLI suite independently using:

./test/cli

Shell Test Suite

The shell test suite exercises individual plugins, binary helpers, configuration invariants, and migration scripts without requiring a graphical environment. Tests reside in test/shell.d/*-test.sh and are coordinated by the ./test/shell runner.

The base-test.sh Harness

Every shell test begins with the same boilerplate that sources test/shell.d/base-test.sh:

#!/usr/bin/env bash
set -euo pipefail
source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/base-test.sh"

The harness performs several setup operations:

• Discovers the repository root and exports it as ROOT (accessible as $OMARCHY_PATH)
• Creates a temporary $HOME using mktemp -d for test isolation
• Prepends a stub bin/ directory to $PATH that logs calls instead of executing side effects like sudo, tmux, or gsettings

TAP-Style Assertions

The harness provides test helpers that output TAP (Test Anything Protocol) format:

• pass "description" — prints ok - description
• fail "description" [detail] — prints not ok - description and aborts the current test file
• require_command <cmd> — fails the file if a required external tool is missing

Because fail exits only the current test file with set -e, the ./test/shell runner continues executing remaining files and aggregates failures at the end, providing per-file granularity.

Compositor Gating for Headless CI

Tests requiring a Wayland compositor call require_compositor "reason". When no compositor socket is available, the helper prints a skip message (ok - no Wayland compositor; skipping …) and exits 0, which TAP treats as a passing test. This design keeps the Omarchy test suite green on headless CI runners while enabling comprehensive tests locally.

Node.js Unit Testing from Bash

Pure JavaScript modules—such as shell/plugins/menu/MenuModel.js—are tested via the run_node_test helper. This function injects a small JS prelude that mirrors the Bash assertions, allowing the same pass/fail output for logic that would otherwise require a running compositor.

Graphical Acceptance Test Suite

The acceptance layer validates end-to-end behavior requiring a real compositor (Quickshell or Hyprland), including panels, keyboard shortcuts, real applications, and system setup flows.

VM-Based Testing with omarchy-iso

Acceptance tests run inside disposable VMs managed by the sibling omarchy-iso repository. The harness drives the compositor via QMP and uses wtype for typing simulation.

To test against an existing installation with your current checkout synced:

cd ../omarchy-iso
./bin/omarchy-iso-test release/<iso>.iso --reuse-base --sync-omarchy ../omarchy --no-preview

For testing changes that affect the installer or shipped defaults, build a fresh ISO without --reuse-base:

./bin/omarchy-iso-make --no-boot-offer --local-source ../omarchy ../omarchy-pkgs
./bin/omarchy-iso-test release/<generated-iso>.iso --no-preview

Test Structure and Screenshot Capture

Acceptance tests reside in test/acceptance.d/*-test.sh and follow the same base-test.sh contract as the shell suite. Each test records artifacts for CI analysis:

• Screenshots named success-<step>.png or failure-<step>.png for each step
• Comprehensive logs collected under test-runs/

The VM harness follows the guidelines in agents/skills/acceptance-tests.md to orchestrate compositor-level shortcuts and application interactions.

Running the Full Test Pyramid

Execute all headless suites sequentially while continuing past individual failures:

./test/all

This runs ./test/cli first to validate routing and metadata, then ./test/shell to verify plugins and migrations, finally printing a summary of any failures. This aggregation ensures hidden problems are not masked by early aborts.

Summary

• The Omarchy test suite is organized into three layers: CLI validation, shell testing, and graphical acceptance.
• CLI tests in ./test/cli verify the router and metadata headers using stubbed $HOME and $PATH environments.
• Shell tests use test/shell.d/base-test.sh to provide TAP-compatible assertions, temporary $HOME isolation, and stubbed binaries.
• Compositor-dependent shell tests gracefully skip on headless systems via require_compositor, maintaining CI compatibility.
• Acceptance tests require the omarchy-iso repository and run in disposable VMs with automated screenshot capture via QMP and wtype.
• Execute the complete headless suite via ./test/all to validate routing, plugins, migrations, and configuration invariants.

Frequently Asked Questions

How do I run only the CLI tests in Omarchy?

Execute ./test/cli from the repository root. This script validates the command router against a fake $HOME, checks that every binary in bin/omarchy-* contains a proper # omarchy:summary= header, and verifies that omarchy commands --check passes metadata linting.

What is the purpose of base-test.sh in the shell test suite?

The test/shell.d/base-test.sh file provides the common harness that every shell test sources. It creates a temporary $HOME via mktemp -d, sets OMARCHY_PATH="$ROOT" to the checkout root, and builds a stub bin/ directory that logs calls to prevent side effects. It also exports pass, fail, and require_command functions that output TAP-compatible results.

Can the Omarchy test suite run without a graphical environment?

Yes. Both the CLI and shell suites run headless on standard CI runners. Shell tests that require a Wayland compositor call require_compositor, which skips the test gracefully when no socket is available, allowing the suite to pass on headless systems. Only the acceptance tests require a running compositor inside a VM.

How are JavaScript modules tested within the shell suite?

The run_node_test helper executes Node.js unit tests from Bash by injecting a prelude that mirrors the shell assertion protocol. This enables testing pure JavaScript logic—such as shell/plugins/menu/MenuModel.js—without spinning up a full graphical environment, outputting pass/fail results in TAP format that the shell runner can aggregate.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}