# How the Omarchy Test Suite Works: CLI, Shell, and Acceptance Testing

> Discover how the Omarchy test suite operates across CLI validation, shell plugin verification, and graphical acceptance testing for robust automated testing.

- Repository: [37signals/omarchy](https://github.com/basecamp/omarchy)
- Tags: how-to-guide
- Published: 2026-08-28

---

**Omarchy's automated testing infrastructure is divided into three independent layers—CLI validation, shell plugin verification, and graphical acceptance testing—that execute via `./test/cli`, `./test/shell`, and disposable VMs through the `omarchy-iso` repository.**

The `basecamp/omarchy` repository maintains a robust, multi-layered Omarchy test suite designed to validate everything from command routing to full desktop interactions. This testing pyramid separates concerns into fast headless checks and comprehensive graphical validation, ensuring reliable deployments across both CI environments and local development machines.

## CLI Test Suite

The CLI test suite validates the command router, metadata linting, and theme pipelines located under `bin/omarchy-*`. It ensures that every binary exposes proper metadata headers and that the routing logic correctly handles help output and required arguments.

### How CLI Tests Validate the Router

According to the source code in [`docs/testing.md`](https://github.com/basecamp/omarchy/blob/main/docs/testing.md), the `./test/cli` script executes against a fake `$HOME` and a stubbed `$PATH` to verify routing behavior. The runner confirms that the `omarchy` command correctly resolves routes, loads metadata, and enforces required arguments as implemented in [`docs/cli-router.md`](https://github.com/basecamp/omarchy/blob/main/docs/cli-router.md). Additionally, it validates that every binary includes a proper `# omarchy:summary=` header and that `omarchy commands --check` passes metadata linting.

### Running the CLI Tests

Execute the CLI suite independently using:

```bash
./test/cli

```

## Shell Test Suite

The shell test suite exercises individual plugins, binary helpers, configuration invariants, and migration scripts without requiring a graphical environment. Tests reside in `test/shell.d/*-test.sh` and are coordinated by the `./test/shell` runner.

### The base-test.sh Harness

Every shell test begins with the same boilerplate that sources [`test/shell.d/base-test.sh`](https://github.com/basecamp/omarchy/blob/main/test/shell.d/base-test.sh):

```bash
#!/usr/bin/env bash
set -euo pipefail
source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/base-test.sh"

```

The harness performs several setup operations:

- Discovers the repository root and exports it as `ROOT` (accessible as `$OMARCHY_PATH`)
- Creates a temporary `$HOME` using `mktemp -d` for test isolation
- Prepends a stub `bin/` directory to `$PATH` that logs calls instead of executing side effects like `sudo`, `tmux`, or `gsettings`

### TAP-Style Assertions

The harness provides test helpers that output TAP (Test Anything Protocol) format:

- `pass "description"` — prints `ok - description`
- `fail "description" [detail]` — prints `not ok - description` and aborts the current test file
- `require_command <cmd>` — fails the file if a required external tool is missing

Because `fail` exits only the current test file with `set -e`, the `./test/shell` runner continues executing remaining files and aggregates failures at the end, providing per-file granularity.

### Compositor Gating for Headless CI

Tests requiring a Wayland compositor call `require_compositor "reason"`. When no compositor socket is available, the helper prints a skip message (`ok - no Wayland compositor; skipping …`) and exits 0, which TAP treats as a passing test. This design keeps the Omarchy test suite green on headless CI runners while enabling comprehensive tests locally.

### Node.js Unit Testing from Bash

Pure JavaScript modules—such as [`shell/plugins/menu/MenuModel.js`](https://github.com/basecamp/omarchy/blob/main/shell/plugins/menu/MenuModel.js)—are tested via the `run_node_test` helper. This function injects a small JS prelude that mirrors the Bash assertions, allowing the same `pass`/`fail` output for logic that would otherwise require a running compositor.

## Graphical Acceptance Test Suite

The acceptance layer validates end-to-end behavior requiring a real compositor (Quickshell or Hyprland), including panels, keyboard shortcuts, real applications, and system setup flows.

### VM-Based Testing with omarchy-iso

Acceptance tests run inside disposable VMs managed by the sibling `omarchy-iso` repository. The harness drives the compositor via QMP and uses `wtype` for typing simulation.

To test against an existing installation with your current checkout synced:

```bash
cd ../omarchy-iso
./bin/omarchy-iso-test release/<iso>.iso --reuse-base --sync-omarchy ../omarchy --no-preview

```

For testing changes that affect the installer or shipped defaults, build a fresh ISO without `--reuse-base`:

```bash
./bin/omarchy-iso-make --no-boot-offer --local-source ../omarchy ../omarchy-pkgs
./bin/omarchy-iso-test release/<generated-iso>.iso --no-preview

```

### Test Structure and Screenshot Capture

Acceptance tests reside in `test/acceptance.d/*-test.sh` and follow the same [`base-test.sh`](https://github.com/basecamp/omarchy/blob/main/base-test.sh) contract as the shell suite. Each test records artifacts for CI analysis:

- Screenshots named `success-<step>.png` or `failure-<step>.png` for each step
- Comprehensive logs collected under `test-runs/`

The VM harness follows the guidelines in [`agents/skills/acceptance-tests.md`](https://github.com/basecamp/omarchy/blob/main/agents/skills/acceptance-tests.md) to orchestrate compositor-level shortcuts and application interactions.

## Running the Full Test Pyramid

Execute all headless suites sequentially while continuing past individual failures:

```bash
./test/all

```

This runs `./test/cli` first to validate routing and metadata, then `./test/shell` to verify plugins and migrations, finally printing a summary of any failures. This aggregation ensures hidden problems are not masked by early aborts.

## Summary

- The Omarchy test suite is organized into three layers: CLI validation, shell testing, and graphical acceptance.
- CLI tests in `./test/cli` verify the router and metadata headers using stubbed `$HOME` and `$PATH` environments.
- Shell tests use [`test/shell.d/base-test.sh`](https://github.com/basecamp/omarchy/blob/main/test/shell.d/base-test.sh) to provide TAP-compatible assertions, temporary `$HOME` isolation, and stubbed binaries.
- Compositor-dependent shell tests gracefully skip on headless systems via `require_compositor`, maintaining CI compatibility.
- Acceptance tests require the `omarchy-iso` repository and run in disposable VMs with automated screenshot capture via QMP and `wtype`.
- Execute the complete headless suite via `./test/all` to validate routing, plugins, migrations, and configuration invariants.

## Frequently Asked Questions

### How do I run only the CLI tests in Omarchy?

Execute `./test/cli` from the repository root. This script validates the command router against a fake `$HOME`, checks that every binary in `bin/omarchy-*` contains a proper `# omarchy:summary=` header, and verifies that `omarchy commands --check` passes metadata linting.

### What is the purpose of base-test.sh in the shell test suite?

The [`test/shell.d/base-test.sh`](https://github.com/basecamp/omarchy/blob/main/test/shell.d/base-test.sh) file provides the common harness that every shell test sources. It creates a temporary `$HOME` via `mktemp -d`, sets `OMARCHY_PATH="$ROOT"` to the checkout root, and builds a stub `bin/` directory that logs calls to prevent side effects. It also exports `pass`, `fail`, and `require_command` functions that output TAP-compatible results.

### Can the Omarchy test suite run without a graphical environment?

Yes. Both the CLI and shell suites run headless on standard CI runners. Shell tests that require a Wayland compositor call `require_compositor`, which skips the test gracefully when no socket is available, allowing the suite to pass on headless systems. Only the acceptance tests require a running compositor inside a VM.

### How are JavaScript modules tested within the shell suite?

The `run_node_test` helper executes Node.js unit tests from Bash by injecting a prelude that mirrors the shell assertion protocol. This enables testing pure JavaScript logic—such as [`shell/plugins/menu/MenuModel.js`](https://github.com/basecamp/omarchy/blob/main/shell/plugins/menu/MenuModel.js)—without spinning up a full graphical environment, outputting `pass`/`fail` results in TAP format that the shell runner can aggregate.