How to Write and Structure CLI Tests for Omarchy: Complete Guide to the TAP-Flavored Framework

Omarchy's CLI test suite is a single, self-contained Bash script at test/cli that implements a TAP-flavored assertion protocol, executes in strict isolation via temporary directories, and validates everything from JSON schema to metadata headers with immediate failure reporting.

Omarchy maintains command-line reliability through a disciplined testing approach that integrates routing verification, help rendering, and metadata validation into one executable specification. Learning how to write and structure CLI tests for Omarchy ensures your contributions align with the project's deterministic testing philosophy and strict isolation requirements.

Core Architecture of the Omarchy CLI Test Suite

Single-File Consolidation at test/cli

Unlike distributed test suites, Omarchy centralizes all CLI validation in test/cli, a solitary script that exercises the entire omarchy command surface. This file employs a lightweight TAP (Test Anything Protocol) convention, emitting ok or not ok status lines for every assertion. The script deliberately exits on the first failure within any test block, allowing the outer runner (./test/shell) to catch early terminations and continue reporting on subsequent files without aborting the entire suite.

TAP-Flavored Assertion Helpers

The script defines explicit assertion functions between lines 20 and 41 that standardize pass/fail semantics. The pass function prints ok - description, while fail emits not ok - description and terminates execution. For output inspection, assert_output_contains validates that command output includes expected substrings, providing clear failure messages when assertions miss. These helpers eliminate boilerplate and enforce consistent diagnostic formatting across all test scenarios.

Environment Isolation and Deterministic Setup

Repository Root Discovery and OMARCHY_PATH

Every test run begins with deterministic environment setup at lines 5 through 9. The script calculates ROOT=$(cd "$(dirname "$0")/.." && pwd) once, then exports OMARCHY_PATH="$ROOT" to ensure the CLI resolves resources relative to the repository root rather than the current working directory. This prevents path contamination when tests execute from arbitrary locations.

Temporary Directory Sanitization with make_tmpdir

Isolation is enforced through the make_tmpdir function (lines 11–19), which creates a scratch directory via mktemp -d and registers cleanup through a shell trap. The test harness constructs a minimal $PATH containing only $ROOT/bin and test-specific fake binaries, stripping system utilities that could introduce non-deterministic behavior. This pristine environment guarantees that omarchy commands resolve dependencies exclusively from the repository's controlled binary set.

Validation Strategies for CLI Output

JSON Schema Verification with jq

The CLI's omarchy commands --json output undergoes rigorous schema validation using jq filters between lines 66 and 97. Tests execute jq -e pipelines to assert the presence of required fields including summary, binary, and routes, verifying that the command discovery mechanism correctly infers routing tables from filesystem structure. These assertions catch breaking changes to the JSON API contract that would otherwise disrupt IDE integrations and shell completions.

Python-Based Cross-Validation for Command Groups

Lines 51 through 84 embed an inline Python block that parses the JSON command structure and validates group-level help output. This cross-check ensures that every command group listed in the metadata correctly renders its constituent commands when invoking omarchy help <group>. The dual-validation approach—Bash for invocation, Python for structural analysis—provides defense-in-depth against routing regressions.

Simulating External Dependencies

Stub Binary Implementation in FAKE_BIN

Commands that invoke system tools like tmux or gsettings are tested through stub binaries placed in a temporary FAKE_BIN directory (lines 125–150). These minimal Bash scripts echo their arguments to log files rather than executing side effects, allowing tests to verify that omarchy-theme-set-tmux and similar commands construct the correct invocation sequences. This technique enables safe testing of destructive operations like terminal multiplexer configuration without requiring actual tmux installations.

Metadata Compliance and Routing Edge Cases

Linting # omarchy: Headers in Executables

Lines 99 through 113 implement a metadata linter that scans every executable in bin/ matching the pattern omarchy-*. Using awk to extract leading comment blocks and grep -q to verify # omarchy:summary= presence, the test enforces that every command exposes its purpose through standardized metadata headers. This prevents undocumented commands from entering the release build.

Resilience Testing for Malformed Metadata

The suite deliberately creates temporary binaries with incomplete or unknown metadata fields—such as omarchy-weird-test and omarchy-partial-meta-test between lines 138 and 165—to verify graceful degradation. These tests confirm that the routing engine accepts commands missing optional metadata or containing deprecated fields without crashing or misrouting requests.

Trailing-Help Semantics Verification

Between lines 190 and 220, tests verify that --help or -h flags appearing after unresolved tokens trigger help display without executing the target command. The harness constructs temporary parent/child command pairs to exercise various flag positions, asserting that trailing help outputs documentation and exits zero rather than invoking potentially destructive operations.

Step-by-Step Guide to Adding New CLI Tests

Test Structure and Setup Pattern

Adding validation for new Omarchy commands follows a four-phase pattern within test/cli:

1. Allocate state: Invoke make_tmpdir VAR to create an isolated scratch space with automatic cleanup
2. Execute command: Capture output via output=$("$CLI" ...) using the CLI variable pointing to bin/omarchy
3. Assert behavior: Pipe results to assert_output_contains or jq for validation, or inspect fake binary logs for side effects
4. Signal completion: Terminate the block with pass "descriptive test name" to emit the TAP ok status

Complete Implementation Example

The following snippet demonstrates validating a new omarchy version subcommand:

# Example: verify that `omarchy version` prints the repository version

output=$("$CLI" version)
assert_output_contains "omarchy version prints repo version" "$output" "$(cat "$ROOT/version")"

Place this logic after the helper function definitions (post-line 41). The surrounding pass call marks the test successful, while any assertion failure triggers immediate script termination with a descriptive not ok message.

Summary

• Centralized testing: All CLI validation lives in a single test/cli script using TAP-flavored ok/not ok output
• Strict isolation: Tests run in mktemp-created directories with sanitized $PATH variables and OMARCHY_PATH locked to repository root
• Multi-layer validation: Combines Bash assertions, jq JSON filtering, and embedded Python to verify routing and help generation
• Dependency simulation: Fake binaries in FAKE_BIN log invocations instead of executing external tools like tmux
• Metadata enforcement: Automated linting ensures every bin/omarchy-* executable includes # omarchy: headers and handles malformed data gracefully

Frequently Asked Questions

What testing protocol does Omarchy use for CLI validation?

Omarchy implements a TAP-flavored protocol where the test/cli script prints ok - description for passing assertions and not ok - description for failures. The suite exits immediately on the first failed assertion, allowing the outer ./test/shell runner to capture the failure and continue with remaining test files.

How does Omarchy isolate CLI tests from the host system?

The make_tmpdir function creates temporary directories via mktemp -d and overrides $HOME and $PATH to include only the repository's bin/ directory and test-specific stubs. By setting OMARCHY_PATH="$ROOT" early in the script, tests operate against the repository root rather than system-wide installations, ensuring deterministic behavior regardless of host configuration.

Where should I add new test cases for Omarchy commands?

New assertions belong directly in the test/cli file after the helper function definitions (line 41 onwards). Each test should allocate temporary state with make_tmpdir, execute the command using the "$CLI" variable, validate output with assert_output_contains or jq, and conclude with a pass invocation to register the TAP success message.

How does Omarchy test commands that rely on external tools like tmux?

Instead of requiring actual tmux or gsettings installations, the test suite creates stub scripts in a FAKE_BIN directory that log their arguments to files. When the CLI executes, it finds these fakes in the sanitized $PATH, allowing tests to verify correct command construction by reading the log files without triggering external side effects.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}