# How to Write and Structure CLI Tests for Omarchy: Complete Guide to the TAP-Flavored Framework

> Learn to write and structure CLI tests for Omarchy using its TAP-flavored framework. This guide covers isolated execution, schema validation, and immediate failure reporting.

- Repository: [Omacom/omarchy](https://github.com/omacom/omarchy)
- Tags: testing
- Published: 2026-09-13

---

**Omarchy's CLI test suite is a single, self-contained Bash script at `test/cli` that implements a TAP-flavored assertion protocol, executes in strict isolation via temporary directories, and validates everything from JSON schema to metadata headers with immediate failure reporting.**

Omarchy maintains command-line reliability through a disciplined testing approach that integrates routing verification, help rendering, and metadata validation into one executable specification. Learning how to write and structure CLI tests for Omarchy ensures your contributions align with the project's deterministic testing philosophy and strict isolation requirements.

## Core Architecture of the Omarchy CLI Test Suite

### Single-File Consolidation at `test/cli`

Unlike distributed test suites, Omarchy centralizes all CLI validation in **`test/cli`**, a solitary script that exercises the entire `omarchy` command surface. This file employs a lightweight TAP (Test Anything Protocol) convention, emitting `ok` or `not ok` status lines for every assertion. The script deliberately **exits on the first failure** within any test block, allowing the outer runner (`./test/shell`) to catch early terminations and continue reporting on subsequent files without aborting the entire suite.

### TAP-Flavored Assertion Helpers

The script defines explicit assertion functions between lines 20 and 41 that standardize pass/fail semantics. The **`pass`** function prints `ok - description`, while **`fail`** emits `not ok - description` and terminates execution. For output inspection, **`assert_output_contains`** validates that command output includes expected substrings, providing clear failure messages when assertions miss. These helpers eliminate boilerplate and enforce consistent diagnostic formatting across all test scenarios.

## Environment Isolation and Deterministic Setup

### Repository Root Discovery and `OMARCHY_PATH`

Every test run begins with deterministic environment setup at lines 5 through 9. The script calculates **`ROOT=$(cd "$(dirname "$0")/.." && pwd)`** once, then exports **`OMARCHY_PATH="$ROOT"`** to ensure the CLI resolves resources relative to the repository root rather than the current working directory. This prevents path contamination when tests execute from arbitrary locations.

### Temporary Directory Sanitization with `make_tmpdir`

Isolation is enforced through the **`make_tmpdir`** function (lines 11–19), which creates a scratch directory via `mktemp -d` and registers cleanup through a shell `trap`. The test harness constructs a minimal `$PATH` containing only `$ROOT/bin` and test-specific fake binaries, stripping system utilities that could introduce non-deterministic behavior. This pristine environment guarantees that `omarchy` commands resolve dependencies exclusively from the repository's controlled binary set.

## Validation Strategies for CLI Output

### JSON Schema Verification with `jq`

The CLI's `omarchy commands --json` output undergoes rigorous schema validation using `jq` filters between lines 66 and 97. Tests execute **`jq -e`** pipelines to assert the presence of required fields including `summary`, `binary`, and `routes`, verifying that the command discovery mechanism correctly infers routing tables from filesystem structure. These assertions catch breaking changes to the JSON API contract that would otherwise disrupt IDE integrations and shell completions.

### Python-Based Cross-Validation for Command Groups

Lines 51 through 84 embed an inline Python block that parses the JSON command structure and validates group-level help output. This cross-check ensures that every command group listed in the metadata correctly renders its constituent commands when invoking `omarchy help <group>`. The dual-validation approach—Bash for invocation, Python for structural analysis—provides defense-in-depth against routing regressions.

## Simulating External Dependencies

### Stub Binary Implementation in `FAKE_BIN`

Commands that invoke system tools like `tmux` or `gsettings` are tested through stub binaries placed in a temporary **`FAKE_BIN`** directory (lines 125–150). These minimal Bash scripts echo their arguments to log files rather than executing side effects, allowing tests to verify that `omarchy-theme-set-tmux` and similar commands construct the correct invocation sequences. This technique enables safe testing of destructive operations like terminal multiplexer configuration without requiring actual `tmux` installations.

## Metadata Compliance and Routing Edge Cases

### Linting `# omarchy:` Headers in Executables

Lines 99 through 113 implement a metadata linter that scans every executable in `bin/` matching the pattern `omarchy-*`. Using `awk` to extract leading comment blocks and `grep -q` to verify `# omarchy:summary=` presence, the test enforces that every command exposes its purpose through standardized metadata headers. This prevents undocumented commands from entering the release build.

### Resilience Testing for Malformed Metadata

The suite deliberately creates temporary binaries with incomplete or unknown metadata fields—such as `omarchy-weird-test` and `omarchy-partial-meta-test` between lines 138 and 165—to verify graceful degradation. These tests confirm that the routing engine accepts commands missing optional metadata or containing deprecated fields without crashing or misrouting requests.

### Trailing-Help Semantics Verification

Between lines 190 and 220, tests verify that `--help` or `-h` flags appearing after unresolved tokens trigger help display without executing the target command. The harness constructs temporary parent/child command pairs to exercise various flag positions, asserting that trailing help outputs documentation and exits zero rather than invoking potentially destructive operations.

## Step-by-Step Guide to Adding New CLI Tests

### Test Structure and Setup Pattern

Adding validation for new Omarchy commands follows a four-phase pattern within `test/cli`:

1. **Allocate state**: Invoke **`make_tmpdir VAR`** to create an isolated scratch space with automatic cleanup
2. **Execute command**: Capture output via `output=$("$CLI" ...)` using the CLI variable pointing to `bin/omarchy`
3. **Assert behavior**: Pipe results to **`assert_output_contains`** or `jq` for validation, or inspect fake binary logs for side effects
4. **Signal completion**: Terminate the block with **`pass "descriptive test name"`** to emit the TAP ok status

### Complete Implementation Example

The following snippet demonstrates validating a new `omarchy version` subcommand:

```bash

# Example: verify that `omarchy version` prints the repository version

output=$("$CLI" version)
assert_output_contains "omarchy version prints repo version" "$output" "$(cat "$ROOT/version")"

```

Place this logic after the helper function definitions (post-line 41). The surrounding `pass` call marks the test successful, while any assertion failure triggers immediate script termination with a descriptive `not ok` message.

## Summary

- **Centralized testing**: All CLI validation lives in a single `test/cli` script using TAP-flavored `ok/not ok` output
- **Strict isolation**: Tests run in `mktemp`-created directories with sanitized `$PATH` variables and `OMARCHY_PATH` locked to repository root
- **Multi-layer validation**: Combines Bash assertions, `jq` JSON filtering, and embedded Python to verify routing and help generation
- **Dependency simulation**: Fake binaries in `FAKE_BIN` log invocations instead of executing external tools like `tmux`
- **Metadata enforcement**: Automated linting ensures every `bin/omarchy-*` executable includes `# omarchy:` headers and handles malformed data gracefully

## Frequently Asked Questions

### What testing protocol does Omarchy use for CLI validation?

Omarchy implements a TAP-flavored protocol where the `test/cli` script prints `ok - description` for passing assertions and `not ok - description` for failures. The suite exits immediately on the first failed assertion, allowing the outer `./test/shell` runner to capture the failure and continue with remaining test files.

### How does Omarchy isolate CLI tests from the host system?

The `make_tmpdir` function creates temporary directories via `mktemp -d` and overrides `$HOME` and `$PATH` to include only the repository's `bin/` directory and test-specific stubs. By setting `OMARCHY_PATH="$ROOT"` early in the script, tests operate against the repository root rather than system-wide installations, ensuring deterministic behavior regardless of host configuration.

### Where should I add new test cases for Omarchy commands?

New assertions belong directly in the `test/cli` file after the helper function definitions (line 41 onwards). Each test should allocate temporary state with `make_tmpdir`, execute the command using the `"$CLI"` variable, validate output with `assert_output_contains` or `jq`, and conclude with a `pass` invocation to register the TAP success message.

### How does Omarchy test commands that rely on external tools like tmux?

Instead of requiring actual `tmux` or `gsettings` installations, the test suite creates stub scripts in a `FAKE_BIN` directory that log their arguments to files. When the CLI executes, it finds these fakes in the sanitized `$PATH`, allowing tests to verify correct command construction by reading the log files without triggering external side effects.