How to Test Caveman Locally: Complete Verification Guide

Run python3 tests/verify_repo.py from the repository root to execute Caveman's comprehensive local verification suite, which validates skill front-matters, synchronized files, JSON manifests, JavaScript syntax, PowerShell scripts, and the Claude hook install flow.

The Caveman project (JuliusBrussee/caveman) ships with a self-contained Python test harness located in tests/verify_repo.py. This pure-Python suite requires no external test runners like pytest and exercises every runtime component—from Markdown skill definitions to the compression CLI—to ensure the repository installs cleanly on fresh machines.

Prerequisites for Local Testing

Before running the verification suite, ensure your environment meets the following requirements:

• Node.js 18+: Executes hook scripts (*.js) and the Node-based installer (bin/install.js).
• Python 3.8+: Runs the main verification driver tests/verify_repo.py.
• Bash: POSIX-compatible shell for parsing src/hooks/*.sh.
• PowerShell 5.1+: Validates Windows-specific install scripts in src/hooks/*.ps1.
• Git: Provides the .git directory used by the install flow verification.

Install the runtime dependencies:

# Install Node dependencies from package-lock

npm ci

# Install Python dependencies (only required for benchmark scripts)

python3 -m pip install --quiet -r requirements.txt

Architecture of the Test Harness

The verify_repo.py script contains isolated check functions that abort on the first failure. Each validates a specific subsystem:

• verify_skill_frontmatter_upload_compatibility: Parses skills/caveman/SKILL.md to ensure skill description fields contain no XML-like angle brackets, which is required for safe upload compatibility.
• verify_synced_files: Confirms that the canonical skill file skills/caveman/SKILL.md is identical to the copy shipped in plugins/caveman/skills/caveman/SKILL.md, and that the packed dist/caveman.skill archive contains the same content.
• verify_manifests_and_syntax: Parses all JSON manifest files (.claude-plugin/*.json, .codex/hooks.json, gemini-extension.json, etc.) and runs node --check on every hook script plus bash -n on shell scripts to catch syntax errors.
• verify_powershell_static: Validates that Windows installer scripts reference the expected JavaScript hooks and remain compatible with PowerShell 5.1.
• verify_compress_fixtures: Loads reference markdown files under tests/caveman-compress/, runs the scripts.validate module against their compressed counterparts, and asserts that detection heuristics flag them as compressible.
• verify_compress_cli: Executes the CLI (python -m scripts) in three paths: a skip path (file already detected as non-NL), a missing-file path (error handling), and the happy path (basic detection).
• verify_hook_install_flow: Simulates a fresh user $HOME/.claude configuration, runs src/hooks/install.sh, verifies generated settings.json contains expected SessionStart and UserPromptSubmit hooks, activates the mode via caveman-activate.js, exercises the tracker via caveman-mode-tracker.js, checks the status-line badge via caveman-statusline.sh, and confirms uninstall.sh restores original settings.

Running the Verification Suite

Execute the Full Test Suite

Navigate to the repository root and run:

python3 tests/verify_repo.py

Expected output (truncated):

== Skill Frontmatter Upload Compatibility ==
Skill frontmatter descriptions avoid XML-like tags
...
All local verification checks passed

Isolate Individual Checks

To run a single check, edit the checks list at the bottom of verify_repo.py:

checks = [
    # verify_skill_frontmatter_upload_compatibility,

    # verify_synced_files,

    verify_compress_fixtures,  # <-- Only this check runs

    # verify_compress_cli,

    # verify_hook_install_flow,

]

Then re-execute:

python3 tests/verify_repo.py

You will see output like:

== Compress Fixtures
Validated 6 caveman-compress fixture pairs

Manual Testing of Specific Components

Testing the Claude Hook Install Flow

For interactive validation of the Claude integration, simulate a fresh environment:

# Create a temporary home directory to avoid polluting your actual config

TMP_HOME=$(mktemp -d)
export HOME=$TMP_HOME
export CLAUDE_CONFIG_DIR=$HOME/.claude

# Run the installer

bash src/hooks/install.sh

# Verify the generated settings contain expected hooks

cat $HOME/.claude/settings.json | jq '.hooks | keys'

# Test activation (should print a banner)

node src/hooks/caveman-activate.js

# Test mode switching via the tracker

echo '{"prompt":"/caveman ultra"}' | node src/hooks/caveman-mode-tracker.js

# Check the status-line badge

bash src/hooks/caveman-statusline.sh

All steps should complete without errors. The final badge line resembles:

[CAVEMAN:ULTRA] 12.4k tokens saved

Testing the Compression CLI

Validate the compression detection logic:

cd skills/caveman-compress
python3 -m scripts ../../src/hooks/install.sh

Sample output when the detector skips non-natural language content:

Detected: code
Skipping: file is not natural language

Key Source Files in the Test Harness

• tests/verify_repo.py: Main verification driver containing all check functions.
• skills/caveman/SKILL.md: Canonical markdown skill definition (source of front-matter tests).
• plugins/caveman/skills/caveman/SKILL.md: Synced copy validated against the canonical file.
• dist/caveman.skill: Packaged zip archive validated against source.
• src/hooks/*.js: JavaScript hook scripts (caveman-activate.js, caveman-mode-tracker.js, etc.).
• src/hooks/*.sh: Bash install, uninstall, and status-line scripts.
• src/hooks/*.ps1: PowerShell equivalents for Windows validation.
• tests/caveman-compress/*.original.md: Reference markdown files for compression fixtures.
• bin/install.js: Node entry-point used by the npx caveman installer.

Summary

• Run the full suite with python3 tests/verify_repo.py to validate the entire repository.
• Prerequisites include Node 18+, Python 3.8+, Bash, and PowerShell 5.1+.
• Isolate failures by commenting out checks in the main() function of verify_repo.py.
• Manual verification of Claude hooks requires simulating a fresh $HOME/.claude directory.
• Pure Python harness requires no external test runners and aborts on first failure with concise diagnostics.

Frequently Asked Questions

What command runs the Caveman local tests?

Execute python3 tests/verify_repo.py from the repository root. This single command runs the comprehensive verification suite that checks every component from skill front-matters to hook install flows.

Do I need to install pytest or another test runner?

No. The test harness in tests/verify_repo.py is pure Python and uses no external test frameworks. Each verification function runs independently and prints results directly to stdout, aborting immediately on the first failure with a clear diagnostic message.

How do I test only the compression features?

Edit the checks list in main() within tests/verify_repo.py to include only verify_compress_fixtures and verify_compress_cli, then rerun the script. Alternatively, manually test the CLI by running python3 -m scripts <file_path> from the skills/caveman-compress directory.

Why does the hook install flow test require a temporary HOME directory?

The verify_hook_install_flow function and manual testing procedures create a temporary $HOME to simulate a fresh user environment without overwriting your actual Claude configuration. This ensures the test validates the complete install, activation, and uninstall cycle safely.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}