How the `refine` Command Performs Incremental Gap Analysis for CLI Coverage in CLI-Anything

The refine command incrementally expands CLI coverage by inventorying existing commands, rescanning target software for capabilities, computing a prioritized gap report, and auto-generating new Click commands without removing existing functionality.

The HKUDS/CLI-Anything framework uses a two-stage workflow where the initial /cli-anything command generates a baseline harness, and the subsequent /cli-anything:refine command closes the coverage gap between exposed CLI functionality and the target application’s full API surface. This process is designed to be completely additive, allowing developers to iteratively converge on production-grade CLI completeness through repeated incremental gap analysis for CLI coverage.

Broad vs. Focused Refinement Modes

The refine agent supports two invocation patterns that control the scope of analysis.

Broad refinement scans the entire application surface. When you invoke /cli-anything:refine <path>, the agent reads all public APIs, CLI tools, and scripting hooks in the target directory, then compares them against the existing coverage map.

Focused refinement limits analysis to a specific functional domain. The command /cli-anything:refine <path> "focus text" restricts the scan to capabilities matching the focus string, performing targeted gap analysis only for that slice.

The Six-Step Incremental Workflow

According to cli-anything-plugin/commands/refine.md, the refinement process follows six well-defined steps that preserve existing functionality while adding new coverage.

Step 1: Inventory Current Coverage

The agent first reads the generated <software>_cli.py file, all modules in core/ and utils/, and the existing test suite. It constructs a persistent coverage map structured as {function_name: covered | not_covered} that serves as the baseline for comparison. This map ensures subsequent runs only address new gaps.

Step 2: Analyze Software Capabilities

The agent re-scans the source tree at <software-path> (or the focused subset) to extract every public API, CLI sub-command, batch-mode operation, and scripting hook that produces observable output. This includes renders, exports, conversions, and internal filters.

Step 3: Gap Analysis

The system compares the inventoried coverage against the full capability set to generate a gap report. This report ranks missing functions by three criteria:

• High impact – frequently used functions that deliver immediate value.
• Easy wins – simple APIs requiring minimal wrapper code.
• Composability – functions that unlock complex workflows when combined with existing commands.

The agent presents this prioritized list for user confirmation before proceeding.

Step 4: Implement New Commands

For each approved gap, the generator creates a Click command (or sub-command) following the architectural patterns defined in cli-anything-plugin/HARNESS.md. The implementation includes proper Click groups, --json flags, session handling, and error handling. New wrappers are placed in core/ or utils/ and wired into the REPL skin.

Step 5: Expand Tests

The workflow auto-generates unit tests in test_core.py and end-to-end tests in test_full_e2e.py for every new command. This ensures that newly added coverage does not break existing functionality, maintaining the invariant that refinement is purely additive.

Step 6: Update Documentation

Finally, the agent updates README.md, TEST.md, and the software-specific SOP (<SOFTWARE>.md) to reflect the new commands and capabilities.

Implementation Patterns and Code Structure

The incremental gap analysis logic relies on specific implementation files within the repository.

cli-anything-plugin/HARNESS.md defines the mandatory patterns for command implementation, including Click group structures and standard flags. All code generated during Step 4 must conform to these specifications.

macrocli/agent-harness/cli_anything/macrocli/backends/gui_agent.py contains the _instruct_with_refine function (lines 419–556), which implements an analogous instruct → refine loop for GUI-based backends. This function sends refine prompts to the LLM, receives refined action JSON, and re-executes after undoing the original step—mirroring the high-level CLI coverage refinement loop.

Test assertions in test_iterative_refinement and test_core verify that newly added commands appear while previously existing ones remain untouched, confirming the non-destructive nature of the process.

Practical Usage Examples

Broad Refinement for Full Coverage

Scan an entire application to identify all missing capabilities:

# Analyze GIMP's complete API surface and expand the CLI

/cli-anything:refine ./gimp

This command inventories current commands in <software>_cli.py, scans GIMP’s source for filters and export options, displays a gap report, and generates new Click commands plus tests.

Focused Refinement for Specific Features

Target only the video-in-video and picture-in-picture functionality:

# Limit analysis to compositing features only

/cli-anything:refine ./shotcut "vid-in-vid and picture-in-picture compositing"

Example Generated Command

The following Python code illustrates the pattern used when implementing new commands during refinement:

@click.command()
@click.argument("input_file", type=click.Path(exists=True))
@click.argument("output_file", type=click.Path())
@click.option("--filter", default="none")
def apply_filter(input_file: str, output_file: str, filter: str) -> None:
    """Apply an image filter using the GIMP backend."""
    result = gimp_backend.apply_filter(input_file, filter=filter)
    write_output(result, output_file)

This structure follows the conventions in HARNESS.md, including type-hinted arguments, Click path validators, and standardized output handling.

Summary

• The refine command performs incremental gap analysis for CLI coverage by comparing existing harness capabilities against the target software’s full API surface.
• Two modes support both broad scans and focused analysis of specific functional areas.
• A six-step workflow ensures persistent coverage maps, prioritized gap reports, and non-destructive implementation of new Click commands.
• All generated code follows patterns defined in HARNESS.md, with corresponding tests in test_core.py and test_full_e2e.py.
• The process is fully additive—existing commands, tests, and documentation are never removed, only enhanced.

Frequently Asked Questions

How does the refine command avoid duplicate work across multiple runs?

The command persists the coverage map generated in Step 1 across invocations. This map tracks which functions are already wrapped as {function_name: covered | not_covered}, ensuring each subsequent run only analyzes new gaps between the existing CLI and the current software state.

What criteria does the gap analysis use to prioritize new commands?

The gap report ranks missing capabilities by high impact (frequently used functions), easy wins (simple APIs requiring minimal wrapper code), and composability (functions that enable new workflows when combined with existing commands).

Can I refine a specific feature without scanning the entire codebase?

Yes. Use focused refinement by providing a focus string: /cli-anything:refine <path> "feature description". This limits Step 2’s capability analysis to the functional area described, generating targeted additions without processing unrelated APIs.

Where is the incremental logic implemented for GUI-based applications?

The instruct → refine loop for GUI automation is implemented in macrocli/agent-harness/cli_anything/macrocli/backends/gui_agent.py within the _instruct_with_refine function (lines 419–556). This mirrors the CLI refinement pattern by undoing original steps and re-executing refined actions based on gap analysis.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}