How to Extend case-review with Custom Evidence Validation in reverse-skill

Extend case-review with custom evidence validation by declaring a new constant set in skills/case-review/scripts/review_case.py, injecting validation logic into the parse_evidence() function, and optionally exposing a CLI flag in main() to toggle the rule.

The case-review skill in the zhaoxuya520/reverse-skill repository provides a read-only evidence-graph auditor that validates forensic case directories against strict schemas. When you need to enforce domain-specific constraints beyond the default checks for workitems, timelines, and scope, you can extend case-review with custom evidence validation by modifying the Python validation script directly.

Understanding the case-review Validation Architecture

The validation engine resides in skills/case-review/scripts/review_case.py. It audits five primary artefacts in every case directory (work/<case>):

• workitems.md: Validates table format, IDs, and status values against the WORKITEM_STATUSES constant set via parse_workitems()
• timeline.md: Verifies chronological ordering and cross-references work-item IDs via parse_timeline()
• scope.md: Checks required fields (targets, network_mode, etc.) and enumerated values against SEVERITIES, EVIDENCE_STATUSES, PATH_TYPES, and NETWORK_MODES via parse_scope()
• evidence/: Parses markdown evidence records, cross-references IDs, and optionally verifies SHA-256 hashes via parse_evidence() when --verify-hashes is passed
• reports/: Guarantees generated reports contain expected sections via parse_reports()

All allowed-value checks use constant sets defined near the top of the file. The issue() function records validation problems with severity levels, while render_markdown() or render_json() generates the final audit report.

Step-by-Step Guide to Adding Custom Evidence Validation

Step 1 – Declare Custom Validation Constants (Lines 21–25)

Add your domain-specific allowed values near the existing constant sets at the top of skills/case-review/scripts/review_case.py:

# skills/case-review/scripts/review_case.py

# lines 21-25 – extend with your custom set

CUSTOM_EVIDENCE_TYPES = {"artifact", "log", "packet", "my_custom_type"}

This follows the same pattern as the built-in sets like EVIDENCE_STATUSES and SEVERITIES.

Step 2 – Inject Validation Logic into parse_evidence() (Line 286)

The parse_evidence() function begins at line 286. Insert your custom check after the existing field validations (around line 320):

# Inside parse_evidence(), after existing field checks (around line 320)

evidence_type = field_value(evidence_md, "type")
if evidence_type and evidence_type not in CUSTOM_EVIDENCE_TYPES:
    issue(
        issues,
        "error",
        "evidence.type",
        f"unsupported custom evidence type: {evidence_type}",
        f"{evidence_path}:{evidence_line}",
    )

This leverages the existing issue() infrastructure to report errors with precise file paths and line numbers, maintaining consistency with the native validation errors.

Step 3 – Expose a CLI Flag in main() (Line 440)

To make the validation optional, modify the argument parser in main() around line 440:

parser.add_argument(
    "--allow-custom-type",
    action="store_true",
    help="Enable validation of custom evidence types defined in CUSTOM_EVIDENCE_TYPES",
)

Update the review_case() function signature at line 348 to accept the new parameter and forward it to parse_evidence():

def review_case(case_root, strict=False, verify_hashes=False, allow_custom_type=False):
    # ... implementation that passes allow_custom_type to parse_evidence ...

    pass

# In main():

review_case(
    case_root,
    strict=args.strict,
    verify_hashes=args.verify_hashes,
    allow_custom_type=args.allow_custom_type
)

Testing Your Custom Validation Rules

Validate your modifications using the existing test suite in skills/case-review/tests/test_review_case.py. Use the default tests as templates for asserting that your new validation logic catches invalid custom evidence types while permitting valid ones. The test file guarantees that default validation passes, providing a safety net when adding domain-specific extensions.

Summary

• Define constants: Add new allowed-value sets near lines 21-25 in skills/case-review/scripts/review_case.py
• Extend parse_evidence(): Insert validation logic after line 320 in the parse_evidence() function to check custom fields against your constants
• Add CLI toggles: Modify main() (line 440) and review_case() (line 348) to accept optional flags for flexible, toggleable validation
• Test thoroughly: Use skills/case-review/tests/test_review_case.py as a template for unit testing custom rules without breaking existing audits

Frequently Asked Questions

What file contains the core validation logic for case-review?

The Python script skills/case-review/scripts/review_case.py implements all validation logic, including the parse_evidence(), parse_workitems(), and parse_timeline() functions that audit case directories and enforce schema compliance.

How do I add a new allowed value set for evidence validation?

Declare a new constant set near the top of skills/case-review/scripts/review_case.py (around lines 21-25), then reference that set in the appropriate parsing function—such as parse_evidence()—to validate field values against your custom enumeration.

Can I make custom validation rules optional via command-line flags?

Yes. Add an argument to the argparse configuration in main() (around line 440), update the review_case() function signature (line 348) to accept the parameter, and pass the flag value down to the specific parsing function to enable or disable the rule at runtime.

Where should I add unit tests for my custom validation rules?

Add test cases to skills/case-review/tests/test_review_case.py, following the existing test patterns that validate default rules. This ensures your custom evidence validation integrates correctly with the existing audit pipeline and maintains backward compatibility.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}