# Understanding the darwin-skill Format for test-prompts.json

> Learn the darwin-skill format for test-prompts.json in kangarooking/cangjie-skill. Understand this JSON schema for automated skill testing and validation with activation and non-activation scenarios.

- Repository: [kangarooking/cangjie-skill](https://github.com/kangarooking/cangjie-skill)
- Tags: internals
- Published: 2026-07-21

---

**The darwin-skill format is a standardized JSON schema that defines how Instagit skills are tested, ensuring automated validation through structured test cases with specific activation and non-activation scenarios.**

The [`test-prompts.json`](https://github.com/kangarooking/cangjie-skill/blob/main/test-prompts.json) file sits at the heart of the skill validation pipeline in the `kangarooking/cangjie-skill` repository. This darwin-skill format enables the Darwin test runner to automatically verify that a skill behaves correctly across positive triggers, negative examples, and edge cases before deployment.

## Structure of the darwin-skill JSON Schema

The darwin-skill format requires a specific top-level object structure that describes the skill metadata and its validation suite.

### Top-Level Metadata Fields

According to the canonical template in `templates/test-prompts.json.template`, the JSON object must include these fields:

- **skill**: The slug identifier for the skill (e.g., `{{skill-slug}}`)
- **version**: Semantic version tracking (typically `"0.1.0"`)
- **source_book**: Human-readable citation format (`{{BOOK_TITLE}} — {{AUTHOR}}`)
- **darwin_compatible**: Boolean flag (must be `true`) signaling that the file conforms to the Darwin test harness expectations
- **minimum_pass_rate**: Decimal threshold (e.g., `0.8`) representing the required success ratio across all `should_trigger` cases
- **notes**: Author guidance regarding test case composition requirements
- **test_cases**: Array of individual test scenario objects

### Test Case Definitions

Each object within the `test_cases` array must specify:

- **id**: Unique identifier for the test case (e.g., `should-trigger-01`)
- **type**: Classification as `should_trigger`, `should_not_trigger`, or `edge_case`
- **prompt**: The exact user utterance to feed the skill
- **expected_behavior**: Detailed description of required skill response or explicit non-activation
- **notes**: Optional rationale explaining the scenario's significance

## Why the darwin-skill Format Is Necessary

The strict schema serves five critical functions in the Instagit ecosystem:

**Standardized Validation**: The `darwin_compatible` flag tells Instagit's Darwin test harness that the JSON complies with the expected schema, enabling automated loading and execution without manual parsing.

**Deterministic Behavior Verification**: By explicitly declaring `should_trigger` versus `should_not_trigger` cases, the format verifies activation logic against both positive and negative scenarios, preventing false positives in production.

**Cross-Skill Guardrails**: The template enforces at least one `should_not_trigger` case referencing a sibling skill (`sibling-skill-slug`), preventing accidental hijacking of other skills within the same book.

**Quantitative Quality Gates**: The `minimum_pass_rate` field provides a concrete metric; skills only pass validation when achieving the configured success threshold across mandatory test types.

**Living Documentation**: Fields such as `source_book`, per-case `notes`, and structured `id` values create maintainable audit trails for future skill updates.

## Implementing the darwin-skill Format

### Minimal test-prompts.json Example

```json
{
  "skill": "my-awesome-skill",
  "version": "0.1.0",
  "source_book": "Great Book — Jane Doe",
  "darwin_compatible": true,
  "test_cases": [
    {
      "id": "should-trigger-01",
      "type": "should_trigger",
      "prompt": "给我推荐一本关于时间旅行的小说",
      "expected_behavior": "应激活 my-awesome-skill, 并 返回推荐列表",
      "notes": "正面场景：用户明确请求推荐"
    },
    {
      "id": "should-not-trigger-01",
      "type": "should_not_trigger",
      "prompt": "今天的天气怎么样？",
      "expected_behavior": "不应激活本 skill, 因为它是天气查询，不在本 skill 范畴",
      "notes": "诱饵：与本 skill 无关的对话"
    },
    {
      "id": "edge-01",
      "type": "edge_case",
      "prompt": "我想要一本适合儿童的时间旅行故事，但不要太科幻",
      "expected_behavior": "应返回适合儿童的时间旅行书目，说明为何符合筛选条件",
      "notes": "边界：混合需求需要合理判断"
    }
  ],
  "minimum_pass_rate": 0.8,
  "notes": "至少 2 条 should_trigger + 1 条 should_not_trigger + 1 条 edge_case。"
}

```

### Validation with the Darwin Runner

While the actual Darwin runner is part of Instagit's internal tooling, the expected contract follows this pattern:

```python
from darwin import TestRunner, load_skill_tests

# Load the JSON description

tests = load_skill_tests("path/to/test-prompts.json")

# Execute the suite

runner = TestRunner()
result = runner.run(tests)

print(f"Pass rate: {result.pass_rate:.2%}")
assert result.pass_rate >= tests["minimum_pass_rate"]

```

## Key Files in the Repository

Several files work together to enforce the darwin-skill format:

- **`templates/test-prompts.json.template`**: The canonical schema definition that all skill authors must follow
- **[`SKILL.md`](https://github.com/kangarooking/cangjie-skill/blob/main/SKILL.md)**: Human-readable specification referencing the JSON validation requirements
- **[`README.md`](https://github.com/kangarooking/cangjie-skill/blob/main/README.md)**: Repository overview and contribution guidelines
- **[`methodology/07-stage5-deliver.md`](https://github.com/kangarooking/cangjie-skill/blob/main/methodology/07-stage5-deliver.md)**: Documentation for the final validation stage where Darwin tests execute

## Summary

- The darwin-skill format requires specific metadata fields including `darwin_compatible: true` and a `minimum_pass_rate` threshold
- Test cases must cover three types: `should_trigger`, `should_not_trigger`, and `edge_case`
- The schema prevents skill conflicts by requiring negative test cases against sibling skills
- Validation occurs through Instagit's Darwin test harness, which parses the standardized JSON automatically
- The template file at `templates/test-prompts.json.template` provides the authoritative reference implementation

## Frequently Asked Questions

### What is the darwin-skill format?

The darwin-skill format is a JSON schema used in the `kangarooking/cangjie-skill` repository to define automated test suites for Instagit skills. It specifies how to structure test cases, metadata, and success criteria so that the Darwin test runner can validate skill behavior automatically.

### Why is the darwin_compatible field required?

The `darwin_compatible` boolean flag signals to Instagit's test infrastructure that the JSON file follows the expected schema. Without this field set to `true`, the Darwin runner will reject the file, preventing execution of the test suite and blocking deployment.

### What are the required test case types in the darwin-skill format?

The format recognizes three mandatory test types: `should_trigger` for positive activation scenarios, `should_not_trigger` for negative examples (including sibling skill references), and `edge_case` for boundary testing. Authors must include at least two positive cases, one negative case, and one edge case per skill.

### How does the minimum_pass_rate field work?

The `minimum_pass_rate` field defines a decimal threshold (typically `0.8` or 80%) that the skill must achieve across all `should_trigger` test cases. The Darwin runner calculates the actual pass rate during validation, and skills failing to meet this threshold are rejected from production deployment.