How to Create `test-prompts.json` with Specific Test Case Types for Cangjie Skills

test-prompts.json is the test-suite definition file that validates Cangjie skill trigger logic, requiring three mandatory test case types: should_trigger, should_not_trigger, and edge_case.

Creating a valid test-prompts.json file is essential for every skill in the kangarooking/cangjie-skill repository. This JSON file defines how the automated test harness verifies that your skill activates when it should, ignores irrelevant prompts, and handles boundary conditions gracefully. The file follows a strict schema defined in templates/test-prompts.json.template.

Core Structure of test-prompts.json

The test file contains five top-level fields that the CI pipeline consumes:

Field	Purpose
skill	Unique slug matching the skill's manifest (replaces {{skill-slug}})
source_book	Human-readable provenance format: {{BOOK_TITLE}} — {{AUTHOR}}
test_cases	Array of individual test prompts with types and expectations
minimum_pass_rate	Global threshold (default 0.8) for aggregate test success
darwin_compatible	Boolean flag for Darwin-skill evolution pipeline eligibility

The Three Required Test Case Types

Each test-prompts.json must include all three types with their specified minimums:

should_trigger (Minimum 3 Cases)

These prompts must cause skill activation with concrete, verifiable actions.

{
  "id": "should-trigger-01",
  "type": "should_trigger",
  "prompt": "我在社交聚会里总是说不出话，怎么办？",
  "expected_behavior": "应激活 effective-communication, 并给出三条开场白技巧",
  "notes": "典型求助场景，正面激活 skill"
}

should_not_trigger (Minimum 2 Cases, Zero Tolerance)

These "bait" prompts must not activate the skill. All cases in this type must pass—any false trigger fails the entire skill validation. Include at least one cross-sibling bait that routes to another skill.

{
  "id": "should-not-trigger-02",
  "type": "should_not_trigger",
  "prompt": "帮我写一封商务邮件",
  "expected_behavior": "不应激活本 skill, 应激活 business-writing",
  "notes": "跨 skill 混淆诱饵：应当交给 sibling-skill-slug = business-writing"
}

edge_case (Minimum 1 Case)

Ambiguous scenarios testing boundary logic where activation may be conditional, but the rationale must be documented.

{
  "id": "edge-01",
  "type": "edge_case",
  "prompt": "我想让对方更喜欢我，帮我想点儿技巧",
  "expected_behavior": "应激活 effective-communication, 因为请求涉及人际技巧，但需说明这是一般性建议而非具体情境",
  "notes": "边界模糊：请求既可视为通用技巧，也可能需要上下文"
}

Step-by-Step Creation Process

Follow these seven steps to build a compliant test-prompts.json:

1. Copy the template from templates/test-prompts.json.template to your skill root directory.

2. Replace placeholders with your skill slug and source book metadata.

3. Add three should_trigger cases covering distinct activation scenarios.

4. Add two should_not_trigger cases, including one explicit cross-skill bait.

5. Add one edge_case documenting a boundary scenario with clear rationale.

6. Set minimum_pass_rate (retain 0.8 unless your skill demands stricter validation).

7. Validate syntax using jsonlint or the repository test command before submission.

Complete Example: effective-communication Skill

This runnable example demonstrates all required fields and test case types for a skill derived from How to Talk to Anyone — Leil Lowndes:

{
  "skill": "effective-communication",
  "version": "0.1.0",
  "source_book": "How to Talk to Anyone — Leil Lowndes",
  "darwin_compatible": true,
  "test_cases": [
    {
      "id": "should-trigger-01",
      "type": "should_trigger",
      "prompt": "我在社交聚会里总是说不出话，怎么办？",
      "expected_behavior": "应激活 effective-communication, 并给出三条开场白技巧",
      "notes": "典型求助场景，正面激活 skill"
    },
    {
      "id": "should-trigger-02",
      "type": "should_trigger",
      "prompt": "请教我如何在第一印象中留下好印象",
      "expected_behavior": "应激活 effective-communication, 并提供 5 条第一印象技巧",
      "notes": "直接请求技巧"
    },
    {
      "id": "should-trigger-03",
      "type": "should_trigger",
      "prompt": "怎样在面试时自然地自我介绍？",
      "expected_behavior": "应激活 effective-communication, 并生成一个面试自我介绍模板",
      "notes": "面试情境属于该 skill 范畴"
    },
    {
      "id": "should-not-trigger-01",
      "type": "should_not_trigger",
      "prompt": "给我推荐一本关于沟通的好书",
      "expected_behavior": "不应激活本 skill, 因为这是纯推荐请求",
      "notes": "诱饵：与技能无关的书籍推荐"
    },
    {
      "id": "should-not-trigger-02",
      "type": "should_not_trigger",
      "prompt": "帮我写一封商务邮件",
      "expected_behavior": "不应激活本 skill, 应激活 business-writing",
      "notes": "跨 skill 混淆诱饵：应当交给 sibling-skill-slug = business-writing"
    },
    {
      "id": "edge-01",
      "type": "edge_case",
      "prompt": "我想让对方更喜欢我，帮我想点儿技巧",
      "expected_behavior": "应激活 effective-communication, 因为请求涉及人际技巧，但需说明这是一般性建议而非具体情境",
      "notes": "边界模糊：请求既可视为通用技巧，也可能需要上下文"
    }
  ],
  "minimum_pass_rate": 0.8,
  "notes": "至少 3 条 should_trigger + 2 条 should_not_trigger + 1 条 edge_case。全部 should_not_trigger 必须通过 (诱饵容错为 0), 且其中至少 1 条是同书兄弟 skill 的场景 (跨 skill 混淆测试)。"
}

Key Source Files in the Repository

File	Purpose	Path
Master template	Defines schema and placeholder tokens	templates/test-prompts.json.template
Skill specification	CI pipeline consumption rules	[SKILL.md](https://github.com/kangarooking/cangjie-skill/blob/main/SKILL.md)
Pressure test stage	Runtime execution details	[methodology/06-stage4-pressure-test.md](https://github.com/kangarooking/cangjie-skill/blob/main/methodology/06-stage4-pressure-test.md)

Summary

To create test-prompts.json with specific test case types in the Cangjie skill framework:

• Use the template at templates/test-prompts.json.template as your starting point
• Include all three mandatory types: should_trigger (≥3), should_not_trigger (≥2, zero tolerance), and edge_case (≥1)
• Document rationale in notes fields for maintainability and automated review
• Route cross-skill baits explicitly using sibling skill slugs
• Validate JSON syntax before submission to prevent CI pipeline failures

Frequently Asked Questions

What happens if should_not_trigger tests fail?

All should_not_trigger cases must pass with zero tolerance. A single false trigger fails the entire skill validation, as these tests verify the skill does not fire in inappropriate contexts where it could interfere with other skills.

Can I use more than the minimum required test cases?

Yes. The minimums (3/2/1) are floors, not ceilings. Adding more diverse cases—especially additional cross-skill baits and edge cases—improves robustness and is encouraged for production skills.

Where does the test runner execute these prompts?

The pressure-test stage runs them against the actual skill implementation. As documented in [methodology/06-stage4-pressure-test.md](https://github.com/kangarooking/cangjie-skill/blob/main/methodology/06-stage4-pressure-test.md), this stage evaluates whether the skill's trigger logic matches the declared expected_behavior for each case type.

What is darwin_compatible and do I need it?

Keep darwin_compatible: true unless you have a specific reason to disable it. This flag enables the skill for evolution by the Darwin-skill pipeline, which automatically improves skills based on test performance. Setting it to false excludes your skill from this optimization loop.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}