# How Cangjie-Skill Handles Parallel Extraction of Knowledge: A 5-Agent Pipeline Explained > Discover how Cangjie-skill achieves parallel knowledge extraction with a 5-agent pipeline. Learn about simultaneous agent execution, automatic chunking, and fallback mechanisms for efficient data processing. - Repository: [kangarooking/cangjie-skill](https://github.com/kangarooking/cangjie-skill) - Tags: deep-dive - Published: 2026-08-11 --- **Cangjie-skill uses a five-stage workflow where Stage 1 spawns five specialized extractor agents simultaneously, each targeting a distinct knowledge type (frameworks, principles, cases, counter-examples, and glossary terms), with automatic chunking for large texts and graceful fallback to serial execution when parallel runtime isn't available.** The `kangarooking/cangjie-skill` repository implements an agentic pipeline designed to extract methodological knowledge from long-form content like books or video transcripts. **Parallel extraction** is the cornerstone of its first stage, enabling comprehensive coverage without sacrificing processing speed. This article examines the architecture, implementation details, and fallback mechanisms that make this approach reliable. ## How Parallel Extraction Works in Stage 1 The extraction pipeline begins with a single coordinated call to the Claude Agent tool. According to the source code in [`SKILL.md`](https://github.com/kangarooking/cangjie-skill/blob/main/SKILL.md) (lines 81-89), the main skill launches **five independent Task agents** in one operation—not sequentially—maximizing throughput and coverage. Each agent receives identical global context ([`BOOK_OVERVIEW.md`](https://github.com/kangarooking/cangjie-skill/blob/main/BOOK_OVERVIEW.md) plus the source text or chunk) paired with its dedicated extractor prompt. They execute **concurrently**, writing results to separate candidate files without blocking one another. ## The Five Specialized Extractor Agents Each sub-agent corresponds to one extractor definition in the `extractors/` folder. The following table maps agents to their responsibilities and output locations: | Agent | Prompt File | Knowledge Type Extracted | Output Path | |-------|-------------|--------------------------|-------------| | Framework extractor | [`extractors/framework-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/framework-extractor.md) | Decision frameworks, mental models | `books//candidates/framework.md` | | Principle extractor | [`extractors/principle-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/principle-extractor.md) | Principles, rules | `books//candidates/principle.md` | | Case extractor | [`extractors/case-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/case-extractor.md) | Concrete author examples | `books//candidates/case.md` | | Counter-example extractor | [`extractors/counter-example-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/counter-example-extractor.md) | Failure patterns, traps | `books//candidates/counter-example.md` | | Glossary extractor | [`extractors/glossary-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/glossary-extractor.md) | Term definitions | `books//candidates/glossary.md` | As documented in [`extractors/framework-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/framework-extractor.md) (lines 33-40), each extractor operates independently: it reads the book overview, scans the (potentially chunked) text, and writes its YAML-formatted candidates to its designated file. This separation of concerns ensures no single agent is overwhelmed and each knowledge type receives focused attention. ## Handling Large Texts Through Chunking When source material exceeds a single sub-agent's context window, Cangjie-skill implements **boundary-aware chunking**. As specified in [`methodology/02-stage1-parallel-extract.md`](https://github.com/kangarooking/cangjie-skill/blob/main/methodology/02-stage1-parallel-extract.md) (lines 24-32), the system splits content on natural divisions: - Chapter boundaries - Volume breaks - Video part transitions **Critical design invariant**: every chunk includes the complete [`BOOK_OVERVIEW.md`](https://github.com/kangarooking/cangjie-skill/blob/main/BOOK_OVERVIEW.md) as a global anchor. This ensures extractors maintain holistic reasoning about the entire book while processing only a limited slice, preserving coherence across chunks. ## Graceful Degradation to Serial Execution The pipeline anticipates runtime constraints. Per [`methodology/02-stage1-parallel-extract.md`](https://github.com/kangarooking/cangjie-skill/blob/main/methodology/02-stage1-parallel-extract.md) (lines 22-23), when the execution environment cannot support parallel agents, Cangjie-skill **degrades gracefully** by running the same five extractor prompts sequentially. Output formats remain identical, ensuring pipeline compatibility regardless of infrastructure. ## Post-Extraction Verification (Stage 1.5) Parallelism introduces potential duplication. The pipeline addresses this through **Stage 1.5 triple verification**, described in [`SKILL.md`](https://github.com/kangarooking/cangjie-skill/blob/main/SKILL.md) (lines 98-105). After all sub-agents complete: 1. Duplicate or overlapping candidates are identified and merged 2. Sourcing confidence is validated 3. Only well-attested knowledge units proceed to subsequent stages This consolidation step guarantees that speed from parallelism never compromises quality. ## Implementation Example The following Python pseudocode illustrates the parallel spawn pattern as implemented in the actual agent tool: ```python # Define the five extractor configurations agents = [ {"name": "framework_extractor", "prompt": read("extractors/framework-extractor.md")}, {"name": "principle_extractor", "prompt": read("extractors/principle-extractor.md")}, {"name": "case_extractor", "prompt": read("extractors/case-extractor.md")}, {"name": "counter_extractor", "prompt": read("extractors/counter-example-extractor.md")}, {"name": "glossary_extractor", "prompt": read("extractors/glossary-extractor.md")}, ] # Single call spawns all agents concurrently results = agent_tool.spawn_many( agents=agents, inputs={ "book_overview": read("books/slug/BOOK_OVERVIEW.md"), "source_text": large_text_or_chunks } ) # Each extractor writes to its dedicated candidate file for r in results: write(f"books/slug/candidates/{r['type']}.md", r["yaml_output"]) ``` ## Key Source Files | File | Role in Parallel Extraction | |------|----------------------------| | [`methodology/02-stage1-parallel-extract.md`](https://github.com/kangarooking/cangjie-skill/blob/main/methodology/02-stage1-parallel-extract.md) | Documents parallel strategy, chunking logic, and serial fallback | | [`SKILL.md`](https://github.com/kangarooking/cangjie-skill/blob/main/SKILL.md) | Pipeline overview with parallel spawn step (lines 81-89, 98-105) | | [`extractors/framework-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/framework-extractor.md) | Template for extractor prompts; defines output format and candidate file writing | | [`extractors/principle-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/principle-extractor.md) | Dedicated prompt for principle/rule extraction | | [`extractors/case-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/case-extractor.md) | Dedicated prompt for concrete example extraction | | [`extractors/counter-example-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/counter-example-extractor.md) | Dedicated prompt for failure pattern extraction | | [`extractors/glossary-extractor.md`](https://github.com/kangarooking/cangjie-skill/blob/main/extractors/glossary-extractor.md) | Dedicated prompt for term definition extraction | | `books//candidates/` | Runtime directory storing raw parallel outputs before verification | ## Summary - **Concurrency model**: Five specialized agents launch simultaneously via single `agent_tool.spawn_many()` call - **Knowledge coverage**: Each agent targets one knowledge type (frameworks, principles, cases, counter-examples, glossary) - **Scalability**: Boundary-aware chunking with global context preservation handles arbitrary text lengths - **Resilience**: Automatic fallback to serial execution maintains pipeline functionality on constrained runtimes - **Quality assurance**: Stage 1.5 verification deduplicates and validates parallel outputs before downstream processing ## Frequently Asked Questions ### What happens if two extractors find the same knowledge unit? The **Stage 1.5 triple verification** process identifies duplicates or overlaps and merges them. According to [`SKILL.md`](https://github.com/kangarooking/cangjie-skill/blob/main/SKILL.md) (lines 98-105), only candidates with confident sourcing survive this consolidation, ensuring the pipeline deduplicates without losing valid extractions. ### Can I run Cangjie-skill on hardware that doesn't support parallel agents? Yes. The pipeline automatically **degrades to serial execution**, running the five extractor prompts sequentially while preserving identical output formats and file locations. This fallback is documented in [`methodology/02-stage1-parallel-extract.md`](https://github.com/kangarooking/cangjie-skill/blob/main/methodology/02-stage1-parallel-extract.md) (lines 22-23). ### How does chunking affect extraction quality? Each chunk includes the **complete [`BOOK_OVERVIEW.md`](https://github.com/kangarooking/cangjie-skill/blob/main/BOOK_OVERVIEW.md)** as a global anchor. Per [`methodology/02-stage1-parallel-extract.md`](https://github.com/kangarooking/cangjie-skill/blob/main/methodology/02-stage1-parallel-extract.md) (lines 24-32), this design lets extractors reason about the whole book's context even when processing limited slices, maintaining coherence across boundaries. ### Where are the parallel extraction results stored before verification? Each extractor writes to its own candidate file in `books//candidates/`, using the pattern `.md` (e.g., [`framework.md`](https://github.com/kangarooking/cangjie-skill/blob/main/framework.md), [`principle.md`](https://github.com/kangarooking/cangjie-skill/blob/main/principle.md)). These files serve as raw inputs to Stage 1.5 verification before qualified candidates advance to later pipeline stages.