Implement the Spec Pipeline for Transforming Requirements into Executable Specs in aios-core

The Spec Pipeline in aios-core is an ADE workflow orchestrated by Epic3Executor that transforms raw requirements into executable specifications through five automated phases—gather, assess, research, write, and critique—driven by declarative YAML configuration and AI agent prompts.

The Spec Pipeline is the core mechanism in aios-core for converting user stories and product requirements into machine-readable development specifications. Implemented within the Autonomous Development Engine (ADE), this pipeline leverages Epic3Executor to orchestrate a declarative workflow defined in spec-pipeline.yaml, ensuring consistent, repeatable transformation of requirements into executable specs.

Architectural Overview of the Spec Pipeline

The Spec Pipeline operates as Epic 3 within the aios-core Autonomous Development Engine (ADE). It is implemented by the Epic3Executor class, which interprets the declarative workflow defined in .aios-core/development/workflows/spec-pipeline.yaml to automate the transformation of raw requirements into structured, executable specifications.

The pipeline follows a five-phase sequential process: gather requirements, assess complexity, research dependencies, write specification, and critique quality. Each phase maps to a dedicated AI agent role—Product Manager (@pm), Architect (@architect), Analyst (@analyst), and Quality Assurance (@qa)—ensuring specialized handling of distinct specification aspects.

Core Components and File Structure

The Spec Pipeline implementation spans orchestration logic, workflow configuration, and agent prompt templates:

Component	File Path	Purpose
MasterOrchestrator	.aios-core/core/orchestration/master-orchestrator.js	Global orchestrator that creates epics, tracks execution state, and emits lifecycle events.
Epic3Executor	.aios-core/core/orchestration/executors/epic-3-executor.js	Implements the Spec Pipeline logic, loading the YAML workflow and iterating through SPEC_PHASES.
EpicExecutor (base)	.aios-core/core/orchestration/executors/epic-executor.js	Provides common utilities including _log, _addArtifact, path resolution, and status handling.
spec-pipeline.yaml	.aios-core/development/workflows/spec-pipeline.yaml	Declarative workflow definition containing phases, triggers, pre-flight checks, and resume configuration.
Task Prompts	.aios-core/development/tasks/spec-*.md	Markdown templates (e.g., spec-gather-requirements.md) containing prompts for AI agents.
CLI Entry	bin/aios.js	Parses *create-spec commands and instantiates the MasterOrchestrator.

The Five Execution Phases

The Epic3Executor processes the SPEC_PHASES array, executing each phase by loading the corresponding task markdown file from .aios-core/development/tasks/.

Gather Requirements

The pipeline begins by invoking the @pm agent via spec-gather-requirements.md. This phase extracts functional requirements, user stories, and acceptance criteria from the input PRD or prompt.

Assess Complexity

Using spec-assess-complexity.md, the @architect agent evaluates technical complexity, identifies potential risks, and determines the appropriate depth of specification required. This phase can trigger complexity-based branching in the workflow.

Research Dependencies

The @analyst agent executes spec-research-dependencies.md to investigate external libraries, API contracts, and system dependencies. This ensures the specification accounts for integration constraints and third-party limitations.

Write Specification

The critical write-spec phase generates the actual spec.md file. If agents are not yet fully wired, Epic3Executor._createStubSpec generates a minimal specification containing metadata (story ID, timestamp, detected tech stack) to ensure downstream phases have valid input.

Critique

The final quality gate uses spec-critique.md to invoke the @qa agent. This phase reviews the generated specification for completeness, consistency, and feasibility, potentially blocking the pipeline if critical issues are detected.

Running the Spec Pipeline via CLI

Execute the pipeline using the aios-core CLI:

node bin/aios.js *create-spec STORY-42

The CLI parses the *create-spec command in bin/aios.js, instantiates MasterOrchestrator, and dispatches to Epic3Executor. The orchestrator resolves the story context, validates pre-flight checks defined in spec-pipeline.yaml, and initiates the five-phase execution loop.

Extending the Spec Pipeline

Adding a New Phase

To insert a custom phase, create the task markdown and update the executor configuration:

1. Create the task prompt file:

cat > .aios-core/development/tasks/spec-security-review.md <<'EOF'

# Security Review

Review the spec for security vulnerabilities, OWASP compliance, and data-privacy concerns.
EOF

2. Update the SPEC_PHASES array in .aios-core/core/orchestration/executors/epic-3-executor.js:

const SPEC_PHASES = [
  'gather-requirements',
  'assess-complexity',
  'research-dependencies',
  'write-spec',
  'security-review',   // <-- new entry
  'critique',
];

3. Register the phase in spec-pipeline.yaml under the phases section with the appropriate agent assignment.

Customizing Pre-flight Checks

Add validation logic to the pre_flight block in spec-pipeline.yaml:

pre_flight:
  checks:
    - id: lint_spec
      description: "Run eslint on generated spec"
      script: |
        const { execSync } = require('child_process');
        const specPath = `docs/stories/${storyId}/spec/spec.md`;
        try { execSync(`npm run lint -- ${specPath}`); return true; }
        catch { return false; }
      blocking: false
      warning: "Spec does not pass lint – pipeline will continue but flagged"

Modifying Output Directories

The output directory resolves via _getPath in the base executor. Override the default in spec-pipeline.yaml or via environment variables:

const outDir = this._getPath(
  this.orchestrator.projectRoot,
  this.orchestrator.config?.outputDir?.replace('{storyId}', storyId) ?? `docs/stories/${storyId}/spec`
);

Resuming Failed Executions

The pipeline supports checkpoint-based recovery through the resume configuration in spec-pipeline.yaml. State persists to docs/stories/{storyId}/spec/.pipeline-state.json.

To resume a failed run:

node bin/aios.js *create-spec STORY-42 --resume

The MasterOrchestrator reads the state file, identifies the last successful checkpoint (e.g., research_complete), and Epic3Executor jumps to the next phase (write-spec) without regenerating prior artifacts.

Programmatic Integration with the Node API

Integrate the Spec Pipeline directly into Node.js applications:

const { MasterOrchestrator } = require('./.aios-core/core/orchestration');
const path = require('path');

// Initialize orchestrator
const orchestrator = new MasterOrchestrator(path.resolve(__dirname));

// Prepare execution context
const context = {
  storyId: 'STORY-007',
  source: 'user',
  prdPath: null,
  techStack: ['node', 'express'],
};

// Execute Spec Pipeline
(async () => {
  const epic3 = orchestrator.getEpicExecutor(3);
  const result = await epic3.execute(context);
  console.log('Generated spec at:', result.artifacts.spec);
})();

Summary

• The Spec Pipeline is implemented as Epic 3 in aios-core, orchestrated by Epic3Executor and defined declaratively in spec-pipeline.yaml.
• Five distinct phases—gather, assess, research, write, and critique—execute sequentially via task markdown files located in .aios-core/development/tasks/.
• Pre-flight checks and resume capabilities are configured in the YAML workflow, enabling robust error handling and checkpoint recovery via .pipeline-state.json.
• Extend the pipeline by modifying the SPEC_PHASES array in epic-3-executor.js and adding corresponding task files, or integrate programmatically via the MasterOrchestrator Node API.

Frequently Asked Questions

What is the Spec Pipeline in aios-core?

The Spec Pipeline is an Autonomous Development Engine (ADE) workflow that transforms raw product requirements into executable, machine-readable specifications. It is implemented as Epic 3 within the aios-core repository and orchestrated by the Epic3Executor class, which processes a declarative YAML workflow to automate specification generation.

How do I add a custom phase to the Spec Pipeline?

To add a custom phase, create a new task markdown file in .aios-core/development/tasks/ (e.g., spec-security-review.md), then append the phase identifier to the SPEC_PHASES array in .aios-core/core/orchestration/executors/epic-3-executor.js. Finally, register the phase configuration in spec-pipeline.yaml with the appropriate agent assignment and execution rules.

Can I resume a failed Spec Pipeline execution?

Yes, the pipeline supports resumption via checkpoint persistence. Execution state is automatically saved to docs/stories/{storyId}/spec/.pipeline-state.json after each phase. To resume a failed run, execute node bin/aios.js *create-spec <storyId> --resume, and the MasterOrchestrator will restore the last successful checkpoint and continue from the next pending phase.

Where are the generated specifications stored?

By default, generated specifications are stored at docs/stories/{storyId}/spec/spec.md, with pipeline state maintained in the same directory at .pipeline-state.json. This output path is configurable via the outputDir field in spec-pipeline.yaml or through environment variables that modify the orchestrator configuration before initialization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}