Typical Outputs of the REVIEW Stage in the ARS Pipeline

The REVIEW stage in the Academic Research Skills (ARS) pipeline generates a comprehensive review package containing an editorial decision, five independent reviewer reports, a consensus indicator, a detailed revision roadmap, and a confidence score.

The REVIEW stage (Stage 3) in the Imbad0202/academic-research-skills repository serves as the first external-editorial-review checkpoint in the Academic Research Skills (ARS) pipeline. When the orchestrator dispatches the academic-paper-reviewer skill in full mode, the stage produces a structured "review package" that provides all artifacts required for authors and downstream automation to determine the next steps.

The Six Components of the REVIEW Stage Output

When the academic-paper-reviewer skill executes in full mode, the editorial_synthesizer_agent consolidates the panel's work into six distinct output components defined in shared/handoff_schemas.md.

Editorial Decision

The editorial decision is a categorical field (line 63 in shared/handoff_schemas.md) that takes one of four values: Accept, Minor Revision, Major Revision, or Reject. This decision drives the next pipeline transition, determining whether the paper advances to REVISE, returns to WRITE, or exits the pipeline.

Five Independent Reviewer Reports

The package contains exactly five independent ReviewerReport objects (defined lines 73-79 in shared/handoff_schemas.md) representing the review panel:

• EIC: Editor-in-Chief
• R1: Methodology reviewer
• R2: Domain reviewer
• R3: Interdisciplinary reviewer
• DA: Devil's Advocate

Each report includes dimension scores, strengths, weaknesses, and specific questions.

Consensus Indicator

The consensus field (line 66 in shared/handoff_schemas.md) captures the panel's overall agreement as one of: CONSENSUS-4, CONSENSUS-3, SPLIT, or DA-CRITICAL. This indicator alerts the orchestrator when the Devil's Advocate dissent is critical or when the panel is divided.

Revision Roadmap

The revision roadmap consists of prioritized RoadmapItem objects (lines 10-18 in shared/handoff_schemas.md). Each item includes an ID, description, assigned reviewer, type (Major/Minor), priority level, target section, suggested action, and verification criteria. This structured data enables automated progress tracking in the subsequent REVISE stage.

Confidence Score

An integer confidence score from 0-100 (line 67 in shared/handoff_schemas.md) represents the editorial team's collective confidence in the decision. Low scores may trigger additional human oversight at the mandatory checkpoint.

Optional Metadata

The schema supports additional fields such as revision_roadmap.consensus_summary and revision_roadmap.dissenting_opinions to capture nuanced panel dynamics not reflected in the formal consensus score.

How the Review Package Is Generated

The artifacts are produced by the editorial_synthesizer_agent, which aggregates and normalizes the five individual reviewer reports into a single handoff package. According to academic-pipeline/SKILL.md (lines 99-103), this stage operates as a mandatory checkpoint—the orchestrator stores the outputs in the state tracker and pauses execution for user review before proceeding to REVISE or RE-REVIEW.

JSON Schema and Example Output

The review package conforms to the schema defined in shared/handoff_schemas.md. Below is a minimal JSON example illustrating the structure:

{
  "editorial_decision": "Minor Revision",
  "reviewer_reports": [
    {
      "reviewer_id": "EIC",
      "role": "Editor‑in‑Chief",
      "dimension_scores": {"novelty": 8, "rigor": 7},
      "strengths": ["Clear contribution", "Strong theoretical grounding"],
      "weaknesses": [
        {"description": "Limited sample size", "severity": "major", "type": "methodology"}
      ],
      "questions": ["Can you provide a power analysis?"]
    },
    {
      "reviewer_id": "R1",
      "role": "Methodology Reviewer",
      "dimension_scores": {"methodology": 6},
      "strengths": ["Appropriate statistical tests"],
      "weaknesses": [
        {"description": "Missing data handling description", "severity": "minor", "type": "methodology"}
      ],
      "questions": []
    }
    /* R2, R3, DA omitted for brevity */
  ],
  "consensus": "CONSENSUS-3",
  "revision_roadmap": {
    "items": [
      {
        "id": "REV-001",
        "description": "Add power analysis for the primary experiment",
        "reviewer": "EIC,R1",
        "type": "Major",
        "priority": "must_fix",
        "target_section": "Methods",
        "suggested_action": "Include a post‑hoc power analysis",
        "verification_criteria": "Power ≥ 0.8"
      }
    ],
    "total_items": 1,
    "must_fix_count": 1,
    "editorial_decision": "Minor Revision",
    "consensus_summary": "Panel agrees on need for stronger statistical justification."
  },
  "confidence_score": 78
}

Key Source Files Reference

The following files in the Imbad0202/academic-research-skills repository define the REVIEW stage outputs:

• academic-pipeline/SKILL.md: Lists the REVIEW stage deliverables (5 reports + decision + roadmap) at lines 99-103.
• shared/handoff_schemas.md: Formal JSON schema for the review package (ReviewerReport, RoadmapItem, and top-level metadata fields).
• academic-paper-reviewer/SKILL.md: Describes the reviewer panel configuration and "full" review mode.
• examples/showcase/stage3_review_report.pdf: Real-world example of a synthesized REVIEW stage output.
• shared/sprint_contract.schema.json: Sprint contract schema that gates the REVIEW stage.

Summary

• The REVIEW stage (Stage 3) produces a structured review package containing six discrete components when operating in full mode.
• The editorial decision (Accept, Minor Revision, Major Revision, Reject) determines the next pipeline transition.
• Five independent reports (EIC, R1, R2, R3, DA) provide multi-perspective coverage including methodology, domain expertise, interdisciplinary insight, and contrarian critique.
• The revision roadmap encodes actionable items with verification criteria, enabling automated compliance tracking in downstream stages.
• Outputs are generated by the editorial_synthesizer_agent and stored at a mandatory checkpoint before the pipeline continues.

Frequently Asked Questions

What are the possible editorial decisions in the ARS REVIEW stage?

The editorial decision field accepts one of four categorical values: Accept, Minor Revision, Major Revision, or Reject. This field is defined in shared/handoff_schemas.md (line 63) and serves as the decisive signal that drives the next pipeline transition, determining whether the paper moves to the REVISE stage, returns to WRITE, or exits the pipeline entirely.

How many reviewers participate in the full REVIEW stage?

The full mode of the academic-paper-reviewer skill engages exactly five independent reviewers: the Editor-in-Chief (EIC), a Methodology reviewer (R1), a Domain reviewer (R2), an Interdisciplinary reviewer (R3), and a Devil's Advocate (DA). This panel structure ensures robust, multi-dimensional coverage of the submitted paper.

What is the purpose of the Devil's Advocate reviewer?

The Devil's Advocate (DA) serves as a contrarian reviewer specifically tasked with identifying weaknesses, assumptions, and alternative interpretations that other reviewers might miss. The DA's input is critical for surfacing potential flaws; when the DA raises critical concerns, the consensus indicator registers as DA-CRITICAL, prompting the orchestrator to flag the submission for additional scrutiny.

Where is the REVIEW stage output schema formally defined?

The complete JSON schema for the review package is defined in shared/handoff_schemas.md within the Imbad0202/academic-research-skills repository. This file specifies the structure for ReviewerReport objects (lines 73-79), RoadmapItem objects (lines 10-18), and the top-level editorial_decision, consensus, and confidence_score fields (lines 63-67).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}