What Test Artifacts Does AI-DLC Generate in the Construction Phase?

AI-DLC generates five distinct test artifacts during the Construction phase: unit-test source files, test configuration files, test-execution scripts, numerical test-file counters, and machine-readable test-report artifacts.

The awslabs/aidlc-workflows repository implements a structured workflow where the Construction phase produces build-time assets that enable automated testing in subsequent CI stages. These artifacts provide the actual test code, runner configuration, execution entry points, and metrics required to validate generated applications.

Overview of Construction Phase Test Artifacts

During the Construction phase, the AI-DLC workflow engine analyzes generated source code and emits a complete testing infrastructure. According to the workflow documentation in docs/WORKING-WITH-AIDLC.md, the Construction phase prepares assets so that "Build and Test closes out the work" (lines 356–358). This closure depends on artifact discovery mechanisms implemented in the metrics and reporting packages.

The artifact generation process creates files across three categories:

• Code artifacts: Test source files that exercise generated modules
• Configuration artifacts: Runner settings for pytest, Jest, and other frameworks
• Metadata artifacts: Counters and reports that track test suite composition

Unit-Test Source Files

The Construction phase generates one test file per source file, following naming conventions like test_my_module.py for a corresponding my_module.py. These files contain test functions that directly exercise the newly generated code.

The metrics.py module discovers these files during the artifact scanning process. In scripts/aidlc-evaluator/packages/execution/src/aidlc_runner/metrics.py, the _scan_artifacts function (lines 85–132) identifies test files by checking for the test_ prefix:

def _scan_artifacts(root: Path) -> dict:
    source_files = test_files = config_files = other_files = 0
    for path in root.rglob("*"):
        if path.is_file():
            if path.suffix in {".py", ".js", ".ts"}:
                source_files += 1
            elif path.name.startswith("test_"):
                test_files += 1
            elif path.name in {"pytest.ini", "jest.config.js"}:
                config_files += 1
            else:
                other_files += 1
    return {
        "source_files": source_files,
        "test_files": test_files,
        "config_files": config_files,
        "other_files": other_files,
    }

This discovery mechanism ensures that every generated test file is accounted for in the final metrics, regardless of language-specific testing frameworks.

Test Configuration Files

AI-DLC generates framework-specific configuration files that allow CI harnesses to discover and execute tests. These include pytest.ini for Python projects, jest.config.js for JavaScript/TypeScript projects, and analogous files for other supported languages.

The reporting system tracks these as config_files within the Artifacts dataclass. In scripts/aidlc-evaluator/packages/reporting/src/reporting/collector.py (lines 50–55), the data structure captures these alongside source and test counters:

@dataclass
class Artifacts:
    source_files: int
    test_files: int
    config_files: int
    other_files: int

These configuration files are essential for the Build & Test stage to execute with correct coverage settings, markers, and environment variables.

Test-Execution Scripts

When a "Build & Test" standing instruction is present in the workflow configuration, the Construction phase emits executable scripts or CI workflow definitions that invoke the test runner. These artifacts typically appear as shell scripts (e.g., run_tests.sh) or CI pipeline definitions (e.g., GitHub Actions workflows).

A generated CI workflow artifact typically follows this structure:

name: CI
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install dependencies
        run: uv sync --dev
      - name: Run tests
        run: uv run pytest --cov

These scripts encapsulate the exact flags and environment setup required to execute the generated test suite with coverage reporting enabled.

Test Metrics and Counters

The Construction phase maintains a running count of test artifacts through the artifacts.test_files metric. This integer is stored in the run-metrics and updated by the _scan_artifacts function in metrics.py (lines 85–132).

The counter serves two purposes:

1. Validation: Ensures that every source file has a corresponding test file
2. Reporting: Provides quantitative data for the evaluation framework to assess completeness

The metrics collector distinguishes between source files (identified by extensions like .py, .js, .ts) and test files (identified by the test_ prefix), ensuring accurate categorization in the final report.

Test-Report Artifacts

Upon completion of the Construction phase, AI-DLC serializes the Artifacts data into a machine-readable format, typically YAML, that becomes part of the evaluation bundle. This summary includes the test-file count, configuration file inventory, and other metadata.

The golden-artifact example in scripts/aidlc-evaluator/test_cases/all-stages/golden-aidlc-docs/aidlc-state.md (line 23) demonstrates this output format, which evaluation frameworks consume to verify that the Construction phase generated the expected testing infrastructure.

This artifact enables downstream systems to verify test suite completeness without scanning the entire file system, providing a compact summary of the generated testing assets.

Summary

AI-DLC generates a comprehensive testing infrastructure during the Construction phase that includes:

• Unit-test source files: One per generated module, identified by the test_ prefix
• Test configuration files: Framework-specific settings (pytest.ini, jest.config.js) tracked in the config_files metric
• Test-execution scripts: Shell scripts or CI workflows that invoke test runners with coverage flags
• Test-file counters: Integer metrics updated by _scan_artifacts() in metrics.py and stored in artifacts.test_files
• Test-report artifacts: YAML-serialized summaries of the Artifacts dataclass from collector.py

These artifacts collectively enable the "Build & Test" stage to validate generated code automatically and provide quantitative evidence of test coverage.

Frequently Asked Questions

What naming convention does AI-DLC use for generated test files?

AI-DLC prefixes test files with test_ (e.g., test_my_module.py). The _scan_artifacts function in scripts/aidlc-evaluator/packages/execution/src/aidlc_runner/metrics.py specifically checks for this prefix using path.name.startswith("test_") to identify and count test files during the artifact discovery phase.

Where does AI-DLC store the test file count during the Construction phase?

The test file count is stored in the artifacts.test_files field of the run-metrics. This value is populated by the _scan_artifacts function in metrics.py (lines 85–132) and later serialized through the Artifacts dataclass defined in scripts/aidlc-evaluator/packages/reporting/src/reporting/collector.py (lines 50–55).

How does AI-DLC distinguish between source files and test files?

The metrics scanner distinguishes files by name and extension. Source files are identified by extensions (.py, .js, .ts), while test files are identified by the test_ prefix. Configuration files like pytest.ini and jest.config.js are tracked separately in the config_files counter according to the logic in metrics.py.

What role does the Construction phase play in the AI-DLC workflow?

According to docs/WORKING-WITH-AIDLC.md (lines 356–358), the Construction phase prepares all build-time assets so that "Build and Test closes out the work." This phase generates the actual test code, configuration, and execution scripts required for the subsequent validation stage, ensuring that generated applications enter the Build & Test stage with a complete, executable test suite.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}