How to Prevent Generated Files from Being Committed: Safeguards in AI Engineering Repositories

The rohitg00/ai-engineering-from-scratch repository prevents generated files like catalog.json and site/data.js from being committed through a three-layer defense system combining Git ignore policies, explicit automation contracts in AGENTS.md, and CI/CD workflows that exclusively control artifact regeneration.

Preventing generated files from being committed is critical for maintaining repository hygiene in AI engineering projects. In this codebase, ephemeral build artifacts such as catalog.json and site/data.js are systematically excluded from version control using source-control policies and automated enforcement mechanisms. This architecture ensures that generated outputs remain transient while curriculum metadata serves as the single source of truth.

Git Ignore Policies for Build Artifacts

The first line of defense operates at the Git level through .gitignore entries that permanently exclude specific files from the staging area. At line 69 of /.gitignore, the repository explicitly lists catalog.json as an ignored file:

# Generated files – never commit

catalog.json

This configuration ensures that git status will never list catalog.json as an untracked change, and git add commands will silently skip the file. When the readme-counts-sync job in the CI pipeline executes scripts/build_catalog.py, the resulting file exists only in the ephemeral workspace and disappears after the workflow completes.

Automation Contracts and Repository Policy

While site/data.js is not listed in .gitignore, the repository enforces a strict automation contract through policy documentation. Hard rule #7 in /AGENTS.md at line 51 explicitly declares that site/data.js is a generated file rebuilt by continuous integration and must never be version-controlled by human contributors.

This policy functions as a social contract reinforced during code review. Any pull request containing manual modifications to site/data.js violates the repository's invariant rules and is automatically rejected, regardless of the technical feasibility of committing the file.

CI/CD Workflows as Exclusive Controllers

The .github/workflows/curriculum.yml file implements mechanical safeguards through two specialized jobs that ensure generated files remain ephemeral and externally managed.

Site Rebuild Automation

The site-rebuild job runs exclusively on pushes to main and maintains sole custody of site/data.js updates. Located at lines 61-66 of the workflow file, this job executes node site/build.js and conditionally commits the output only when the CI process itself detects changes:

jobs:
  site-rebuild:
    name: site/data.js auto-rebuild (main only)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: rebuild site/data.js
        run: node site/build.js
      - name: commit + push if site/data.js changed
        run: |
          if git diff --quiet site/data.js; then
            echo "site/data.js already in sync"
            exit 0
          fi
          git config user.name "github-actions[bot]"
          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
          git add site/data.js
          git commit -m "chore(site): rebuild data.js"
          git push

This workflow guarantees that any modification to site/data.js originates from the automated build process rather than a developer's local environment.

Ephemeral Catalog Generation

The readme-counts-sync job executes scripts/build_catalog.py to produce catalog.json temporarily for README statistics updates. Because the file is ignored by Git and the workflow never explicitly stages it, the artifact exists solely within the runner's temporary storage and never reaches the repository history.

Build Scripts and Runtime Generation

The generation logic in site/build.js at line 5 creates site/data.js dynamically from phase and lesson metadata on every execution:

// site/build.js – generates data.js from all phases/lessons
const OUTPUT_PATH = path.join(__dirname, 'data.js');
// ...logic that reads phases/* and writes the aggregated JSON
fs.writeFileSync(OUTPUT_PATH, generatedData);

This build-time regeneration ensures that site/data.js always reflects the current state of the curriculum directories, rendering manual commits redundant and policy-violating. The script's existence reinforces the architectural principle that generated files are disposable outputs rather than source assets.

Summary

• .gitignore entries mechanically block catalog.json from Git's staging area at /.gitignore#L69.
• AGENTS.md Hard rule #7 establishes an automation contract forbidding manual commits of site/data.js at /AGENTS.md#L51.
• CI workflow exclusivity ensures only automated processes modify site/data.js via the site-rebuild job in /.github/workflows/curriculum.yml.
• Ephemeral generation by scripts/build_catalog.py and site/build.js makes version control of outputs technically unnecessary.
• Policy enforcement through code review rejects any pull requests containing manually committed generated files.

Frequently Asked Questions

Why doesn't site/data.js appear in .gitignore if it shouldn't be committed?

The file is intentionally tracked by Git but exclusively managed by the CI pipeline. Unlike catalog.json, which is completely ignored, site/data.js must exist in the repository for the site to function but can only be modified by the site-rebuild job. This hybrid approach allows the file to be deployed while preventing human contributors from altering it.

What happens if a contributor manually stages catalog.json?

Git will refuse to stage the file due to the explicit ignore rule at /.gitignore#L69. Even if a developer uses git add -f to force the file, the repository's code review policies and automated checks will reject the pull request for violating the "never commit generated files" rule documented in AGENTS.md.

How does the CI workflow prevent merge conflicts on site/data.js?

The site-rebuild job runs only on the main branch after pull requests merge, ensuring linear updates from a single authority. By restricting write access to the GitHub Actions bot and checking for diffs before committing (git diff --quiet), the workflow eliminates race conditions and guarantees that site/data.js reflects the canonical state of the curriculum metadata.

Can these safeguards be adapted for other generated files like model binaries or temporary datasets?

Yes. The pattern scales effectively: add patterns to .gitignore for completely ephemeral files, document hard rules in AGENTS.md for files requiring CI-exclusive management, and implement dedicated workflow jobs that regenerate and commit artifacts when necessary. This three-layer approach works for any generated asset in AI engineering workflows, from trained model weights to compiled documentation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}