# How to Implement Incremental Builds in Gatsby: CI/CD Optimization Guide > Optimize Gatsby CI/CD with incremental builds. Learn how to regenerate only affected pages, slashing build times from minutes to seconds by persisting the Gatsby cache. - Repository: [Gatsby/gatsby](https://github.com/gatsbyjs/gatsby) - Tags: how-to-guide - Published: 2026-03-06 --- **Gatsby incremental builds regenerate only pages affected by data changes by persisting the `.cache` and `public` directories between runs, cutting build times from minutes to seconds when properly configured in CI/CD pipelines.** The **gatsbyjs/gatsby** repository implements incremental builds as a core runtime feature since version 3.0, automatically tracking node changes and selectively rendering HTML without requiring explicit flags in open-source deployments. This guide explains the underlying mechanisms, critical source files, and CI/CD configurations needed to maintain persistent state across builds. ## How Incremental Builds Work in Gatsby Gatsby’s incremental build system operates through a sophisticated dirty-page tracking mechanism that compares current data against cached state. The implementation relies on several interconnected subsystems within the core packages. ### Cache Preservation and State Tracking The foundation of incremental builds rests on two directories: `.cache` and `public`. During the build process, Gatsby stores the **graph structure**, **schema metadata**, and **node data** in `.cache`, while generated HTML assets reside in `public`. According to the source code in [`packages/gatsby/src/schema/index.js`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/schema/index.js), the build system initializes "build metadata for type inference and start[s] updating it incrementally." Underlying this persistence is the LMDB datastore implementation found in [`packages/gatsby/src/datastore/lmdb/lmdb-datastore.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/datastore/lmdb/lmdb-datastore.ts), which provides the transactional storage layer for node data between builds. ### Node Change Detection During the data-sourcing phase, source plugins (such as `gatsby-source-wordpress` or `gatsby-source-contentful`) mark nodes as added, updated, or deleted. The runtime maintains this state in the Redux store, specifically within [`packages/gatsby/src/redux/reducers/inference-metadata.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/redux/reducers/inference-metadata.ts), which contains the `incrementalBuild` step flag and tracks which pages depend on modified nodes. When a node changes, the system flags all connected pages as "dirty" using internal utilities like `findConnectedNodes`, which invokes `touchNode` to propagate change notifications through the dependency graph. ### Selective Page Regeneration When the HTML generation step executes, Gatsby checks the internal `dirtyPages` set. Only pages appearing in this set are passed to the SSR pipeline for regeneration; unchanged pages are copied directly from the previous `public` output. This selective approach bypasses the costly process of rebuilding unchanged static assets. The `step: StepsEnum.incrementalBuild` logic in the inference metadata reducer controls this gating mechanism, ensuring that builds scale with the magnitude of content changes rather than total site size. ## Requirements for CI/CD Optimization To achieve sub-minute builds in continuous integration environments, you must configure artifact persistence explicitly. Without proper caching, each CI run performs a full rebuild, negating the benefits of incremental compilation. ### Persisting Cache Between Runs Your CI pipeline must preserve both `.cache` and `public` as build artifacts. The general pattern involves: 1. **Restoring** cached directories at the start of each job 2. **Executing** `gatsby build` (or `npm run build`) 3. **Saving** the updated directories as artifacts for subsequent runs Most platforms (GitHub Actions, CircleCI, Netlify) provide caching mechanisms that support this workflow. The cache key should incorporate the runner OS and commit hash, with fallback restore keys to maximize hit rates. ### Configuring Build Commands Avoid executing `gatsby clean` in CI environments unless specifically requiring a full rebuild. The clean command deletes `.cache` and `public`, forcing the next build to start from scratch. Instead, rely on Gatsby’s internal dirty-checking logic to determine which pages require regeneration. In Gatsby v3 and later, **incremental builds are enabled by default** with no environment flags required. For legacy Gatsby v2 deployments, you must set `GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=1`. ### Unsupported Patterns to Avoid Certain coding patterns disable incremental builds automatically. Direct usage of Node.js `fs` modules within [`gatsby-ssr.js`](https://github.com/gatsbyjs/gatsby/blob/main/gatsby-ssr.js) or other SSR files triggers the unsafe filesystem detection, forcing a full rebuild. To maintain incremental capabilities, use ES-module imports or Gatsby’s data layer APIs exclusively, avoiding side effects that bypass the tracked node system. ## Implementation Examples ### Basic Gatsby Configuration No special configuration is required for standard incremental builds in Gatsby v3+. A typical [`gatsby-config.js`](https://github.com/gatsbyjs/gatsby/blob/main/gatsby-config.js) with source plugins automatically participates in the incremental pipeline: ```javascript module.exports = { plugins: [ { resolve: `gatsby-source-wordpress`, options: { url: `https://example.com/graphql`, }, }, `gatsby-plugin-image`, ], } ``` ### GitHub Actions Workflow The following configuration persists cache directories between runs using `actions/cache`: ```yaml name: Gatsby CI on: push: branches: [main] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Cache Gatsby directories uses: actions/cache@v3 with: path: | .cache public key: ${{ runner.os }}-gatsby-${{ github.sha }} restore-keys: | ${{ runner.os }}-gatsby- - name: Install dependencies run: npm ci - name: Build run: npm run build ``` ### Disabling Incremental Builds (Debug Mode) In rare cases requiring full rebuilds for debugging, you can explicitly disable the feature: ```javascript // gatsby-node.js exports.onCreateWebpackConfig = ({ actions }) => { actions.setWebpackConfig({ plugins: [ new webpack.DefinePlugin({ GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES: JSON.stringify(false) }) ], }) } ``` ## Summary - **Gatsby incremental builds** rely on persistent `.cache` and `public` directories to track state between runs. - The system uses **LMDB datastores** and **Redux reducers** (specifically [`inference-metadata.ts`](https://github.com/gatsbyjs/gatsby/blob/main/inference-metadata.ts)) to track dirty pages and node dependencies. - **CI/CD optimization** requires configuring artifact caching to preserve these directories; avoid `gatsby clean` in automated pipelines. - **Gatsby v3+** enables incremental builds by default, while v2 requires experimental flags. - **Unsafe filesystem access** in SSR code disables incremental builds automatically. ## Frequently Asked Questions ### Do I need Gatsby Cloud to use incremental builds? No. Incremental builds are available in the open-source Gatsby core since version 3.0. While Gatsby Cloud provides additional optimizations like Deferred Static Generation (DSG) and Parallel Rendering, the fundamental incremental build capability works in any environment where the `.cache` and `public` directories persist between builds. ### What happens if I run `gatsby clean` in CI? Running `gatsby clean` deletes the `.cache` and `public` directories, forcing the subsequent build to perform a full regeneration of all pages. This eliminates the performance benefits of incremental builds. Only use this command when explicitly requiring a fresh build due to schema changes or corrupted cache states. ### Which Gatsby version supports incremental builds? Gatsby v3.0 and later support incremental builds by default without configuration. Gatsby v2 supported incremental builds experimentally using the `GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES` environment variable, though this is deprecated in favor of the automatic implementation in v3+. ### How do source plugins support incremental builds? Source plugins participate in the incremental pipeline by calling `touchNode` for modified content, which flags dependent pages as dirty. The internal `findConnectedNodes` utility (implemented in plugins like `gatsby-source-wordpress`) identifies which pages query specific nodes, ensuring only affected pages enter the `dirtyPages` set for regeneration.