# How the Gatsby Cache System Works: Build Performance Optimization Guide

> Understand how Gatsby's cache system works and optimize build performance. Leverage LMDB and tune memory for faster builds.

- Repository: [Gatsby/gatsby](https://github.com/gatsbyjs/gatsby)
- Tags: internals
- Published: 2026-03-06

---

**Gatsby's cache system uses a layered multi-cache architecture combining an in-memory LRU store with a persistent disk cache to minimize redundant work during builds, and you can significantly improve performance by switching to the LMDB-backed implementation and tuning memory limits.**

The Gatsby cache system is a critical component of the `gatsbyjs/gatsby` repository that persists intermediate build data between runs to avoid redundant processing. By storing query results, page data, and transformed assets in a layered cache, Gatsby dramatically reduces build times for subsequent runs. Understanding how this system works under the hood allows you to optimize your own plugins and build pipelines for maximum performance.

## Core Architecture of the Gatsby Cache System

The cache implementation lives primarily in [`packages/gatsby/src/utils/cache.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache.ts), where the `GatsbyCache` class orchestrates a multi-layer storage strategy.

### Multi-Layer Caching Strategy

Each `GatsbyCache` instance composes two distinct stores using the `cache-manager` library:

1. **Memory cache** – An LRU-style store limited to `MAX_CACHE_SIZE` (250 entries by default) with a practically infinite TTL. This provides microsecond-level access for hot data.
2. **Disk cache** – A persistent store written to `.cache/caches/<name>` on the filesystem, ensuring data survives process restarts.

The `init()` method in [`cache.ts`](https://github.com/gatsbyjs/gatsby/blob/main/cache.ts) wires these together into a `multiCaching` instance:

```typescript
// packages/gatsby/src/utils/cache.ts
export default class GatsbyCache {
  // ...
  init(): GatsbyCache {
    fs.ensureDirSync(this.directory)

    const configs: Array<StoreConfig> = [
      { store: `memory`, max: MAX_CACHE_SIZE, ttl: TTL },          // memory layer
      { store: this.store, ttl: TTL, options: { path: this.directory } }, // disk layer
    ]

    const caches = configs.map(cache => manager.caching(cache))
    this.cache = manager.multiCaching(caches)                      // combines both
    return this
  }
}

```

### Disk Store Implementation

The filesystem persistence layer is implemented in [`packages/gatsby/src/cache/cache-fs.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/cache/cache-fs.ts). The `DiskStore` class handles low-level read/write logic with several optimizations:

- **Key hashing** – Cache keys are hashed using `md5` to generate deterministic filenames (`diskstore-<hash>`), preventing filesystem issues with special characters.
- **Sub-directory sharding** – When the `subdirs` option is enabled, files are distributed across subdirectories (using the first 3 characters of the hash) to avoid directory size limits.
- **File locking** – A `globalGatsbyCacheLock` ensures safe concurrent reads/writes during a single build process.

```typescript
// packages/gatsby/src/cache/cache-fs.ts (excerpt)
DiskStore.prototype._getFilePathByKey = function (key): string {
  const hash = crypto.createHash(`md5`).update(key + ``).digest(`hex`)
  return this.options.subdirs
    ? path.join(this.options.path, `diskstore-` + hash.slice(0, 3), hash.slice(3))
    : path.join(this.options.path, `diskstore-` + hash)
}

```

The actual JSON serialization (including optional gzip compression for large buffers) is handled by [`packages/gatsby/src/cache/json-file-store.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/cache/json-file-store.ts), which externalizes large `Buffer` objects to separate `.bin` files when necessary.

### LMDB Cache Alternative

Since Gatsby v4, the framework offers an **LMDB-backed cache** (`GatsbyCacheLmdb`) implemented in [`packages/gatsby/src/utils/cache-lmdb.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache-lmdb.ts). LMDB (Lightning Memory-Mapped Database) stores data in a memory-mapped B-tree, providing significantly lower latency than the JSON file-system store, especially for thousands of small entries like query hash lookups.

The LMDB implementation uses the same API (`get`, `set`, `del`) as the standard cache, making substitution seamless:

```typescript
// packages/gatsby/src/utils/cache-lmdb.ts (excerpt)
private static getStore(): RootDatabase {
  // ...
  rootDb = open({ name: `root`, path: dbPath, compression: true, maxDbs: 200 })
  // ...
}

```

## Where Gatsby Uses Caching Internally

Gatsby leverages its cache system across several critical build paths to avoid redundant computation:

- **Query Runner** – `resultHashCache` in [`packages/gatsby/src/query/query-runner.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/query/query-runner.ts) stores hashed query results to skip re-running identical queries.
- **Page Data Generation** – `lmdbPageQueryResultsCache` in [`packages/gatsby/src/utils/page-data.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/page-data.ts) caches serialized page data between builds.
- **Date Formatting** – `formatDateCache` in [`packages/gatsby/src/schema/types/date.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/schema/types/date.ts) memoizes expensive date formatting operations.

These internal caches demonstrate how persisting intermediate state dramatically reduces build times for large sites.

## Strategies to Improve Gatsby Build Performance

Optimizing the Gatsby cache system involves tuning both the memory and persistence layers. Here are the most effective strategies:

| Strategy | What It Does | How to Apply |
|----------|--------------|--------------|
| **Enable LMDB cache** | Replaces slow JSON-file disk cache with a memory-mapped database. | Set environment variable `GATSBY_CACHE_LMDB=true` or instantiate `new GatsbyCacheLmdb({ name: 'cacheName' }).init()` in custom plugins. |
| **Increase memory cache size** | Retains more hot data in RAM, avoiding disk I/O. | Adjust `MAX_CACHE_SIZE` in [`packages/gatsby/src/utils/cache.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache.ts) or set `process.env.GATSBY_MAX_CACHE_SIZE` to a higher value than the default 250. |
| **Enable compression** | Reduces disk I/O for large JSON payloads. | Pass `{ zip: true }` to the `DiskStore` options when constructing `GatsbyCache`, or ensure `compression: true` is set in LMDB initialization. |
| **Use sub-directories** | Prevents filesystem slowdown when a cache grows beyond thousands of files. | Set `subdirs: true` in `DiskStore` options to shard files across subdirectories using hash prefixes. |
| **Clear stale caches** | Removes orphaned files that waste space and cause lock contention. | Run `await cache.reset()` on `GatsbyCache` or `GatsbyCacheLmdb` instances, or delete `.cache/caches` between major version upgrades. |
| **Leverage incremental builds** | Re-uses previously persisted cache across builds instead of rebuilding from scratch. | Ensure the `.cache` folder persists between CI runs by mounting it as a cache artifact; avoid clearing it between `gatsby develop` sessions. |
| **Avoid duplicate cache instances** | Prevents redundant I/O when multiple plugins use the same cache name. | Share a single cache instance via `require("gatsby/src/utils/get-cache")` or export a common cache from a utility module. |

### Practical Code Examples

To implement these optimizations in your own plugins or site configuration, use the following patterns:

**Using LMDB for Custom Plugin Caching**

```typescript
// src/custom-plugin.ts – using the LMDB cache for expensive image processing
import GatsbyCacheLmdb from "gatsby/src/utils/cache-lmdb"

const imageCache = new GatsbyCacheLmdb({ name: `image-processing` }).init()

export async function processImage(hash: string, src: string) {
  // 1️⃣ Try to read from cache
  const cached = await imageCache.get<string>(hash)
  if (cached) return cached

  // 2️⃣ Perform heavy work (e.g., sharp transformations)
  const result = await heavyTransform(src)

  // 3️⃣ Store result for next run
  await imageCache.set(hash, result)

  return result
}

```

**Resetting Caches After Major Updates**

```typescript
// src/utility.ts – resetting caches after a major version bump
import { resetCache } from "gatsby/src/utils/cache-lmdb"

export async function cleanAllCaches() {
  // Clears LMDB caches
  await resetCache()
  // Clears FS caches (if you also use GatsbyCache)
  // await new GatsbyCache().init().reset()
}

```

## Key Source Files

Understanding the Gatsby cache system requires familiarity with these specific modules:

| File | Role | Link |
|------|------|------|
| [`packages/gatsby/src/utils/cache.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache.ts) | High-level multi-cache factory (memory + disk) | [cache.ts](https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/utils/cache.ts) |
| [`packages/gatsby/src/cache/cache-fs.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/cache/cache-fs.ts) | Low-level file-system store, locking, key hashing | [cache-fs.ts](https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/cache/cache-fs.ts) |
| [`packages/gatsby/src/cache/json-file-store.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/cache/json-file-store.ts) | JSON (and optional gzip) serialization, external buffer handling | [json-file-store.ts](https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/cache/json-file-store.ts) |
| [`packages/gatsby/src/utils/cache-lmdb.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache-lmdb.ts) | LMDB-backed cache implementation (fast, memory-mapped) | [cache-lmdb.ts](https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/utils/cache-lmdb.ts) |
| [`packages/gatsby/src/query/query-runner.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/query/query-runner.ts) | Example consumer: caches query hash results | [query-runner.ts](https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/query/query-runner.ts) |
| [`packages/gatsby/src/utils/page-data.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/page-data.ts) | Example consumer: caches page query results | [page-data.ts](https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/utils/page-data.ts) |

## Summary

- The Gatsby cache system implements a **multi-layer architecture** in [`packages/gatsby/src/utils/cache.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache.ts), combining a fast in-memory LRU cache (250 entries by default) with a persistent disk cache stored in `.cache/caches/<name>`.
- **Disk persistence** uses hashed filenames and optional sub-directory sharding via [`packages/gatsby/src/cache/cache-fs.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/cache/cache-fs.ts), with file locking to prevent race conditions during builds.
- **LMDB integration** (available since Gatsby v4) in [`packages/gatsby/src/utils/cache-lmdb.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache-lmdb.ts) offers a high-performance alternative to JSON file storage, using memory-mapped B-trees for lower latency.
- To optimize build performance, **enable LMDB caching**, **increase memory cache limits**, **enable compression**, and **persist the `.cache` directory** between CI runs to leverage incremental builds.

## Frequently Asked Questions

### How do I clear the Gatsby cache when builds behave unexpectedly?

Run `gatsby clean` from your terminal to delete the `.cache` and `public` directories entirely. For programmatic control in custom plugins, invoke `await cache.reset()` on your `GatsbyCache` or `GatsbyCacheLmdb` instance to clear specific cache namespaces without wiping the entire build directory.

### Should I use the LMDB cache or the standard file-system cache for my plugin?

Choose **LMDB** (`GatsbyCacheLmdb`) when your plugin performs frequent small reads/writes (such as query result caching or date formatting), as the memory-mapped B-tree structure in [`packages/gatsby/src/utils/cache-lmdb.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache-lmdb.ts) provides lower latency than JSON file I/O. Use the standard `GatsbyCache` from [`packages/gatsby/src/utils/cache.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache.ts) for larger, infrequently accessed payloads where compression and simple file storage suffice.

### How does Gatsby handle cache persistence between CI builds?

Gatsby writes all cache data to the `.cache` directory at the project root, including both the multi-layer disk stores and LMDB databases. To enable **incremental builds** in CI environments, configure your build platform (Netlify, Vercel, GitHub Actions, etc.) to preserve the `.cache` folder as a build artifact between runs, ensuring that `gatsby build` can reuse previously computed query results and page data.

### What is the default memory cache limit and how do I increase it?

The default `MAX_CACHE_SIZE` is **250 entries** as defined in [`packages/gatsby/src/utils/cache.ts`](https://github.com/gatsbyjs/gatsby/blob/main/packages/gatsby/src/utils/cache.ts). To increase this limit and keep more hot data in RAM, set the environment variable `GATSBY_MAX_CACHE_SIZE` to a higher value (e.g., `500` or `1000`) before running your build, or modify the `MAX_CACHE_SIZE` constant directly in a forked version of the cache utility if you require persistent changes across projects.