Implementing Custom Cache Systems with the CacheInterface in Transformers.js

The Transformers.js library abstracts model-file caching behind a minimal CacheInterface that mirrors the Web Cache API, allowing you to override default storage behavior by setting env.useCustomCache = true and assigning a custom object to env.customCache that implements match(), put(), and optionally delete() methods.

When loading models from the Hugging Face Hub, Transformers.js automatically manages file caching to avoid redundant downloads. While the library defaults to the browser's Cache API or a filesystem-based FileCache, production deployments often require specialized persistence layers such as Redis, IndexedDB, or encrypted stores. By implementing the CacheInterface defined in src/utils/cache.js, you can intercept all cache operations and route them through your own backend.

Understanding the CacheInterface Architecture

The caching subsystem selects a backend according to a strict priority order defined in src/utils/cache.js. When the library initializes via getCache(), it checks:

1. Custom cache — Used when env.useCustomCache is enabled and env.customCache contains a valid implementation.
2. Browser Cache API — Invoked via caches.open when running in a browser environment.
3. File-system cache — Falls back to FileCache (implemented in src/utils/hub/files.js) for Node.js, Deno, or Bun when a writable filesystem is available.

The interface contract requires only two mandatory methods and one optional:

• async match(request): Accepts a string URL and returns a Response (or undefined if not cached).
• async put(request, response, progress_callback?): Stores the response, optionally reporting download progress.
• async delete?(request): (Optional) Removes an entry, enabling full cache-clearing support via src/utils/model_registry/clear_cache.js.

Enabling Custom Cache Mode

Before the library loads any model files, you must flag the environment to use a custom backend. This must be set before importing models or calling pipeline() to ensure getCache() selects your implementation.

import { env } from '@huggingface/transformers';

// Enable custom cache lookup
env.useCustomCache = true;

Implementing a Custom Cache

Your custom cache must be an object or class instance that conforms to the CacheInterface. Below is a complete in-memory implementation using JavaScript Maps, suitable for testing or ephemeral caching.

Minimal In-Memory Cache Example

// memoryCache.js
class MemoryCache {
    constructor() {
        this.store = new Map(); // key → {body, headers, status}
    }

    async match(request) {
        const entry = this.store.get(request);
        if (!entry) return undefined;

        // Reconstruct a Web API compatible Response
        return new Response(entry.body, {
            status: entry.status,
            headers: entry.headers,
        });
    }

    async put(request, response, progress_callback) {
        // Capture response body as ArrayBuffer
        const body = await response.arrayBuffer();
        
        // Preserve metadata
        const headers = {};
        response.headers.forEach((v, k) => (headers[k] = v));

        this.store.set(request, { body, headers, status: response.status });

        // Report completion
        progress_callback?.({ 
            progress: 100, 
            loaded: body.byteLength, 
            total: body.byteLength 
        });
    }

    async delete(request) {
        return this.store.delete(request);
    }
}

export const customCache = new MemoryCache();

Wiring the Custom Cache

After defining your implementation, attach it to the environment object. All subsequent calls to getCache() throughout src/utils/hub.js and the model registry will now delegate to your object.

import { env } from '@huggingface/transformers';
import { customCache } from './memoryCache.js';

env.useCustomCache = true;
env.customCache = customCache; // Must follow the enable flag

// Subsequent model loads use your cache
import { pipeline } from '@huggingface/transformers';
const classifier = await pipeline('sentiment-analysis');

Advanced Custom Cache Patterns

Distributed Caching with Redis

For server-side deployments, you can implement the interface using Redis to share cached model files across multiple Node.js instances. Store binary data as base64 to ensure safe JSON serialization.

// redisCache.js
import { createClient } from 'redis';

class RedisCache {
    constructor() {
        this.client = createClient();
        this.client.connect();
    }

    async match(request) {
        const data = await this.client.get(request);
        if (!data) return undefined;

        const { body, status, headers } = JSON.parse(data);
        const uint8 = Uint8Array.from(atob(body), c => c.charCodeAt(0));
        
        return new Response(uint8.buffer, { status, headers });
    }

    async put(request, response, progress_callback) {
        const body = await response.arrayBuffer();
        const headers = {};
        response.headers.forEach((v, k) => (headers[k] = v));

        const payload = JSON.stringify({
            body: Buffer.from(body).toString('base64'),
            status: response.status,
            headers,
        });

        await this.client.set(request, payload);
        progress_callback?.({ 
            progress: 100, 
            loaded: body.byteLength, 
            total: body.byteLength 
        });
    }

    async delete(request) {
        const result = await this.client.del(request);
        return result > 0;
    }
}

export const redisCache = new RedisCache();

Persistent Browser Storage

For browser applications requiring survival across page reloads, implement match and put using IndexedDB or localStorage. Store the raw ArrayBuffer and reconstruct the Response object in match, mirroring the in-memory example but with asynchronous IndexedDB transactions.

How the Library Consumes Your Cache

Your custom implementation integrates seamlessly with the library's internal workflows:

• Model loading: Files in src/utils/model_registry/get_file_metadata.js call getCache() and invoke match() to check for local copies before downloading.
• Hub utilities: src/utils/hub.js builds standardized cache keys and passes your cache to checkCachedResource, delegating storage to your put() method during downloads.
• Cache clearing: When clear_cache.js is invoked, it iterates through cached files and calls delete() on your object if implemented, allowing full eviction support.

Summary

• Enable custom caching by setting env.useCustomCache = true before any model imports.
• Implement three methods: match(request), put(request, response, progress_callback?), and optionally delete(request) to conform to the CacheInterface in src/utils/cache.js.
• Attach your instance to env.customCache to override the default browser or filesystem backends.
• Support cache clearing by implementing the optional delete method, which src/utils/model_registry/clear_cache.js uses during cache invalidation.
• Handle binary data by storing ArrayBuffer contents and reconstructing Response objects in match().

Frequently Asked Questions

What happens if I don't implement the delete method?

The delete method is optional according to the CacheInterface definition in src/utils/cache.js. If omitted, cache-clearing operations initiated via the library's utilities will skip your custom store, leaving entries intact. Your cache will still function normally for reading and writing model files.

Can I use a custom cache in Node.js environments?

Yes. When running in Node.js, the library normally defaults to FileCache defined in src/utils/hub/files.js. By enabling env.useCustomCache and providing your own implementation, you can redirect caching to Redis, an S3 bucket, or any other storage backend, completely bypassing the filesystem.

How do I report download progress with a custom cache?

The put method receives a third argument, progress_callback, which you should invoke with an object containing progress, loaded, and total values. This feeds progress bars in UI components. If your backend supports streaming (e.g., S3 multipart uploads), you can call this callback multiple times to report incremental progress rather than a single 100% completion.

Does the custom cache replace the browser Cache API entirely?

When env.useCustomCache is true and env.customCache is set, the library uses your implementation exclusively for all cache operations. It will not fall back to the browser's caches.open or the filesystem FileCache for that session. Your object becomes the single source of truth for getCache() calls throughout src/utils/hub.js and the model registry.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}