How CoCache Prevents Cache Stampede with MissingGuard: Per-Key Locking and Sentinel Values

CoCache prevents cache stampede by implementing fine-grained per-key locking combined with MissingGuard placeholder values that temporarily cache negative results, ensuring only one thread queries the database for missing keys while subsequent requests hit the sentinel value.

In distributed caching architectures, a cache stampede (or cache breakdown) occurs when multiple concurrent requests simultaneously miss the cache and bombard the underlying data source. The CoCache library (ahoo-wang/cocache) eliminates this risk through a coherent cache implementation that uses synchronized key-scoped locks and special sentinel objects to guard against both cache penetration and thundering herds.

The Cache Stampede Problem

A cache stampede happens when a popular cache entry expires or a non-existent key receives high concurrent traffic. Without protection, every request simultaneously attempts to load data from the primary source, overwhelming databases and services. Traditional solutions like global locks are too coarse-grained, while simply allowing misses wastes resources and degrades performance.

CoCache's Two-Layer Defense Mechanism

CoCache employs a dual-strategy approach implemented in DefaultCoherentCache.kt to solve this problem efficiently.

Fine-Grained Per-Key Locking

Before falling back to the underlying data source, CoCache obtains a lock unique to the specific cache key using keyLocks.computeIfAbsent(cacheKey). This creates a synchronized block where only one thread can execute the expensive load operation for that specific key. All other concurrent callers block on the lock, re-check the cache after the lock releases, and retrieve the freshly cached value without hitting the database.

MissingGuard Sentinel Values

When the data source returns no value (a cache miss), CoCache stores a special MissingGuard object (DefaultCacheValue.missingGuard) instead of leaving the cache empty. This sentinel has a short TTL and signals that the key is known to be absent. Subsequent requests within the guard's TTL instantly recognize the sentinel via the isMissingGuard property, returning null immediately without querying the source.

Implementation Flow in DefaultCoherentCache

The getCache method in DefaultCoherentCache orchestrates this protection through a specific execution flow:

1. L2 Lookup: Check client-side cache via getL2Cache (lines 52-58)
2. L1 Lookup: Check distributed cache and populate L2 on hit (lines 65-73)
3. Lock Acquisition: keyLocks.computeIfAbsent(cacheKey) creates the lock object used for synchronization (lines 78-86)
4. Double-Check Pattern: Inside the synchronized block, re-run getL2Cache to avoid race conditions (lines 100-106)
5. Source Loading: cacheSource.loadCacheValue(key) fetches data from the primary source (lines 112-116)
6. Guard Insertion: If the source returns null, store DefaultCacheValue.missingGuard(ttl, ttlAmplitude) via setCache (line 129)
7. Value Caching: If data exists, store the real value in both L2 and L1 layers (lines 113-115)

This sequence ensures that even for keys that do not exist in the database, the system caches the negative result temporarily, preventing cache penetration attacks and stampede scenarios.

Practical Code Example

The following Kotlin example demonstrates how MissingGuard protection works in practice:

import me.ahoo.cache.consistency.DefaultCoherentCache
import me.ahoo.cache.DefaultCacheValue

// Initialize the coherent cache with configuration
val coherentCache = DefaultCoherentCache<String, User>(config, cacheEvictedEventBus)

// Normal retrieval - checks L2, then L1, then source if needed
val user: CacheValue<User>? = coherentCache.getCache("user:123")

// Simulating a cache miss for a non-existent key
// First request acquires lock, queries source (returns null), stores guard
val missing = coherentCache.getCache("user:999")   // Returns null, guard cached

// Subsequent calls within guard TTL hit the sentinel instantly
// No database query occurs
val stillMissing = coherentCache.getCache("user:999")   // Returns null, no DB hit

You can explicitly check for guard values to implement custom fallback logic:

val cached = coherentCache.getCache("user:999")
if (cached?.isMissingGuard == true) {
    // Key is confirmed absent; implement alternative logic
    // without expensive source lookup
}

Key Source Files

Understanding the complete implementation requires examining these specific files in the ahoo-wang/cocache repository:

• cocache-core/src/main/kotlin/me/ahoo/cache/consistency/DefaultCoherentCache.kt: Contains the getCache method implementing per-key locking and guard insertion logic
• cocache-core/src/main/kotlin/me/ahoo/cache/MissingGuard.kt: Defines the sentinel interface and isMissingGuard extension property
• cocache-core/src/main/kotlin/me/ahoo/cache/DefaultCacheValue.kt: Supplies the missingGuard factory method for creating placeholder values
• cocache-api/src/main/kotlin/me/ahoo/cache/api/source/CacheSource.kt: Abstracts the underlying data source interface that may return null values

Summary

• Per-key locking in DefaultCoherentCache ensures only one thread loads data from the source for any specific key, eliminating concurrent database queries during cache misses.
• MissingGuard sentinel values cache negative results with short TTLs, preventing repeated lookups for non-existent keys and stopping cache penetration attacks.
• The double-check pattern after lock acquisition prevents race conditions where multiple threads might otherwise load the same data redundantly.
• L1/L2 coherence ensures that once a value (or guard) is cached, it propagates correctly through both local and distributed layers according to the source code implementation.

Frequently Asked Questions

How does CoCache handle concurrent requests for the same missing key?

When multiple threads simultaneously request a key that doesn't exist, the first thread acquires the per-key lock via keyLocks.computeIfAbsent(cacheKey) while others wait. The locked thread queries the source, stores a MissingGuard when null is returned, and releases the lock. Subsequent threads then retrieve the guard value immediately without database access, effectively serializing the expensive operation while maintaining high throughput for cache hits.

What is the difference between MissingGuard and a regular null value?

A regular null value indicates the cache entry doesn't exist, which would trigger a source lookup on the next request. The DefaultCacheValue.missingGuard is a concrete sentinel object with metadata including a TTL. The isMissingGuard property allows the cache logic to distinguish between "value not yet cached" and "value confirmed absent," enabling the system to skip database queries for known missing keys during the guard's lifetime.

Can the MissingGuard TTL be configured separately from regular cache entries?

Yes, the MissingGuard accepts both a base TTL and an amplitude parameter (DefaultCacheValue.missingGuard(ttl, ttlAmplitude)) that creates jitter to prevent simultaneous expiration of multiple guards. This configuration helps avoid stampede scenarios when many missing keys expire simultaneously, distributing the potential load over time.

Does per-key locking impact performance for high-throughput applications?

The keyLocks mechanism uses a concurrent hash map with weak references to ensure locks are scoped precisely to individual keys without global contention. Since the lock is only held during source loading (a relatively slow operation) and bypassed entirely for cache hits, the overhead is negligible compared to the cost of database queries. The implementation specifically optimizes for the case where cached values exist, ensuring hot paths remain lock-free.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}