How CoCache Ensures Cache Consistency Across Multiple Instances

CoCache ensures cache consistency across multiple instances by combining a distributed L1 cache (e.g., Redis), a local in-process L2 cache, and a publish/subscribe eviction event bus that broadcasts invalidation events to all instances whenever data changes.

The ahoo-wang/cocache library implements an eventual consistency model for JVM-based applications running in clustered environments. By tiering cache access and propagating invalidation events through a lightweight event bus, CoCache allows each instance to maintain fast local reads while guaranteeing that stale data is evicted cluster-wide within milliseconds.

The Three-Layer Consistency Architecture

CoCache achieves cache consistency across multiple instances through three coordinated layers:

Distributed Cache (L1)

The L1 layer serves as the authoritative shared backing store, typically backed by Redis. This layer holds the single source of truth for all cache entries across the cluster. When an instance writes data, it updates L1 first, ensuring that any other instance can retrieve the fresh value directly from the distributed store if their local cache misses.

Client-Side Cache (L2)

The L2 layer is a high-speed, in-process map (such as Caffeine or Guava Cache) local to each JVM instance. This layer provides microsecond-level read latency for hot data. However, because each instance maintains its own isolated L2, CoCache must actively invalidate these entries when remote instances perform updates.

The Eviction Event Bus

The cache-eviction event bus is the coordination mechanism that ties the layers together. Implemented in GuavaCacheEvictedEventBus.kt, this component uses Guava’s EventBus to publish and subscribe to CacheEvictedEvent objects. When one instance mutates a key, it publishes an event; all other instances consume this event and evict the corresponding entry from their L2 caches.

Write Operations and Event Propagation

All write operations in DefaultCoherentCache.kt follow a strict protocol to maintain consistency. Located at cocache-core/src/main/kotlin/me/ahoo/cache/consistency/DefaultCoherentCache.kt, this class implements the core logic for the setCache, evict, and event handling methods.

Setting and Evicting Keys

When an instance calls setCache() (lines 42–49), the implementation performs three atomic steps:

1. Writes the value to the local L2 cache for immediate visibility
2. Writes the value to the distributed L1 cache for persistence
3. Publishes a CacheEvictedEvent to the bus to invalidate peer L2 entries

The evict() method (lines 51–56) follows a similar pattern, removing the entry from both L1 and L2 before publishing the eviction event:

// DefaultCoherentCache.kt – evict logic (simplified)
override fun evict(key: K) {
    clientSideCache.evict(key)    // Clear local L2
    distributedCache.evict(key)   // Clear shared L1
    publishEvictEvent(key)        // Notify other instances
}

Handling Remote Eviction Events

Each cache instance registers an @Subscribe annotated handler named onEvicted() (lines 58–73). When the event bus delivers a CacheEvictedEvent, this method checks if the event originated from a different clientId. If so, it evicts only the local L2 entry, leaving the L1 data intact:

// DefaultCoherentCache.kt – event handler
@Subscribe
override fun onEvicted(cacheEvictedEvent: CacheEvictedEvent) {
    if (cacheEvictedEvent.clientId == clientId) {
        return // Ignore self-generated events
    }
    clientSideCache.evict(cacheEvictedEvent.key)
}

Read Operations with Cache Stampede Protection

The getCache() method (lines 88–135) in DefaultCoherentCache.kt implements a cache-aside pattern with cache stampede protection. When a key is not found in L2 or L1, the implementation acquires a fine-grained lock from keyLocks to ensure that only one thread per instance loads the value from the underlying data source.

After loading, the method populates both L1 and L2 and publishes an eviction event. This guarantees that other instances will invalidate their stale L2 entries before the next read cycle:

// DefaultCoherentCache.kt – getCache with stampede protection
override fun getCache(key: K): CacheValue<V>? {
    // 1. Check L2
    clientSideCache.get(key)?.let { return it }
    
    // 2. Check L1
    distributedCache.getCache(key)?.let { 
        clientSideCache.set(key, it) // Backfill L2
        return it 
    }
    
    // 3. Acquire lock and load from source
    return keyLocks.lock(key) {
        // Load from data source...
        // Write to L1 and L2
        // Publish eviction event for consistency
    }
}

Configuring the Event Bus

The concrete implementation GuavaCacheEvictedEventBus wraps Guava’s EventBus to provide the publish(), register(), and unregister() methods. This class is defined in cocache-core/src/main/kotlin/me/ahoo/cache/consistency/GuavaCacheEvictedEventBus.kt:

// GuavaCacheEvictedEventBus.kt
class GuavaCacheEvictedEventBus : CacheEvictedEventBus {
    private val eventBus = EventBus()
    
    override fun publish(event: CacheEvictedEvent) {
        eventBus.post(event)
    }
    
    override fun register(subscriber: CacheEvictedSubscriber) {
        eventBus.register(subscriber)
    }
}

In production deployments, you can replace this with a Redis Pub/Sub or Kafka-backed implementation by implementing the CacheEvictedEventBus interface, though the Guava version suffices for single-node multi-instance testing.

Multi-Instance Setup Example

The following Kotlin example demonstrates how to configure two distinct instances that share the same distributed cache and event bus, ensuring cache consistency across multiple instances:

import me.ahoo.cache.consistency.*
import me.ahoo.cache.api.client.ClientSideCache
import me.ahoo.cache.distributed.DistributedCache
import me.ahoo.cache.converter.KeyConverter

// Create shared infrastructure
val distributedCache: DistributedCache<String> = RedisCache()
val clientSideCache: ClientSideCache<String> = CaffeineCache()
val eventBus = GuavaCacheEvictedEventBus()
val factory = DefaultCoherentCacheFactory(eventBus)

val keyConverter = object : KeyConverter<Int> {
    override fun toStringKey(key: Int) = key.toString()
    override fun fromStringKey(stringKey: String) = stringKey.toInt()
}

// Instance A (JVM 1)
val cacheA = factory.create(
    CoherentCacheConfiguration(
        cacheName = "userCache",
        clientId = "instanceA",
        keyConverter = keyConverter,
        distributedCache = distributedCache,
        clientSideCache = clientSideCache
    )
)

// Instance B (JVM 2)
val cacheB = factory.create(
    CoherentCacheConfiguration(
        cacheName = "userCache",
        clientId = "instanceB",
        keyConverter = keyConverter,
        distributedCache = distributedCache,
        clientSideCache = clientSideCache
    )
)

// Write from Instance A
cacheA.setCache(42, CacheValue("Alice"))

// Read from Instance B
// L2 miss → L1 hit → updates B's L2 automatically
val value = cacheB.getCache(42) // Returns "Alice"

The test specification MultipleInstanceSyncSpec.kt in the cocache-test module validates this behavior, simulating concurrent writes and reads across two cache instances to verify eventual consistency.

Summary

• CoCache uses a tiered architecture combining distributed L1 (authoritative) and local L2 (fast) caches to balance performance with consistency.
• Write operations in DefaultCoherentCache.kt update both layers and publish CacheEvictedEvent notifications via the event bus.
• Remote instances receive events through the onEvicted() handler and invalidate their local L2 entries, ensuring subsequent reads fetch fresh data from L1.
• Cache stampede protection is implemented via keyLocks in the getCache() method to prevent thundering herd scenarios during cache misses.
• Eventual consistency is achieved without distributed transactions, making the system suitable for high-throughput, low-latency applications.

Frequently Asked Questions

How does CoCache handle network partitions between instances?

CoCache guarantees eventual consistency rather than strong consistency. If the event bus becomes partitioned, instances will continue to serve stale data from their L2 caches until connectivity is restored and eviction events are delivered. The L1 distributed cache (Redis) remains authoritative, so reads will eventually converge to the correct values once the network heals or the L2 entries expire naturally.

What happens if two instances update the same key simultaneously?

CoCache does not implement distributed locking for write conflicts. The last write to the L1 distributed cache wins based on the underlying storage’s consistency model (e.g., Redis single-threaded execution). Both instances will publish eviction events, causing all L2 caches to clear and reload the final value from L1 on the next access.

Can I use a different message broker instead of Guava EventBus?

Yes. The CacheEvictedEventBus interface in cocache-core abstracts the messaging layer. You can implement this interface using Redis Pub/Sub, RabbitMQ, or Kafka by overriding the publish() and register() methods. The DefaultCoherentCache remains unchanged as long as the custom implementation delivers CacheEvictedEvent objects to all subscribed instances.

Does CoCache support transactional updates across the cache and database?

CoCache does not provide built-in transaction coordination between the cache layers and external databases. When using getCache() with cache-aside loading, the data source write and cache update are separate operations. For atomicity, you should implement the cache-aside pattern at the application layer: update the database first, then call setCache() or evict() to invalidate the cache, accepting that a small window of inconsistency may exist between the two systems.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}