# How CoCache's Multi-Level Caching Architecture Works: L1 Client-Side vs L2 Distributed

> Explore CoCache's multi-level caching: fast L1 client-side and shared L2 distributed cache. Learn how event-driven eviction ensures consistency between caches. Understand CoCache architecture.

- Repository: [Ahoo Wang/cocache](https://github.com/ahoo-wang/cocache)
- Tags: architecture
- Published: 2026-02-23

---

**CoCache implements a two-level caching hierarchy where L1 serves as a fast, client-side in-process cache and L2 acts as a shared distributed store, coordinated by `DefaultCoherentCache` to ensure consistency through event-driven eviction propagation.**

The `ahoo-wang/cocache` library provides a coherent caching solution that bridges local JVM performance with distributed consistency. By combining an **L1 client-side cache** (in-process) with an **L2 distributed cache** (Redis, Memcached), CoCache minimizes network latency for hot data while maintaining cross-instance coherence through a publish-subscribe eviction mechanism.

## Architecture Overview

CoCache's multi-level design centers on the `DefaultCoherentCache` class, which orchestrates data flow between three tiers: **L0** (the underlying data source), **L1** (local JVM cache), and **L2** (distributed shared cache). The architecture is configured through `CoherentCacheConfiguration`, allowing developers to plug in specific implementations for each level.

The **L1 cache** defaults to `MapClientSideCache`, a simple in-process `ConcurrentHashMap` stored within the application heap. The **L2 cache** implements the `DistributedCache` interface, typically backed by Redis or similar stores. This separation allows microservices to serve repeated requests from local memory while ensuring that cache updates propagate to all nodes via the distributed layer.

## Configuration and Setup

Cache hierarchy behavior is defined in [`CoherentCacheConfiguration.kt`](https://github.com/ahoo-wang/cocache/blob/main/CoherentCacheConfiguration.kt). The data class accepts both cache implementations along with consistency settings:

```kotlin
data class CoherentCacheConfiguration<K, V>(
    override val cacheName: String,
    override val clientId: String,
    val keyConverter: KeyConverter<K>,
    val distributedCache: DistributedCache<V>,        // L2
    val clientSideCache: ClientSideCache<V> = MapClientSideCache(),  // L1
    val cacheSource: CacheSource<K, V> = CacheSource.noOp(),
    val keyFilter: KeyFilter = NoOpKeyFilter
)

```

To instantiate a coherent cache with Redis as L2 and the default in-process L1:

```kotlin
val redisCache: DistributedCache<String> = RedisCacheBuilder
    .builder()
    .host("localhost")
    .port(6379)
    .build()

val config = CoherentCacheConfiguration(
    cacheName = "userCache",
    clientId = UUID.randomUUID().toString(),
    keyConverter = StringKeyConverter(),
    distributedCache = redisCache,
    clientSideCache = MapClientSideCache()
)

val cache = DefaultCoherentCacheFactory().create(config)

```

## Read Path and Cache Lookup Flow

The `getCache` method in [`DefaultCoherentCache.kt`](https://github.com/ahoo-wang/cocache/blob/main/DefaultCoherentCache.kt) implements a hierarchical lookup strategy with cache breakdown protection. As implemented in lines 49-75, the method follows a strict L1 → L2 → L0 progression:

```kotlin
override fun getCache(key: K): CacheValue<V>? {
    val cacheKey = keyConverter.toStringKey(key)
    
    // Step 1: Attempt L1 (client-side) lookup
    getL2Cache(cacheKey)?.let { return it }
    
    // Step 2: Acquire fine-grained lock to prevent thundering herd
    val lock = getLock(cacheKey)
    synchronized(lock) {
        // Double-check after lock acquisition
        getL2Cache(cacheKey)?.let { return it }
        
        // Step 3: Load from L0 (source) and populate both levels
        cacheSource.loadCacheValue(key)?.let {
            setCache(cacheKey, it)
            return it
        }
        
        // Step 4: Store missing guard for negative caching
        setCache(cacheKey, DefaultCacheValue.missingGuard(ttl, ttlAmplitude))
        return null
    }
}

```

**L1 promotion** occurs automatically when L2 returns a hit. The `getL2Cache` helper method first consults `clientSideCache.getCache()`. If absent, it queries `distributedCache.getCache()` and, upon success, writes the value back to L1 via `clientSideCache.setCache()`. This ensures subsequent requests for the same key resolve entirely in local memory.

## Write Path and Consistency Guarantees

Writes propagate atomically to both levels. The `setCache` implementation (lines 42-48 in [`DefaultCoherentCache.kt`](https://github.com/ahoo-wang/cocache/blob/main/DefaultCoherentCache.kt)) updates L1 and L2 simultaneously before broadcasting an eviction event:

```kotlin
override fun setCache(key: K, value: CacheValue<V>) {
    if (value.isExpired) return
    val cacheKey = keyConverter.toStringKey(key)
    clientSideCache.setCache(cacheKey, value)    // L1 write
    distributedCache.setCache(cacheKey, value)   // L2 write
    cacheEvictedEventBus.publish(
        CacheEvictedEvent(cacheName, cacheKey, clientId)
    )
}

```

This **write-through pattern** ensures that the distributed layer always holds the latest value, while the accompanying `CacheEvictedEvent` signals other application instances to invalidate their local L1 copies. The `clientId` field prevents nodes from processing their own eviction events.

## Eviction Propagation and Coherence

Cache coherence relies on an event-driven invalidation strategy. When any node evicts a key, the `CacheEvictedEventBus` distributes the event to all registered listeners. The `onEvicted` handler (lines 58-80 in [`DefaultCoherentCache.kt`](https://github.com/ahoo-wang/cocache/blob/main/DefaultCoherentCache.kt)) processes these messages:

```kotlin
@Subscribe
override fun onEvicted(cacheEvictedEvent: CacheEvictedEvent) {
    if (cacheEvictedEvent.cacheName != cacheName) return
    if (cacheEvictedEvent.publisherId == clientId) return  // Ignore self
    
    // Evict only from L1; L2 is already updated by the publisher
    clientSideCache.evict(cacheEvictedEvent.key)
}

```

This approach maintains **eventual consistency** across the cluster. The originating node updates L2, while peer nodes purge their L1 snapshots upon receiving the event, forcing them to fetch fresh data from L2 on the next access.

## Concurrency Control and Cache Breakdown Protection

CoCache prevents thundering-herd scenarios through **fine-grained locking** per cache key. The `keyLocks` map stores key-specific lock objects, ensuring that only one thread executes the L0 data source query when both cache levels miss:

```kotlin
private fun getLock(key: String): Any {
    return keyLocks.computeIfAbsent(key) { Any() }
}

private fun releaseLock(key: String) {
    keyLocks.remove(key)
}

```

Locks are created on demand via `computeIfAbsent` and released after the critical section completes. This mechanism, defined in lines 77-86 of [`DefaultCoherentCache.kt`](https://github.com/ahoo-wang/cocache/blob/main/DefaultCoherentCache.kt), guarantees that expensive database queries execute exactly once per cache key during a miss storm.

## Summary

- **Two-level hierarchy**: L1 (`ClientSideCache`) provides sub-millisecond local access, while L2 (`DistributedCache`) ensures cross-node consistency.
- **Hierarchical reads**: Lookups cascade from L1 → L2 → L0, with automatic promotion from distributed to local cache.
- **Atomic writes**: The `setCache` method updates both levels and publishes eviction events to maintain coherence.
- **Event-driven eviction**: `CacheEvictedEvent` bus ensures that L1 entries are invalidated cluster-wide when data changes.
- **Thundering-herd protection**: Per-key locks in `DefaultCoherentCache` serialize access to the underlying data source during cache misses.

## Frequently Asked Questions

### What happens when L1 and L2 both miss?

When both cache levels miss, CoCache acquires a per-key lock and queries the **L0 source** (database or remote service). The result is written to both L1 and L2 before releasing the lock, ensuring subsequent requests hit the cache. If the source returns null, a "missing guard" placeholder is stored to prevent repeated useless queries.

### How does CoCache handle cache eviction across multiple application instances?

Eviction uses a **pub-sub model**. When one instance writes or evicts a key, it publishes a `CacheEvictedEvent` to the bus. Other instances receive this event and evict the key from their local L1 only, leaving L2 (the shared source of truth) intact. This ensures all nodes refresh their local snapshots on the next read.

### Can I disable the L1 client-side cache?

Yes. While `CoherentCacheConfiguration` defaults to `MapClientSideCache()`, you can pass a no-op implementation of `ClientSideCache` or configure the cache to use L2 exclusively. However, this eliminates the performance benefits of local heap access and increases load on the distributed cache.

### What distributed cache implementations does CoCache support?

The `DistributedCache` interface in `cocache-api` supports any backing store. The repository provides concrete implementations for **Redis** (via `cocache-spring-redis`) and can be extended for Memcached, Hazelcast, or other systems. The L2 implementation is injected into `CoherentCacheConfiguration` at instantiation time.