Cache-Aside Pattern with Moka: A Complete Rust Implementation Guide

Use moka::sync::Cache::get_with or moka::future::Cache::get_with_async to implement the cache-aside pattern, ensuring single-flight data loading and automatic thundering-herd protection during cache misses.

Moka is a high-performance, thread-safe in-memory caching library for Rust that provides both synchronous (moka::sync) and asynchronous (moka::future) APIs. The cache-aside (or lazy-loading) pattern fits naturally into Moka's architecture, allowing applications to check the cache first and load data from an underlying source only when necessary. According to the moka-rs/moka source code, this pattern is implemented through specialized methods that handle cache misses with built-in concurrency controls and efficient expiration management.

How Moka Implements Cache-Aside

The cache-aside pattern in Moka follows a three-step workflow that balances read performance with data consistency. When implemented correctly using Moka's API, the cache provides lock-free reads and guarantees that expensive loading operations execute only once per key during concurrent access.

Read-Through on Cache Hit

When an application requests a key, Moka first performs a fast lock-free lookup in its internal storage segments. If the entry exists and has not expired, the cached value returns immediately without hitting the underlying data source. This path is optimized for zero-contention reads across multiple threads.

Single-Flight Loading on Miss

If the lookup fails, Moka initiates the cache-aside loading sequence. In src/sync/cache.rs, the Cache::get_with method (and its async counterpart in src/future/cache.rs) accepts a closure that executes only when the key is absent. Internally, the cache uses a KeyLock structure defined in src/sync/key_lock.rs to serialize concurrent loaders for the same key, preventing the "thundering herd" problem where multiple threads simultaneously fetch the same missing value.

The loader closure runs exactly once, and its result is stored in the cache's segment structure (src/sync/segment.rs) for subsequent requests. The timer wheel (src/common/timer_wheel.rs) schedules expiration based on configured time-to-live (TTL) settings, while eviction policies such as LRU or FIFO are handled in src/policy.rs.

Explicit Invalidation and Updates

When underlying data changes, applications maintain consistency through explicit invalidation. The Invalidator component in src/sync/invalidator.rs handles the Cache::invalidate method, which removes stale entries immediately. Alternatively, Cache::insert performs a write-through update, replacing the cached value atomically.

Synchronous Cache-Aside Implementation

The synchronous API in src/sync/cache.rs provides the most common implementation path. Use Cache::builder() (defined in src/sync/builder.rs) to configure capacity and eviction policies, then call get_with to handle cache misses automatically.

use moka::sync::Cache;
use std::time::Duration;

// Build a cache with a 10-second TTL and max capacity of 10,000 entries.
let cache: Cache<u64, String> = Cache::builder()
    .max_capacity(10_000)
    .time_to_live(Duration::from_secs(10))
    .build();

// Simulated expensive load function (e.g., a database query).
fn load_user_name(user_id: u64) -> String {
    format!("User #{}", user_id)
}

// Cache-aside read: fetch from cache or load & populate on miss.
fn get_user_name(cache: &Cache<u64, String>, user_id: u64) -> String {
    // `get_with` runs the closure only when the key is absent.
    cache.get_with(user_id, || load_user_name(user_id))
}

The get_with method guarantees that multiple concurrent calls with the same key result in only one execution of the loader closure, protecting the underlying database or service from redundant requests.

Asynchronous Cache-Aside with Tokio

For async Rust applications, the moka::future module provides identical semantics with asynchronous loaders. The implementation in src/future/cache.rs offers get_with_async, which accepts an async closure and properly handles concurrent awaits. Configure the cache using Cache::builder() from src/future/builder.rs.

use moka::future::Cache;
use std::time::Duration;
use tokio::time::sleep;

// Build an async cache.
let cache: Cache<u64, String> = Cache::builder()
    .max_capacity(5_000)
    .time_to_live(Duration::from_secs(30))
    .build();

// Async loader simulating a remote service call.
async fn fetch_profile(user_id: u64) -> String {
    sleep(Duration::from_millis(150)).await;
    format!("AsyncUser#{}", user_id)
}

// Async cache-aside read.
async fn get_profile(cache: &Cache<u64, String>, user_id: u64) -> String {
    // `get_with_async` runs the async closure only on a miss.
    cache
        .get_with_async(user_id, async move { fetch_profile(user_id) })
        .await
}

This pattern is particularly effective for I/O-bound applications where loaders perform network requests or database queries.

Handling Cache Invalidation

Maintaining consistency requires explicit cache management when the underlying data source changes. Moka provides atomic operations to invalidate or replace entries.

// Invalidate a stale entry after an external update.
cache.invalidate(&user_id);

// Or replace it atomically with write-through semantics:
cache.insert(user_id, new_name);

The invalidate method triggers the invalidation logic in src/sync/invalidator.rs, immediately removing the entry from the cache and preventing expired data from being served to new requests.

Production-Ready Service Architecture

In real-world applications, wrap Moka in a service struct that encapsulates both the cache and the data source. This pattern leverages Arc<Cache> for thread-safe sharing across async or sync contexts.

use moka::sync::Cache;
use std::sync::Arc;
use std::time::Duration;

struct DbClient;
impl DbClient {
    fn query_user(&self, id: u64) -> String {
        format!("UserFromDB#{}", id)
    }
}

struct UserService {
    db: DbClient,
    cache: Arc<Cache<u64, String>>,
}

impl UserService {
    fn new() -> Self {
        let cache = Cache::builder()
            .max_capacity(20_000)
            .time_to_live(Duration::from_secs(60))
            .build();
        Self {
            db: DbClient,
            cache: Arc::new(cache),
        }
    }

    // Cache-aside read with automatic population.
    fn get_user(&self, user_id: u64) -> String {
        let db = &self.db;
        self.cache.get_with(user_id, || db.query_user(user_id))
    }

    // Write-through update.
    fn update_user(&self, user_id: u64, new_name: String) {
        // Update underlying store (omitted)...
        self.cache.insert(user_id, new_name);
    }

    // Explicit eviction.
    fn evict_user(&self, user_id: u64) {
        self.cache.invalidate(&user_id);
    }
}

This architecture ensures that the KeyLock mechanism in src/sync/key_lock.rs serializes concurrent requests for the same user ID, while the segment structure in src/sync/segment.rs provides sharded storage to minimize lock contention across different keys.

Summary

• Use Cache::get_with (sync) or Cache::get_with_async (async) to implement cache-aside with automatic single-flight loading.
• The KeyLock component in src/sync/key_lock.rs prevents thundering-herd problems by ensuring only one loader executes per missing key.
• Configure eviction policies and TTL through the Builder pattern in src/sync/builder.rs or src/future/builder.rs.
• Maintain consistency with invalidate (removes entries) or insert (atomic replacement).
• Review the official examples in examples/try_append_value_sync.rs and examples/try_append_value_async.rs for complete working demonstrations.

Frequently Asked Questions

What is the cache-aside pattern?

The cache-aside pattern is a caching strategy where the application code explicitly checks the cache before querying the primary data source. If the data is present (a cache hit), it is returned immediately. If absent (a cache miss), the application loads the data from the source, stores it in the cache for future requests, and then returns the value. This pattern provides full control over cache population and invalidation logic.

How does Moka prevent duplicate loads during cache misses?

Moka prevents duplicate loads through a mechanism called single-flight. When multiple threads or tasks request the same missing key simultaneously, the KeyLock structure in src/sync/key_lock.rs serializes access so that only one loader closure executes. The remaining waiters receive the cached result once the loader completes, eliminating redundant database queries or network requests.

Can I use Moka with async Rust runtimes?

Yes. Moka provides a dedicated asynchronous API under moka::future that is compatible with Tokio and other async runtimes. The Cache::get_with_async method in src/future/cache.rs accepts async closures and properly handles await points during the loading phase, providing the same single-flight guarantees as the synchronous API.

How do I handle cache expiration in Moka?

Moka handles expiration automatically through a timer wheel implementation in src/common/timer_wheel.rs. Configure time-to-live (TTL) or time-to-idle (TTI) settings using the Builder pattern (e.g., .time_to_live(Duration::from_secs(60))). The cache checks expiration during lookups and removes stale entries in the background, requiring no manual intervention for standard expiration scenarios.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}