How to Use Moka Async Cache in Rust: The Complete Guide to Future-Aware Caching

Moka provides a fully asynchronous, futures-aware cache in the moka::future module that uses lock-free reads and per-key async locks to coalesce concurrent initialization, making it compatible with Tokio, async-std, or any executor implementing Future.

The moka-rs/moka crate implements a high-performance concurrent cache that supports both synchronous and asynchronous operations. The async variant in src/future/ is built on the same lock-free, segmented hash table as the synchronous version, but all mutating operations return Futures and employ sophisticated concurrency control to prevent duplicate work.

Understanding the Moka Async Cache Architecture

The async cache relies on several core components that work together to provide safe, concurrent access:

• Cache<K,V,S> (src/future/cache.rs) – The public async cache type that holds a reference-counted BaseCache and a ValueInitializer to serialize concurrent initialization of the same key.
• BaseCache<K,V,S> (src/future/base_cache.rs) – Implements the low-level lock-free segments, buckets, eviction policies, and housekeeping tasks.
• CacheBuilder<K,V> (src/future/builder.rs) – Provides the builder pattern for configuring capacity, expiration policies, and async eviction listeners.
• ValueInitializer (src/future/value_initializer.rs) – Stores a per-key async_lock::Mutex so that concurrent get_with calls for the same missing key are coalesced into a single future evaluation.
• Housekeeper (src/future/housekeeper.rs) – Runs background expiration and eviction tasks on a timer wheel, driven automatically or via run_pending_tasks().await.

How Asynchrony Is Achieved

Lock-free reads use atomic operations on segments, allowing Cache::get to clone values without blocking. When a key is missing, Cache::get_with obtains a per-key async lock via ValueInitializer. The first caller evaluates the user-provided future while others await the same lock, guaranteeing that expensive initialization happens exactly once. Background expiration tasks run on the async runtime via tokio::spawn or your executor of choice.

Creating and Configuring the Async Cache

Build a cache using Cache::builder() or the shorthand Cache::new(max_capacity). The builder supports time-to-live (TTL), time-to-idle (TTI), size-based eviction, and async eviction listeners.

use moka::future::Cache;
use std::time::Duration;

#[tokio::main]
async fn main() {
    // Build a cache with 10,000 entries, 30-minute TTL, 5-minute TTI
    let cache = Cache::builder()
        .max_capacity(10_000)
        .time_to_live(Duration::from_secs(30 * 60))
        .time_to_idle(Duration::from_secs(5 * 60))
        .build();

    // Insert a value
    cache.insert("answer", 42).await;

    // Retrieve a clone of the value
    assert_eq!(cache.get(&"answer").await, Some(42));
}

The CacheBuilder::build method is implemented in src/future/builder.rs, while Cache::insert and Cache::get reside in src/future/cache.rs. The cache uses Arc internally, so cloning it to share across tasks is cheap.

Coalescing Concurrent Initialization with get_with

The get_with method prevents thundering herds by ensuring that concurrent requests for the same missing key execute only one initialization future. This is implemented via ValueInitializer::try_init_or_read using a per-key async lock from src/future/key_lock.rs.

use moka::future::Cache;
use std::sync::Arc;

#[tokio::main]
async fn main() {
    let cache = Cache::new(100);

    // Simulate a costly async computation
    async fn compute_heavy() -> Arc<Vec<u8>> {
        println!("computing…");
        Arc::new(vec![0u8; 10 * 1024 * 1024]) // 10 MiB
    }

    // Spawn several tasks that all request the same key
    let handles: Vec<_> = (0..4)
        .map(|i| {
            let c = cache.clone();
            tokio::spawn(async move {
                let v = c.get_with("big", compute_heavy()).await;
                println!("task {i} got {} bytes", v.len());
            })
        })
        .collect();

    futures_util::future::join_all(handles).await;
}

Only the first task prints "computing…"; others receive the cached result. For fallible initialization, use try_get_with, and for optional initialization, use optionally_get_with.

Using the Entry Selector API for Atomic Operations

The entry selector API provides fine-grained control over insert-or-update logic. Cache::entry returns an OwnedKeyEntrySelector (defined in src/future/entry_selector.rs) that supports conditional insertion and atomic updates.

use moka::future::Cache;

#[tokio::main]
async fn main() {
    let cache: Cache<String, u64> = Cache::new(100);
    let key = "counter".to_string();

    // Insert if missing, otherwise get existing
    let entry = cache.entry(key.clone()).or_insert(1).await;
    assert!(entry.is_fresh()); // freshly inserted
    assert_eq!(entry.into_value(), 1);

    // Increment atomically using and_upsert_with
    let entry = cache
        .entry(key.clone())
        .and_upsert_with(|maybe| async move {
            let next = maybe.map_or(1, |e| e.into_value() + 1);
            next
        })
        .await;
    assert!(!entry.is_fresh()); // updated, not fresh
    assert_eq!(entry.into_value(), 2);
}

The and_compute_with and and_upsert_with methods accept async closures, allowing you to perform asynchronous logic during the update while holding the entry lock.

Configuring Async Eviction Listeners and Custom Expiration

For resource cleanup, attach an AsyncEvictionListener to handle evictions asynchronously. The listener receives the key, value, and RemovalCause, returning a ListenerFuture (a boxed future defined in src/notification.rs).

use moka::future::Cache;
use moka::notification::{AsyncEvictionListener, ListenerFuture, RemovalCause};
use std::time::Duration;

#[tokio::main]
async fn main() {
    let listener: AsyncEvictionListener<&'static str, String> =
        |k, v, cause| -> ListenerFuture {
            Box::pin(async move {
                println!("evicted key={k}, value={v}, cause={:?}", cause);
            })
        };

    let cache = Cache::builder()
        .max_capacity(2)
        .time_to_live(Duration::from_secs(2))
        .async_eviction_listener(listener)
        .build();

    cache.insert("a", "alpha".to_string()).await;
    
    // Force pending evictions to run
    tokio::time::sleep(Duration::from_secs(3)).await;
    cache.run_pending_tasks().await;
}

Per-Entry Expiration with the Expiry Trait

Implement the Expiry trait (defined in src/lib.rs) to set custom TTLs based on entry content:

use moka::{future::Cache, Expiry};
use std::time::{Duration, Instant};

enum Exp { Short, Long, Never }

impl Exp {
    fn as_duration(&self) -> Option<Duration> {
        match self {
            Exp::Never => None,
            Exp::Short => Some(Duration::from_secs(5)),
            Exp::Long => Some(Duration::from_secs(30)),
        }
    }
}

struct MyExpiry;
impl Expiry<u32, (Exp, String)> for MyExpiry {
    fn expire_after_create(&self, _: &u32, v: &(Exp, String), _: Instant) -> Option<Duration> {
        v.0.as_duration()
    }
}

#[tokio::main]
async fn main() {
    let cache = Cache::builder()
        .max_capacity(10)
        .expire_after(MyExpiry)
        .build();

    cache.get_with(0, async { (Exp::Short, "tmp".to_string()) }).await;
    tokio::time::sleep(Duration::from_secs(6)).await;
    cache.run_pending_tasks().await;
    
    assert!(!cache.contains_key(&0));
}

Summary

• Use moka::future::Cache for fully asynchronous caching compatible with any async runtime.
• Leverage get_with to prevent duplicate work; the ValueInitializer in src/future/value_initializer.rs coalesces concurrent requests for the same key using per-key async locks.
• Clone the cache freely to share across tasks—it only increments an internal Arc.
• Use the entry selector API (cache.entry(key).or_insert().await) for atomic insert-or-update operations.
• Configure expiration via builder methods or implement the Expiry trait for per-entry policies.
• Attach async eviction listeners to clean up external resources when entries expire or are evicted.
• Call run_pending_tasks().await to force immediate execution of background expiration tasks when testing or during shutdown.

Frequently Asked Questions

What is the difference between moka::sync::Cache and moka::future::Cache?

The synchronous cache in moka::sync blocks the current thread during maintenance operations and uses synchronous locks, while moka::future::Cache in src/future/cache.rs returns Futures for all mutating operations and uses async-aware primitives like async_lock::Mutex for the ValueInitializer. The async version runs housekeeping tasks (expiration, eviction) on the async runtime rather than in background threads.

How does Moka prevent multiple tasks from computing the same value simultaneously?

When using get_with or try_get_with, Moka uses a ValueInitializer that maintains a per-key async lock (see src/future/key_lock.rs). The first task acquires the lock and executes the initialization future; subsequent tasks for the same key await the same lock and receive the cached result once insertion completes, ensuring the computation runs exactly once.

Can I use Moka async cache with async-std instead of Tokio?

Yes. The moka::future cache is runtime-agnostic because it relies on standard Future traits and the futures crate's asynchronous primitives. While the examples use tokio::main, the cache works with async-std, smol, or any executor that can drive Futures to completion.

When should I call run_pending_tasks()?

Normally, the cache's internal Housekeeper (defined in src/future/housekeeper.rs) schedules expiration and eviction automatically during cache operations. However, you should call cache.run_pending_tasks().await during graceful shutdown to ensure all pending eviction listeners execute, or in tests when you need deterministic timing for expiration verification.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}