What is moka-rs? A Deep Dive into the High-Performance Rust Cache Library

moka-rs is a high-performance, thread-safe cache library for Rust that implements TinyLFU admission and LRU eviction policies on top of a lock-free concurrent hash table.

The moka-rs/moka repository provides a concurrent caching solution for Rust applications that rivals Java's Caffeine library in performance and features. It offers both synchronous and asynchronous APIs designed for high-concurrency scenarios without blocking threads or requiring background maintenance tasks.

Core Architecture of moka-rs

At the heart of moka-rs lies a lock-free concurrent hash table derived from the cht crate. This data structure, implemented in src/cht/segment.rs, allows multiple threads to read and write cache entries without traditional locking mechanisms, minimizing contention in hot paths.

Unlike many cache implementations that spawn background threads for maintenance, moka-rs adopts an eventual consistency model for its policy structures. Reads and writes are recorded on bounded channels (described in src/lib.rs), and maintenance tasks—such as updating the TinyLFU sketch or LRU queues—are executed lazily on the thread that invokes a cache method. This design eliminates background thread overhead while keeping the hash table strongly consistent.

Cache Variants and APIs

Synchronous Caching with sync::Cache

The sync::Cache type in src/sync/cache.rs provides a thread-safe, synchronous API suitable for standard multi-threaded applications. It implements Clone cheaply, allowing references to be shared across threads without complex lifetime management.

use moka::sync::Cache;
use std::thread;

fn value(n: usize) -> String {
    format!("value {n}")
}

fn main() {
    // Cache that holds up to 10,000 entries.
    let cache = Cache::new(10_000);

    // Populate the cache from multiple threads.
    const THREADS: usize = 4;
    const KEYS_PER_THREAD: usize = 32;

    let handles: Vec<_> = (0..THREADS).map(|i| {
        let c = cache.clone();                     // cheap clone
        thread::spawn(move || {
            for k in (i * KEYS_PER_THREAD)..((i + 1) * KEYS_PER_THREAD) {
                c.insert(k, value(k));
                assert_eq!(c.get(&k), Some(value(k)));
            }
        })
    }).collect();

    for h in handles { h.join().expect("thread panicked"); }

    // Verify a few entries.
    assert_eq!(cache.get(&5), Some(value(5)));
}

Asynchronous Caching with future::Cache

For async runtimes like Tokio, async-std, or actix-rt, the future::Cache in src/future/cache.rs provides a futures-aware API. It supports async insertion methods and handles blocking operations correctly within async contexts.

use moka::future::Cache;
use tokio::time::{sleep, Duration};

#[tokio::main]
async fn main() {
    // Async cache with a 5-second TTL.
    let cache = Cache::builder()
        .time_to_live(Duration::from_secs(5))
        .max_capacity(100)
        .build();

    // Insert or retrieve a value atomically.
    let v = cache.get_with("key", || async { expensive_compute().await }).await;
    println!("value = {v}");

    // Wait for expiration.
    sleep(Duration::from_secs(6)).await;
    assert!(cache.get(&"key").await.is_none());
}

async fn expensive_compute() -> String {
    // Simulate work.
    "computed".to_string()
}

Advanced Configuration Options

Admission and Eviction Policies

moka-rs implements the TinyLFU (Tiny Least Frequently Used) admission policy combined with LRU (Least Recently Used) eviction by default. This approach, defined in src/policy.rs, maintains a frequency sketch to admit only high-value entries while evicting the least recently used items when capacity is exceeded. Users can optionally switch to a pure LRU policy if workload characteristics demand it.

Size-Aware Eviction

Beyond entry counting, moka-rs supports weighted capacity through the weigher function in src/sync/builder.rs. This allows caches to limit total memory consumption based on actual byte sizes or custom weight metrics.

use moka::sync::Cache;

let cache = Cache::builder()
    .weigher(|_k: &u32, v: &String| v.len() as u32) // weight = byte length
    .max_capacity(32 * 1024 * 1024)               // 32 MiB total weight
    .build();

cache.insert(1, "a".repeat(1_000_000)); // ~1 MiB
cache.insert(2, "b".repeat(2_000_000)); // ~2 MiB
// When total weight exceeds 32 MiB, least-recently-used entries are evicted.

Entry Expiration

moka-rs provides flexible expiration mechanisms through hierarchical timer wheels implemented in src/common/timer_wheel.rs. Developers can configure time-to-live (TTL) for absolute expiration, time-to-idle (TTI) for expiration after periods of inactivity, or assign per-entry variable expiration based on runtime conditions.

Eviction Listeners

For monitoring or cleanup operations, moka-rs supports eviction listeners via src/notification.rs. These callbacks trigger on every removal—whether through eviction, expiration, or manual invalidation—providing the key, value, and removal cause.

use moka::sync::Cache;
use moka::notification::RemovalCause;

let listener = |k: &u32, v: &String, cause: RemovalCause| {
    println!("evicted key={k}, value={v}, cause={:?}", cause);
};

let cache = Cache::builder()
    .max_capacity(2)
    .eviction_listener(listener)
    .build();

cache.insert(1, "one".to_string());
cache.insert(2, "two".to_string());
cache.insert(3, "three".to_string()); // evicts the LRU entry (key 1)

Platform Support and Concurrency Model

moka-rs targets 64-bit and 32-bit Unix systems, Windows, and macOS. It explicitly does not support WebAssembly (WASM/WASI) due to its reliance on OS-specific threading primitives.

The library's concurrency model is distinctive: it spawns no background threads. Instead, maintenance operations run on the thread that invokes cache methods, using bounded channels to prevent blocking under heavy load. This design, documented in src/lib.rs, ensures predictable resource usage and eliminates the complexity of background thread lifecycle management.

Summary

• moka-rs is a Rust cache library implementing TinyLFU admission and LRU eviction policies on a lock-free concurrent hash table.
• It provides two main APIs: sync::Cache for synchronous multi-threaded applications and future::Cache for async runtimes like Tokio.
• Advanced features include size-aware eviction via custom weighers, hierarchical timer wheels for TTL/TTI expiration, and eviction listeners for removal notifications.
• The library uses a unique maintenance model with bounded channels and no background threads, ensuring strong consistency for the hash table and eventual consistency for policy metadata.
• It supports major desktop platforms (Linux, macOS, Windows) but does not target WebAssembly.

Frequently Asked Questions

What is moka-rs used for?

moka-rs is used to add high-performance, concurrent caching to Rust applications. It is ideal for web servers, data processing pipelines, and embedded systems where multiple threads need fast access to shared data without lock contention. The library's support for both synchronous and asynchronous contexts makes it suitable for blocking I/O workloads as well as async runtimes like Tokio and async-std.

How does moka-rs differ from other Rust cache libraries?

Unlike many cache libraries that rely on mutex-based locking or spawn background threads for maintenance, moka-rs uses a lock-free concurrent hash table (in src/cht/segment.rs) and performs maintenance lazily on caller threads. It uniquely implements the TinyLFU admission policy, which filters out low-value entries before they enter the cache, improving hit rates for skewed workloads compared to pure LRU implementations.

Does moka-rs require a background thread for maintenance?

No. moka-rs explicitly avoids background threads. Maintenance tasks—such as updating the TinyLFU sketch or processing eviction notifications—are executed on the thread that calls a cache method. The library uses bounded channels (documented in src/lib.rs) to buffer these tasks, preventing blocking reads and throttling writes under heavy load. This design simplifies deployment and resource management.

Can I use moka-rs in WebAssembly projects?

No, moka-rs does not support WebAssembly (WASM or WASI). The library relies on OS-specific threading primitives and atomic operations that are not available in WASM environments. According to the project documentation, it is designed for native 64-bit and 32-bit Unix systems, Windows, and macOS only.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}