# Moka Cache Lookup Performance: How the Lock-Free Design Achieves O(1) Speed

> Discover Moka cache lookup performance advantages. Its lock-free design achieves O(1) speed with atomic operations and epoch-based memory management for sub-nanosecond read overhead.

- Repository: [moka-rs/moka](https://github.com/moka-rs/moka)
- Tags: performance
- Published: 2026-03-07

---

**Moka delivers O(1) amortized lookup performance through a lock-free segmented hash map that uses atomic operations and crossbeam-epoch memory management, eliminating global locks and enabling sub-nanosecond read overhead compared to standard HashMap.**

The `moka-rs/moka` crate is a high-performance caching library for Rust that prioritizes **Moka cache lookup performance** through innovative concurrency primitives. Unlike traditional caches that rely on coarse-grained locking, Moka implements a lock-free architecture that allows concurrent reads to scale linearly with CPU cores.

## How Moka Cache Lookup Performance Works Under the Hood

### The Lock-Free Segmented Hash Map Architecture

At the core of Moka's speed lies a **lock-free, segmented hash map** (`cht::HashMap`) implemented in [`src/cht/segment.rs`](https://github.com/moka-rs/moka/blob/main/src/cht/segment.rs). The map divides entries across multiple independent segments—defaulting to **2 × CPU count**—with each segment owning an array of bucket pointers accessible via `crossbeam_epoch::Atomic` pointers.

This design eliminates global locks entirely. When a thread calls `Cache::get`, it hashes the key, selects a segment using high bits of the hash (`segment_shift`), and performs atomic reads on that segment's bucket array. The crossbeam-epoch memory management ensures safe, lock-free reclamation of old entries without stopping readers.

### The Seven-Step Lookup Path

A single `Cache::get` call traverses a precise, optimized path through the codebase:

1. **Public API Entry** ([`src/sync/cache.rs`](https://github.com/moka-rs/moka/blob/main/src/sync/cache.rs), lines 995-1002): `Cache::get` forwards to `BaseCache::get_with_hash`.

2. **Hash Computation** ([`src/sync/base_cache.rs`](https://github.com/moka-rs/moka/blob/main/src/sync/base_cache.rs), lines 217-224): `BaseCache::get_with_hash` creates a closure to record the read operation and passes it to the internal routine.

3. **Core Logic** ([`src/sync/base_cache.rs`](https://github.com/moka-rs/moka/blob/main/src/sync/base_cache.rs), lines 265-272): `BaseCache::do_get_with_hash` checks whether the map is disabled, computes the current instant, and calls `inner.get_key_value_and_then` on the segmented hash map.

4. **Segment Selection** ([`src/cht/segment.rs`](https://github.com/moka-rs/moka/blob/main/src/cht/segment.rs), lines 282-288): The segmented hash map selects the appropriate segment using `segment_shift` on the hash, then reads the bucket array via an atomic pointer.

5. **Expiration Check** ([`src/sync/base_cache.rs`](https://github.com/moka-rs/moka/blob/main/src/sync/base_cache.rs), lines 297-304): The entry is validated against TTL/TTI policies and explicit invalidation flags. If expired, the entry is treated as a miss.

6. **Metrics Recording** ([`src/sync/base_cache.rs`](https://github.com/moka-rs/moka/blob/main/src/sync/base_cache.rs), lines 221-224): For valid entries, `record(ReadOp::Get, now)` updates the historic popularity estimator and refreshes the idle timer using atomic increments.

7. **Return**: The value is cloned or moved out and returned to the caller.

Notably, `Cache::contains_key` follows the same path but skips step 6, avoiding the atomic record operation and making it slightly faster for pure existence checks ([`src/sync/cache.rs`](https://github.com/moka-rs/moka/blob/main/src/sync/cache.rs), lines 671-678).

## Optimizing Moka Cache Lookup Performance

While Moka's defaults provide excellent performance, you can tune specific levers for maximum throughput:

- **Number of segments**: Use `CacheBuilder::num_segments(x)` to increase shards beyond the default (2 × CPU count). More segments reduce contention under extreme thread counts at the cost of slightly higher memory usage.

- **Hasher selection**: Replace the default SipHash-1-3 with a faster non-cryptographic hasher like `ahash` using `CacheBuilder::with_hasher(ahash::RandomState::new())`. This significantly reduces hash computation time for small keys.

- **Expiration policies**: Disable TTL and TTI checks (`time_to_live(None)`, `time_to_idle(None)`) if you need pure lookup speed. This skips the expiration validation step entirely.

- **Read operation recording**: Use `CacheBuilder::record_read_ops(false)` to disable the popularity estimator. This removes the atomic increment in step 6, yielding pure read performance comparable to a raw `HashMap`.

## Code Examples for High-Performance Lookups

Here are practical implementations demonstrating optimized lookup patterns.

### Synchronous High-Throughput Cache

```rust
use moka::sync::Cache;
use std::time::Duration;

fn main() {
    // Build a cache optimized for raw lookup speed
    let cache = Cache::builder()
        .max_capacity(1_000_000)
        .num_segments(8)               // More segments reduce contention
        .time_to_live(None)            // Disable expiration checks
        .record_read_ops(false)        // Skip popularity tracking
        .build();

    // Insert a value (expensive to compute only once)
    cache.insert("answer", 42);

    // O(1) lock-free lookup
    if let Some(v) = cache.get("answer") {
        println!("The answer is {v}");
    }

    // Existence check without affecting metrics
    let present = cache.contains_key("answer");
    println!("Key present: {present}");
}

```

### Asynchronous Cache with Lazy Initialization

```rust
use moka::future::Cache;
use std::time::Duration;

#[tokio::main]
async fn main() {
    let cache = Cache::builder()
        .max_capacity(500_000)
        .time_to_idle(Some(Duration::from_secs(30)))
        .build();

    // get_with records the read and lazily creates if missing
    let val = cache
        .get_with("config", async {
            // Simulate async initialization
            tokio::time::sleep(Duration::from_millis(10)).await;
            "loaded_config".to_string()
        })
        .await;
    
    println!("config = {val}");
}

```

## Key Source Files and Implementation Details

Understanding the lookup path requires familiarity with these specific files in the `moka-rs/moka` repository:

| File | Role | Key Components |
|------|------|----------------|
| [`src/sync/cache.rs`](https://github.com/moka-rs/moka/blob/main/src/sync/cache.rs) | Public API surface | `Cache::get`, `Cache::contains_key`, entry API |
| [`src/sync/base_cache.rs`](https://github.com/moka-rs/moka/blob/main/src/sync/base_cache.rs) | Core lock-free logic | `get_with_hash`, `do_get_with_hash`, expiration checks |
| [`src/cht/segment.rs`](https://github.com/moka-rs/moka/blob/main/src/cht/segment.rs) | Segmented hash map | Segment selection, `segment_shift`, bucket array access |
| [`src/cht/map/bucket.rs`](https://github.com/moka-rs/moka/blob/main/src/cht/map/bucket.rs) | Bucket storage | Linear probing, open addressing |
| [`src/common/concurrent/arc.rs`](https://github.com/moka-rs/moka/blob/main/src/common/concurrent/arc.rs) | Reference counting | Lock-free `Arc` wrapper for entries |
| [`src/policy.rs`](https://github.com/moka-rs/moka/blob/main/src/policy.rs) | Cache policies | TTL, TTI, weight definitions affecting lookup validation |

These files demonstrate how Moka transforms a simple `get` call into a lock-free segment lookup, expiration validation, and optional metrics recording—all executing in constant time with minimal CPU overhead.

## Summary

- Moka achieves **O(1) amortized lookup performance** through a lock-free segmented hash map architecture that eliminates global locks.
- The lookup path traverses seven optimized steps: API entry, hash computation, core logic, segment selection, expiration checking, metrics recording, and return.
- **Lock-free reads** via `crossbeam_epoch::Atomic` pointers and segmented sharding (default 2× CPU count) enable linear scalability with thread count.
- You can optimize further by tuning segment count, switching to faster hashers like `ahash`, disabling expiration policies, and turning off read-op recording.

## Frequently Asked Questions

### What is the time complexity of Moka cache lookups?

Moka cache lookups run in **O(1) amortized time complexity**, similar to standard hash maps. Each lookup performs a constant amount of work: hashing the key, selecting a segment via bit shifting, following an atomic pointer to the bucket array, and validating the entry. The lock-free design ensures that thread contention does not degrade this complexity, maintaining constant-time performance even under high concurrency.

### How does Moka achieve lock-free concurrency?

Moka implements lock-free concurrency using a **segmented hash map** built on `crossbeam_epoch::Atomic` pointers. Instead of locking the entire table, the map shards entries across multiple segments (defaulting to twice the CPU count). Each segment maintains its own bucket array accessible through atomic operations. The crossbeam-epoch memory management allows safe, lock-free reclamation of removed entries without stopping readers, eliminating global lock contention entirely.

### Can I disable features to improve lookup speed?

Yes, you can disable several features to maximize **Moka cache lookup performance**. Use `CacheBuilder::record_read_ops(false)` to disable the popularity estimator and remove atomic increments from the read path. Set `time_to_live(None)` and `time_to_idle(None)` to skip expiration validation checks entirely. Additionally, increasing segments via `num_segments()` reduces contention, while switching to a faster hasher like `ahash` minimizes hash computation overhead.

### How does Moka compare to standard HashMap performance?

Moka provides **sub-nanosecond overhead** compared to a raw `std::collections::HashMap` when configured optimally. The benchmark suite in the repository's `benches/` directory demonstrates that Moka's lock-free reads, with features like read-op recording and expiration disabled, perform nearly identically to standard hash map lookups while adding thread safety and eviction policies. The primary difference emerges under high concurrency, where Moka's segmented design scales linearly while standard HashMap would require locking wrappers that degrade performance.