How Moka Handles Cache Size Limits: Understanding max_capacity and Eviction

Moka enforces cache size limits through the max_capacity builder configuration, which propagates to an internal Policy struct and triggers eviction via TinyLFU or LRU algorithms when entry counts or weighted sizes exceed the configured threshold.

Moka is a fast, concurrent caching library for Rust that prevents unbounded memory growth through sophisticated admission control. Understanding how moka cache size limits work requires tracing the path from the public CacheBuilder API through internal policy storage to the runtime eviction mechanisms that maintain capacity constraints across single-segment and segmented architectures.

Configuring max_capacity in the Cache Builder

The size limit lifecycle begins when you invoke max_capacity() on a CacheBuilder. This method records the user-provided value as Some(u64) for later propagation to the cache internals.

In src/sync/builder.rs (lines 386-388), the builder stores the limit:

// Conceptual implementation from src/sync/builder.rs L386-L388
pub fn max_capacity(mut self, max_capacity: u64) -> Self {
    self.max_capacity = Some(max_capacity);
    self
}

This value persists in the builder until build() is called, at which point it flows into the concrete Cache or SegmentedCache implementation and eventually into the Policy engine.

Policy Propagation and Storage

Once the builder constructs the cache, it passes the capacity limit to Policy::new, which stores it in the Policy.max_capacity field. This centralizes the constraint within the policy engine that drives all eviction decisions.

According to the moka source in src/policy.rs (lines 11-19), the Policy struct maintains this state:

// From src/policy.rs L11-L19
pub struct Policy {
    max_capacity: u64,
    // ... additional policy configuration
}

For single-segment caches, the desired capacity equals the user-specified max_capacity. For segmented caches, the value is divided among segments to reduce lock contention while maintaining the total bound.

Capacity Distribution in Segmented Caches

When using SegmentedCache, moka distributes the total max_capacity across multiple segments. Each segment receives an equal share calculated via ceiling division to ensure the sum meets or exceeds the user limit.

In src/sync/segment.rs (lines 723-724), the per-segment capacity is computed:

// src/sync/segment.rs L723-L724
let per_segment = max_capacity.div_ceil(actual_num_segments as u64);

This division ensures that concurrent write operations on different segments can proceed without global locks while respecting the aggregate cache size limit.

Runtime Eviction and Admission Control

Every write operation—including insert, get_with, and related methods—checks whether the cache exceeds its limit. When the threshold is reached, the admission policy evaluates whether to accept the new entry and which existing entries to evict.

The policy consultation occurs in src/sync/segment.rs (lines 138-140):

// src/sync/segment.rs L138-L140
policy.set_max_capacity(self.inner.desired_capacity);

By default, moka uses the TinyLFU (Time-decaying Least Frequently Used) admission policy, with LRU (Least Recently Used) available as an alternative. These policies estimate entry value to maximize hit rates while strictly enforcing the size constraint through atomic checks.

Zero-Capacity Edge Case

Setting max_capacity(0) creates a disabled cache that behaves as a pass-through. As implemented in src/sync/segment.rs (lines 787-801), inserts are silently ignored and read operations always miss, allowing applications to disable caching without modifying consumer code.

Weighted Size-Aware Eviction

Beyond simple entry counting, moka supports weighted capacity limits. When a weigher function is provided, the max_capacity constraint applies to the sum of weights rather than the entry count.

The cache maintains an internal weighted_size counter that it compares against max_capacity during eviction. This enables precise memory management when caching objects of varying sizes, such as large buffers or complex data structures.

Implementation Examples

The following examples demonstrate practical usage of moka cache size limits in Rust applications.

Basic Entry Count Limit

Build a cache that holds at most 100 entries, automatically evicting the least valuable entries when the limit is exceeded:

use moka::sync::Cache;

// Build a cache that may hold at most 100 entries.
let cache = Cache::builder()
    .max_capacity(100)               // <-- size limit
    .build();

// Insert 101 items – the oldest / least‑frequent one is evicted automatically.
for i in 0..101 {
    cache.insert(i, i * 2);
}

// The cache never exceeds the configured capacity.
assert_eq!(cache.entry_count(), 100);

Monitoring Eviction Events

Observe eviction behavior in real-time using an eviction listener:

use moka::sync::Cache;
use moka::notification::RemovalCause;

// Observe eviction via a listener.
let evicted = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let listener = {
    let evicted = evicted.clone();
    move |k: &u32, v: &u32, cause: RemovalCause| {
        evicted.lock().unwrap().push((*k, *v, cause));
    }
};

let cache = Cache::builder()
    .max_capacity(2)       // tiny limit → eviction on every third insert
    .eviction_listener(listener)
    .build();

cache.insert(1, 10);
cache.insert(2, 20);
cache.insert(3, 30);      // triggers eviction of one entry

// Verify that an eviction occurred.
assert!(!evicted.lock().unwrap().is_empty());

Weighted Capacity Constraints

Use a weigher function to limit cache size by weighted value rather than entry count:

use moka::sync::Cache;

// Size‑aware eviction using a weigher (weighted capacity of 50).
let weigher = |_k: &u32, v: &u32| *v;               // weight = value itself
let cache = Cache::builder()
    .max_capacity(50)        // weighted limit
    .weigher(weigher)
    .build();

cache.insert(1, 30);        // total weight = 30
cache.insert(2, 25);        // exceeds 50 → evicts the least‑valuable entry
assert!(cache.get(&1).is_none() || cache.get(&2).is_none());

Summary

Moka implements cache size limits through a coordinated pipeline of configuration, policy propagation, and runtime enforcement:

• Builder Configuration: CacheBuilder::max_capacity stores the limit as Some(u64) in src/sync/builder.rs (lines 386-388)
• Policy Storage: The limit propagates to Policy::new and stores in Policy.max_capacity within src/policy.rs (lines 11-19)
• Segment Distribution: Segmented caches divide capacity via ceil(max_capacity / actual_num_segments) in src/sync/segment.rs (lines 723-724)
• Runtime Enforcement: Write operations check bounds and trigger TinyLFU/LRU admission via policy.set_max_capacity in src/sync/segment.rs (lines 138-140)
• Weighted Limits: Optional weigher functions switch enforcement from entry count to weighted size totals tracked via weighted_size
• Edge Case Handling: Zero-capacity caches gracefully degrade to pass-through behavior as implemented in src/sync/segment.rs (lines 787-801)

Frequently Asked Questions

What happens when a Moka cache reaches its max_capacity?

When the cache reaches its limit, new insertions trigger the eviction policy before admission. The default TinyLFU algorithm evaluates whether the new entry is more valuable than existing entries, evicting the least valuable items to maintain the bound. This atomic check occurs during write operations in src/sync/cache.rs and src/sync/segment.rs.

How does max_capacity work with SegmentedCache?

For segmented caches, moka divides max_capacity equally among segments using ceiling division. Each segment receives ceil(max_capacity / actual_num_segments) capacity as calculated in src/sync/segment.rs (lines 723-724). The eviction policy applies independently per segment, though the total cache size never exceeds the configured limit.

Can I use max_capacity to limit memory usage instead of entry count?

Yes, by providing a weigher closure to the builder, max_capacity applies to the sum of weights rather than entry counts. The cache tracks weighted_size against the limit, enabling byte-accurate memory constraints. This is essential when caching large objects or variable-size data structures where entry count is a poor proxy for memory consumption.

What is the behavior of a Moka cache with max_capacity(0)?

A cache configured with max_capacity(0) behaves as if disabled. As implemented in src/sync/segment.rs (lines 787-801), all insert operations are ignored and get operations always return None. This provides a safe fallback configuration for "caching disabled" scenarios without requiring conditional logic in application code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}