Why Moka Has No Background Threads: Zero-Cost Housekeeping Explained

Moka eliminated background threads in v0.12.0 to perform cache housekeeping lazily on the caller's thread, reducing resource usage and improving compatibility with async runtimes.

Since version 0.12.0, the moka-rs/moka cache library operates with moka no background threads, moving all eviction, expiration, and notification work to the thread that accesses the cache. This architectural shift removes the scheduled-thread-pool dependency and converts housekeeping tasks to run on-demand via the Housekeeper component.

What Changed in v0.12.0?

The removal of background threads represents a fundamental redesign of how Moka manages internal state. Prior to v0.12.0, the library spawned dedicated thread pools for housekeeping tasks.

• Thread pool removal: All cache types—future::Cache, sync::Cache, and sync::SegmentedCache—no longer spawn background threads. The README explicitly documents this change at lines 59-60: "No more background threads: All cache types … no longer spawn background threads."

• Dependency cleanup: The scheduled-thread-pool crate was completely removed. According to the CHANGELOG for v0.12.0, this involved removing the thread pool from the future cache (PR #294) and sync caches (PR #316).

• Async conversion: Many previously synchronous housekeeping methods were converted to async to run on the caller's task rather than a dedicated OS thread.

How Lazy Housekeeping Works

Instead of a background thread sweeping periodically, Moka now performs maintenance lazily through the Housekeeper component located in src/common/concurrent/housekeeper.rs.

When you call get, insert, or invalidate, the cache invokes Housekeeper::run_pending_tasks via BaseCache::apply_reads_writes_if_needed. This function runs under a lightweight Mutex guard in src/sync/base_cache.rs (for synchronous caches) or src/future/base_cache.rs (for async caches), processing any pending evictions or expirations immediately on your thread.

This design means housekeeping only occurs when the cache is actually accessed, eliminating idle thread overhead while maintaining consistent performance characteristics.

Architectural Rationale for Removing Background Threads

The shift to moka no background threads provides five key technical advantages:

• Zero-cost background processing: Housekeeping only consumes CPU when a client performs an operation. The cache avoids waking dedicated threads that would otherwise sit idle, bounding latency to the user's own operation timeline.

• Reduced dependency surface: Eliminating the scheduled-thread-pool crate shrinks the dependency tree and simplifies builds, particularly for embedded targets where minimal binary size matters.

• Predictable latency: All eviction and expiration work happens in the same call stack as the user's read or write operation. This prevents unexpected pauses from asynchronous background sweeps that could interfere with application timing.

• Better async runtime compatibility: Converting housekeeping methods to async allows them to be awaited on the user's existing runtime—whether Tokio, async-std, or Actix-rt—without requiring separate OS threads. This aligns with Moka's "futures-aware" design philosophy.

• Lower resource usage: No dedicated threads means reduced memory and CPU overhead, especially beneficial when running multiple cache instances on resource-constrained platforms.

Practical Code Examples

The following examples demonstrate that you never need to spawn or configure a background thread—the cache operates entirely on the thread invoking its API.

Synchronous Cache

use moka::sync::Cache;

// A cache does not spawn any background thread.
let cache = Cache::new(10_000);

// The first `get`/`insert` call will also trigger any pending evictions.
cache.insert(1, "one".to_string());

// Eviction listener runs on the same thread that calls `invalidate`.
cache.set_eviction_listener(|k, v| {
    println!("evicted {k:?} => {v:?}");
});
cache.invalidate(&1); // listener runs here, no extra thread involved.

Asynchronous Cache

use moka::future::Cache;
use tokio::time::{sleep, Duration};

#[tokio::main]
async fn main() {
    // Async cache also has no background thread.
    let cache = Cache::new(100);

    // `insert` returns a future that runs on the Tokio runtime.
    cache.insert("a", 42).await;

    // The housekeeper will be run lazily on the next operation.
    let v = cache.get(&"a").await;
    assert_eq!(v, Some(42));

    // Sleep a bit – nothing runs in the background.
    sleep(Duration::from_secs(1)).await;

    // When we call `invalidate`, any pending eviction work is performed now.
    cache.invalidate(&"a").await;
}

Key Source Files

File	Role
src/common/concurrent/housekeeper.rs	Central house-keeping component; runs pending eviction/expiration tasks on demand.
src/sync/base_cache.rs	Sync cache core that calls Housekeeper::run_pending_tasks during reads/writes.
src/future/base_cache.rs	Async cache core that invokes the housekeeper in async contexts.
src/sync/builder.rs	Cache builder exposing housekeeper_config (no thread pool).
src/future/builder.rs	Async cache builder with housekeeper_config field passed to Housekeeper::new.
README.md (v0.12 section)	Announces the removal of background threads.
CHANGELOG.md (v0.12.0)	Documents the design change and removal of scheduled-thread-pool.

These files demonstrate how Moka achieves a thread-free housekeeping model while preserving high-performance, concurrent cache semantics.

Summary

• Moka removed all background threads in v0.12.0, eliminating the scheduled-thread-pool dependency.
• Housekeeping now runs lazily via the Housekeeper component in src/common/concurrent/housekeeper.rs, triggered by user operations.
• The run_pending_tasks method executes on the caller's thread through BaseCache::apply_reads_writes_if_needed.
• This architecture reduces resource usage, improves async runtime compatibility, and provides predictable latency bounds.
• Builders in src/sync/builder.rs and src/future/builder.rs still expose housekeeper_config for tuning timing parameters without requiring thread management.

Frequently Asked Questions

Does Moka require manual configuration to avoid background threads?

No configuration is required. Since v0.12.0, moka no background threads is the default and only mode of operation. Both sync::Cache and future::Cache automatically use lazy housekeeping through the Housekeeper component without spawning OS threads.

How does lazy housekeeping affect cache performance?

Performance remains high because Housekeeper::run_pending_tasks runs under a lightweight lock only when the cache is accessed. The work is amortized across user operations, avoiding the context-switching overhead of dedicated background threads while ensuring evictions and expirations are processed promptly.

Can I still configure housekeeping behavior in Moka v0.12.0+?

Yes. The cache builders expose a housekeeper_config field (found in src/sync/builder.rs and src/future/builder.rs) that allows you to tweak timing parameters. However, there is no option to re-enable background threads—the housekeeping will always run on-demand on the accessing thread.

What happened to the scheduled-thread-pool dependency?

The scheduled-thread-pool crate was completely removed in v0.12.0 as documented in the CHANGELOG. This dependency elimination reduces compile times and binary size, particularly benefiting applications running on embedded devices or those requiring minimal dependency trees.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}