How to Set Up a Cache Cluster Using Consistent Hashing in libCacheSim

libCacheSim provides a complete consistent hashing implementation in example/cacheCluster that uses a virtual node ring with MD5-based hashing (ketama_hash) to distribute requests across multiple cache servers, supporting dynamic topology changes and weighted load distribution.

The 1a1a11a/libcachesim repository includes a production-ready cache cluster example demonstrating how to scale beyond single-node simulations. By implementing a consistent hashing ring with virtual nodes, the library enables horizontal scaling of cache servers while minimizing key reshuffling during topology changes. This guide walks through the architectural layers, key APIs, and implementation steps required to build a multi-node cache cluster using the consistent hashing module.

Understanding the Consistent Hashing Architecture

The consistent hashing implementation in libCacheSim follows the Ketama algorithm design, mapping both cache objects and servers onto a 32-bit hash ring. The system uses virtual nodes (vnodes) to ensure uniform distribution, where each physical server is represented by multiple points on the ring proportional to its weight.

When a request arrives, the object ID is hashed to a point on the ring, and the system walks clockwise to find the first virtual node. The corresponding server_id determines which physical cache server handles the request. This approach guarantees that adding or removing a server only affects the keys immediately adjacent to that server's points on the ring, leaving the majority of mappings intact.

Core Components of the Cache Cluster

The implementation splits responsibilities across three distinct layers to maintain separation between hashing logic and cache operations.

Consistent Hash Ring Layer

Located in example/cacheCluster/include/consistentHash.h and example/cacheCluster/consistentHash.c, this layer defines the ring_t and vnode_t structures and implements all ring operations. The ch_ring_create_ring() function builds the hash ring from a normalized weight vector, while ch_ring_get_server_from_uint64() performs O(log n) lookups to locate the responsible server for a given 64-bit object ID.

Cache Cluster Abstraction

The CacheCluster class (defined in example/cacheCluster/include/cacheCluster.hpp and implemented in example/cacheCluster/cacheCluster.cpp) serves as the orchestration layer. It maintains a vector of CacheServer objects and owns the consistent hash ring instance. The get(request_t* req) method hashes the request's obj_id, dispatches to the appropriate server, and returns the hit/miss status.

Cache Server Implementation

Each physical node is represented by a CacheServer object (example/cacheCluster/include/cacheServer.hpp), which contains one or more libCacheSim Cache objects. This design allows individual servers to run different eviction algorithms or cache sizes while presenting a unified interface through the get() method.

Implementation Guide

Building the Example

Compile the cache cluster example using CMake and Ninja after installing the base libCacheSim library:

mkdir -p example/cacheCluster/_build
cd example/cacheCluster/_build
cmake -G Ninja ..
ninja

Initializing the Cluster

Create a CacheCluster instance and populate it with weighted servers. Each server requires a configured libCacheSim cache instance:

#include "include/cacheCluster.hpp"
#include "libCacheSim/cache.h"

using namespace CDNSimulator;

// Create cluster instance
CacheCluster cluster;

// Add three servers with equal weights
for (int i = 0; i < 3; ++i) {
    // Create LRU cache with 1 MiB capacity
    Cache cache = Cache_create(1 << 20, "LRU", 0);
    
    CacheServer server(i);
    server.add_cache(std::move(cache));
    
    // Weight of 1.0 (normalized automatically in ch_ring_create_ring)
    cluster.add_server(std::move(server), 1.0);
}

The add_server() method internally calls ch_ring_create_ring() to rebuild the hash ring with the updated server topology and normalized weight vector.

Handling Requests

Route incoming requests through the cluster's get() method, which uses ch_ring_get_server_from_uint64() (lines 46-50 in consistentHash.c) to determine the target server:

request_t req;
req.obj_id = 0xdeadbeef12345678ULL;
req.size = 4096;

bool hit = cluster.get(&req);

The cluster hashes the 64-bit obj_id to a 32-bit point on the ring, locates the first virtual node with a point greater than or equal to the hash, and forwards the request to that server's CacheServer::get() implementation.

Replication and Fault Tolerance

Request multiple distinct servers for replication using ch_ring_get_servers() or skip unavailable nodes with ch_ring_get_available_servers() (lines 150-178 in consistentHash.c):

// Get 2 distinct replicas for key "myKey"
unsigned int replicas[2];
ch_ring_get_servers("myKey", cluster._ring, 2, replicas);

// Simulate server failure
cluster.fail_one_server(1);  // Marks server 1 as unavailable

// Recover server
cluster.recover_one_server(1);

When a server is marked unavailable via fail_one_server(), subsequent requests automatically map to the next available server on the ring, maintaining service continuity without manual reconfiguration.

Dynamic Topology Changes

Remove servers at runtime to trigger automatic ring reconstruction:

cluster.remove_server(2);  // Removes server index 2 and rebuilds ring

The removal destroys the old ring and creates a new one with the remaining servers, ensuring deterministic mapping while minimizing key redistribution.

Summary

• Consistent hashing in libCacheSim uses an MD5-based virtual node ring (ketama_hash) to distribute requests across multiple cache servers with O(log n) lookup complexity.
• The architecture separates concerns into three layers: the hash ring library (consistentHash.c), the cluster coordinator (cacheCluster.cpp), and individual server instances (cacheServer.hpp).
• Server weights are normalized automatically to allocate virtual nodes proportionally, enabling heterogeneous cluster configurations.
• Fault tolerance is built-in through availability bitmaps and automatic failover to the next server on the ring when nodes fail.
• The implementation supports dynamic scaling, allowing runtime addition and removal of servers with minimal key reshuffling.

Frequently Asked Questions

What hashing algorithm does libCacheSim use for consistent hashing?

libCacheSim implements the Ketama consistent hashing algorithm using MD5-based hashing (ketama_hash) to map virtual nodes to points on a 32-bit ring. This approach is implemented in example/cacheCluster/consistentHash.c and provides uniform distribution of keys across servers while maintaining stability during topology changes.

How does libCacheSim handle server failures in a cache cluster?

The CacheCluster class provides fail_one_server() and recover_one_server() methods that toggle a server's availability flag. When a server is unavailable, the consistent hash ring automatically skips that node during lookup, routing requests to the next available server on the ring. This behavior is handled transparently in ch_ring_get_available_servers() without requiring a full ring rebuild.

Can I use different cache eviction algorithms for different servers?

Yes. Each CacheServer instance maintains its own vector of libCacheSim Cache objects, configured independently during initialization. You can mix LRU, LFU, FIFO, or other supported eviction algorithms across different servers in the same cluster by passing different algorithm strings to Cache_create() when building each server.

How do server weights affect request distribution?

Server weights determine the number of virtual nodes allocated to each physical server on the consistent hash ring. In ch_ring_create_ring(), the weight vector is normalized, and each server receives a proportional number of vnodes. Higher-weighted servers occupy more points on the ring, consequently receiving a larger percentage of requests. Weights can be set via the second parameter of CacheCluster::add_server().

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}