How CoSky Achieves 100K+ QPS High Performance with Redis

CoSky achieves 100K+ QPS by executing atomic server-side Lua scripts, using a reactive non-blocking Redis client, and maintaining local in-memory caches synchronized via Redis Pub/Sub to eliminate network round-trips.

CoSky is an open-source service discovery and configuration platform built on Redis. This article explains how CoSky achieves 100K+ QPS high performance with Redis through three architectural pillars: server-side Lua execution, reactive I/O, and tiered caching with Pub/Sub invalidation.

Server-Side Lua Scripts for Atomic, Single-Step Operations

CoSky implements all mutating operations—service registration, deregistration, heartbeat renewal, and metadata updates—as server-side Lua scripts. These scripts reside in cosky-discovery/src/main/resources/ (e.g., registry_register.lua, registry_deregister.lua, registry_renew.lua).

Why Lua Scripts Eliminate Network Round-Trips

Unlike client-side transactions that require multiple commands and round-trips, Lua scripts execute atomically within the Redis process. A single EVALSHA call can perform complex logic—such as adding an instance to a set, storing hash fields, setting TTLs, and publishing invalidation messages—without returning intermediate results to the client.

In RedisServiceRegistry.kt, the register() method builds the argument list and executes DiscoveryRedisScripts.SCRIPT_REGISTRY_REGISTER, which runs registry_register.lua. The script returns 1 instantly, confirming the operation succeeded with minimal latency.

Registration and Discovery Scripts in Action

The registry_register.lua script performs four atomic operations:

1. Adds the instance ID to a service index set using sadd.
2. Stores instance metadata in a hash via hmset.
3. Sets expiration (TTL) for ephemeral instances.
4. Publishes a "register" event to trigger cache invalidation across nodes.

For discovery, discovery_get_instances.lua reads the service index (smembers), fetches each instance's hash (hgetall), calculates remaining TTLs, and prunes expired entries—all in a single server-side execution.

Reactive, Non-Blocking Redis Client Architecture

CoSky uses Spring's ReactiveStringRedisTemplate backed by the Lettuce driver to achieve fully non-blocking I/O. All public methods in RedisServiceRegistry.kt and RedisConsistencyConfigService.kt return Mono or Flux types, ensuring the calling thread never blocks waiting for Redis.

Eliminating Thread-Per-Request Overhead

Traditional blocking clients allocate one thread per request, limiting throughput to a few thousand QPS before context-switching overhead dominates. CoSky's reactive pipeline uses a single event-loop thread to handle tens of thousands of concurrent connections. When ConsistencyRedisServiceDiscovery.getInstances() is called, it either returns cached data immediately or issues a reactive Redis command that yields the result asynchronously without blocking the web server thread.

Local In-Memory Caching with Pub/Sub Invalidation

To achieve millions of operations per second for read-heavy workloads, CoSky implements a two-tier caching strategy using local ConcurrentHashMap instances synchronized via Redis Pub/Sub.

ConcurrentHashMap for Nanosecond Reads

ConsistencyRedisServiceDiscovery maintains an instanceCache (a ConcurrentHashMap) that stores service instance lists in JVM heap. When getInstances() is invoked, it first checks this cache. If present, the method returns instantly with nanosecond-level latency, avoiding any network I/O to Redis.

Redis Pub/Sub for Cache Coherence

When a write operation modifies service state—such as registration or configuration updates—the executing Lua script publishes an invalidation message to a Redis channel (e.g., publish … "register"). The ReactiveRedisMessageListenerContainer subscribed by RedisServiceEventListenerContainer receives these messages and invokes hooks like hookOnResetInstanceCache to evict the specific entry from the local cache.

This pattern ensures that subsequent reads fetch fresh data from Redis only after a mutation occurs, balancing extreme read performance with strong consistency.

Performance Benchmarks and Real-World Results

CoSky's architecture delivers measurable results in JMH benchmarks. The ConsistencyRedisServiceDiscoveryBenchmark reports over 70 million operations per second for service discovery when leveraging local caching, while ConsistencyRedisConfigServiceBenchmark achieves over 240 million operations per second for configuration reads.

These figures demonstrate that CoSky scales far beyond the 100K QPS target, with the reactive stack and local caching enabling horizontal scalability limited only by Redis Cluster capacity.

Summary

• Server-side Lua scripts execute complex mutations atomically in Redis, eliminating multiple network round-trips.
• Reactive Redis client (Lettuce via Spring's ReactiveStringRedisTemplate) enables non-blocking I/O that handles tens of thousands of concurrent connections per thread.
• Local in-memory caching with Redis Pub/Sub invalidation provides nanosecond read latency while maintaining strong consistency across distributed nodes.

Frequently Asked Questions

How does CoSky handle cache consistency across multiple nodes?

CoSky uses Redis Pub/Sub to broadcast invalidation events. When any node writes data, the Lua script publishes a message to a Redis channel. All other nodes listen via ReactiveRedisMessageListenerContainer and clear the relevant entry from their local ConcurrentHashMap cache, ensuring subsequent reads fetch fresh data.

Why did CoSky choose Lua scripts over Redis transactions?

Lua scripts provide true atomic execution without the complexity of WATCH/MULTI/EXEC transactions. A single Lua script can perform conditional logic, read data, and write updates atomically in one server-side execution, reducing network latency from multiple round-trips to a single EVALSHA call.

What Redis client does CoSky use and why?

CoSky uses the Lettuce driver exposed through Spring's ReactiveStringRedisTemplate. Lettuce supports reactive, non-blocking I/O based on Netty, allowing a single JVM thread to manage thousands of concurrent Redis connections without thread-per-request overhead, which is essential for achieving 100K+ QPS.

Can CoSky scale beyond 100K QPS?

Yes. Benchmarks show CoSky achieves over 70 million ops/s for service discovery and 240 million ops/s for configuration reads when utilizing local caching. The architecture scales horizontally by adding Redis Cluster nodes and application instances, with throughput limited primarily by Redis I/O capacity rather than application code.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}