How r-nacos Achieves 80K QPS for Config Queries and 17.6K TPS for Config Writes

r-nacos achieves 80,000 queries per second (QPS) for configuration reads and 17,600 transactions per second (TPS) for writes through a lock-free in-memory cache, Actix actor model concurrency, and Raft-based asynchronous persistence that eliminates disk I/O from the hot path.

The nacos-group/r-nacos project is a high-performance Rust implementation of the Nacos configuration and service discovery platform. Its architecture eliminates traditional database bottlenecks by keeping all configuration data in memory and using message-passing concurrency to achieve horizontal scalability. This article examines the specific technical implementations in the source code that enable these extreme throughput figures.

The Zero-Latency Read Path (80K QPS)

All configuration queries hit a hot in-memory cache before any other subsystem is involved. In src/config/core.rs, the ConfigActor maintains a standard HashMap<ConfigKey, ConfigValue> that serves as the primary data store for reads.

When an HTTP or gRPC request arrives, the Actix web server (src/main.rs#L53-L66) routes it to the InvokerHandler, which sends a ConfigCmd::GET(key) message to the ConfigActor. The actor performs a single hash lookup and returns the ConfigValue immediately. Because the data never leaves RAM, there is no network round-trip or disk seek latency.

The implementation uses Arc<String> for both keys and values, ensuring that cloning operations only increment reference counts rather than copying byte data. This zero-copy approach allows the system to handle tens of thousands of concurrent read requests on modest hardware without garbage collection pauses or memory pressure.

The Asynchronous Write Path (17.6K TPS)

Write operations maintain durability through Raft consensus without blocking the request thread on disk I/O. When a client publishes a configuration change, the InvokerHandler generates a ConfigAsyncCmd::Add message that the ConfigActor converts into a ClientWriteRequest containing ClientRequest::ConfigSet (see src/config/core.rs#L332-L340).

The Raft leader appends this entry to its log and replicates it to followers asynchronously. The leader acknowledges the write to the client once the entry is committed to the log, not after it is flushed to disk on all nodes. This decoupling allows the system to batch multiple writes and perform heavy I/O operations in the background via the SnapshotWriterActor (src/config/core.rs#L54-L66), which periodically captures the entire state to bound log growth.

After Raft commits the log entry, the ConfigActor applies the change to its local HashMap cache and notifies subscribers. This design yields high TPS while preserving strong consistency across the cluster.

Memory-Efficient Data Structures

r-nacos minimizes allocation overhead through careful data structure selection. The ConfigKey::build_key method (src/config/core.rs#L63-L68) creates a compact binary representation using a delimited string format (data_id\x02group\x02tenant) that serves as the direct lookup key. This avoids the overhead of nested structs or serialization during hash map operations.

By storing configurations as simple key-value pairs in a HashMap guarded by a single-threaded actor, the system eliminates lock contention entirely. The actor model ensures that only one thread accesses the map at a time, making reads and writes lock-free from the perspective of client code.

Actor Model Concurrency with Actix and Tokio

The system partitions functionality into independent Actix actors—ConfigActor, NamingActor, RaftActor, and MetricsManager—each running on the Tokio async runtime. Message passing between actors replaces traditional mutex-based synchronization, preventing contention points that typically throttle high-throughput services.

In src/main.rs, the bootstrap code registers these actors with the Actix system, allowing the scheduler to multiplex thousands of lightweight tasks across a small pool of OS threads. This architecture scales vertically on multi-core machines without the complexity of manual thread management.

Raft Optimization and Snapshotting

The Raft implementation in src/raft/store/mod.rs uses asynchronous log replication and periodic snapshots to maintain performance under write pressure. The SnapshotWriterActor (src/config/core.rs#L54-L66) handles the expensive work of serializing the full configuration state to disk, ensuring that the hot path remains unblocked by I/O operations.

By batching log entries and applying them to the state machine in chunks, r-nacos amortizes the cost of consensus across many write operations. Production deployments can tune Raft heartbeat intervals and thread counts via AppSysConfig (src/config/mod.rs#L112-L118) to optimize for specific hardware configurations.

Minimal-Overhead Metrics Collection

Performance monitoring adds negligible latency through the CounterManager implementation in src/metrics/counter.rs#L14-L21. Each request triggers a MetricsRequest::Record(item) message that executes a simple hash map lookup and atomic integer increment:

self.counter_manager.increment(item.metrics_type, v);

Because metrics collection happens via the same actor-based message passing as business logic, it never blocks the request handling path. The MetricsManager (src/metrics/core.rs) aggregates these counters asynchronously and exports them at configurable intervals without impacting the 80K QPS read performance.

Load Testing and Benchmarking

The repository includes a dedicated loadtest crate that verifies the advertised performance figures. The benchmark binaries in loadtest/src/bin/http_config_query.rs simulate thousands of concurrent clients using ratelimiter_rs::QpsLimiter to throttle requests and measure sustainable throughput.

use ratelimiter_rs::QpsLimiter;
use std::sync::Arc;

#[derive(Clone)]
struct ConfigQuery {
    limiter: QpsLimiter,
    client: Arc<ConfigClient>,
}

impl ConfigQuery {
    async fn run(&self) {
        self.limiter.acquire().await;
        let _ = self.client.get("my-data", "DEFAULT").await;
    }
}

These tests demonstrate that a single r-nacos node can sustain 80,000 configuration queries per second while maintaining sub-millisecond latency, validating the efficiency of the in-memory cache design.

Summary

• Lock-free in-memory cache: Configuration data resides in a HashMap<ConfigKey, ConfigValue> inside ConfigActor, eliminating disk I/O from the read path entirely.
• Actor model architecture: Actix actors with Tokio async runtime eliminate mutex contention by using message passing for all cross-thread communication.
• Asynchronous Raft replication: Writes are durably logged via Raft but acknowledged immediately upon commit, with disk persistence handled asynchronously by SnapshotWriterActor.
• Zero-copy data structures: Arc<String> and compact binary key formats minimize memory allocation and cloning overhead.
• Lightweight metrics: CounterManager::increment uses atomic operations on a hash map, adding negligible overhead to the hot path.
• Configurable tuning: Parameters like metrics_collect_interval_second and Raft timeouts can be adjusted via AppSysConfig for specific deployment scenarios.

Frequently Asked Questions

How does r-nacos handle persistence without slowing down writes?

r-nacos uses Raft consensus to replicate writes across nodes before acknowledging the client, but persists data to disk asynchronously. The SnapshotWriterActor (src/config/core.rs#L54-L66) handles periodic snapshots and log compaction in the background, ensuring that the request thread never blocks on file system operations. This allows the system to achieve 17.6K TPS while maintaining strong consistency guarantees.

What hardware is required to achieve 80K QPS for configuration queries?

The 80K QPS figure is achievable on modest commodity hardware with sufficient RAM to hold the configuration dataset entirely in memory. Since reads are served from a lock-free HashMap without disk access, performance depends primarily on CPU speed for hashing and network bandwidth for HTTP/gRPC serialization. The Actix/Tokio runtime efficiently utilizes all available CPU cores through work-stealing schedulers.

How does the actor model improve performance over traditional mutex-based concurrency?

Traditional mutexes create contention points where threads block waiting for locks, limiting scalability under high concurrency. r-nacos uses Actix actors where each subsystem (config, naming, metrics) runs as an independent event loop processing messages sequentially. This design ensures that the ConfigActor's HashMap is accessed by only one thread at a time without explicit locking, eliminating cache coherence traffic and context switches associated with mutex contention.

Can r-nacos scale horizontally while maintaining these performance metrics?

Yes. Because the Raft implementation handles consensus across nodes, you can deploy a cluster of r-nacos instances to distribute read load (via follower reads or client-side caching) and write load (via leader election and log replication). The state machine remains consistent through the Raft log, and the SnapshotWriterActor ensures that new nodes can recover quickly from snapshots without replaying the entire log history.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}