How GreptimeDB's Distributed Architecture Distributes Data Across Frontend, Datanode, and Metasrv Components

GreptimeDB distributes time-series data by storing actual columnar data in datanode regions while using metasrv to manage region routing metadata, with frontends acting as query routers that locate region leaders via the partition manager and forward requests directly to the appropriate datanodes.

GreptimeDB's distributed deployment model separates concerns across three core components to achieve horizontal scalability and high availability. In the greptimeteam/greptimedb repository, the architecture delegates storage to stateful datanodes, coordination to a metadata service cluster, and query routing to stateless frontend instances.

How Data Flows Through GreptimeDB's Three-Tier Architecture

Metasrv: The Metadata Authority

The metasrv component maintains the single source of truth for all cluster metadata, including table schemas, region definitions, and node topology. It stores region routes—mappings that associate each region number with its current leader peer—in a distributed KV backend.

When a datanode starts, it registers itself with metasrv and begins sending periodic heartbeats. The metasrv tracks lease expiration using intervals defined in src/meta-srv/src/metasrv.rs, where the frontend_from function configures heartbeat timing:

// src/meta-srv/src/metasrv.rs
pub fn frontend_from(base_interval: Duration) -> Self {
    Self {
        interval: frontend_heartbeat_interval(base_interval),
        retry_interval: base_interval,
    }
}

If a region leader fails to report within the lease window, metasrv triggers fail-over logic to elect a new leader from available replicas, ensuring continuous data availability without manual intervention.

Datanode: The Storage Engine

Datanodes are the workhorses that persist actual time-series data. Each datanode manages one or more regions (shards), with each region representing a distinct partition of a table's data. Within a region group, one datanode acts as the leader handling read and write operations, while others serve as replicas for redundancy.

The storage interface in src/datanode/src/store.rs exposes region-level operations:

// src/datanode/src/store.rs
pub async fn write(&self, region_id: RegionId, data: Vec<u8>) -> Result<(), BoxedError> {
    // Persists data to the region's columnar files, updates WAL
    Ok(())
}

pub async fn read(&self, region_id: RegionId) -> Result<Vec<u8>, BoxedError> {
    // Reads from the region's storage files
    Ok(vec![])
}

Datanodes report their health and region status through the heartbeat mechanism implemented in src/datanode/src/heartbeat.rs, allowing metasrv to maintain an accurate view of cluster topology.

Frontend: The Query Router

Frontends are stateless query gateways that accept SQL, PromQL, and gRPC requests from clients. Rather than storing data, they act as intelligent routers that parse incoming queries, determine which regions contain the required data, and forward requests to the appropriate datanode leaders.

The routing process begins when the frontend extracts a region_id from the query plan and invokes the PartitionRuleManager to resolve the leader location. This resolution chain demonstrates how GreptimeDB's distributed architecture separates metadata lookup from data retrieval.

Code Deep Dive: Region Resolution and Request Routing

When a query arrives at the frontend, the system executes a precise sequence of metadata lookups and network calls to reach the correct data. Here is the actual implementation flow from the source code:

Step 1: Frontend Locates the Region Leader

In src/frontend/src/instance/region_query.rs, the frontend handles incoming region queries by first identifying the responsible peer:

// src/frontend/src/instance/region_query.rs
let region_id = request.region_id;
let peer = self
    .partition_manager
    .find_region_leader(region_id)
    .await
    .context(FindRegionPeerSnafu { region_id, read_preference })?;

let client = self.node_manager.datanode(peer).await;
client.handle_query(request).await.context(RequestQuerySnafu)

The find_region_leader call initiates a lookup against the partition manager's cached view of the cluster metadata.

Step 2: Partition Manager Queries the Router

The PartitionRuleManager in src/partition/src/manager.rs maintains a read-locked view of region routes and delegates to the router module to extract the leader:

// src/partition/src/manager.rs
pub async fn find_region_leader(&self, region_id: RegionId) -> Result<Peer> {
    let region_number = region_id.region_number();
    let region_routes = self.routes.read().await;
    router::find_region_leader(region_routes, region_number)
        .context(error::FindRegionLeaderSnafu { region_id })
}

This method transforms the region identifier into a region number and queries the routing table without blocking on network calls, as the routes are cached locally and updated asynchronously via metasrv watches.

Step 3: Router Maps Region to Leader Peer

The core lookup logic resides in src/common/meta/src/rpc/router.rs, where the router searches a HashMap<RegionNumber, RegionRoute> to find the leader peer:

// src/common/meta/src/rpc/router.rs
pub fn find_region_leader(
    routes: &HashMap<RegionNumber, RegionRoute>,
    region_number: RegionNumber,
) -> Result<&Peer> {
    Router::find_region_leader(routes, region_number)
        .context(error::RegionNotFoundSnafu { region_number })
}

The RegionRoute struct contains an optional leader field referencing a Peer (containing datanode address and identifier), enabling the frontend to establish a gRPC connection to the exact node holding the data.

Step 4: Datanode Executes the Request

Once the frontend establishes a connection to the leader datanode, the datanode executes the operation against its local storage engine. The DistTable abstraction in src/table/src/dist_table.rs coordinates these distributed operations, delegating to the RegionQueryHandler for execution across multiple regions when necessary.

Summary

• Metasrv stores metadata only, maintaining region routes and leader information in a distributed KV store while orchestrating fail-over through heartbeat lease management.
• Datanodes store actual data in columnar format within regions, with exactly one leader per region group handling write and consistent read operations.
• Frontends remain stateless and route queries by consulting the PartitionRuleManager to find region leaders, then forwarding requests directly to the responsible datanodes via the NodeManager.
• Region resolution follows a strict hierarchy: frontend → PartitionRuleManager → Router → HashMap lookup, ensuring minimal latency for metadata discovery.
• Heartbeats and leases drive the autonomous fail-over mechanism, allowing the cluster to recover from datanode failures without manual reconfiguration.

Frequently Asked Questions

How does GreptimeDB handle fail-over when a datanode crashes?

When a datanode stops sending heartbeats to metasrv, its region leader leases expire based on the intervals defined in src/meta-srv/src/metasrv.rs. Metasrv detects the expired lease and triggers the region migration manager to elect a new leader from the surviving replicas. Clients experience brief retry periods while the frontend refreshes its partition manager cache to discover the new leader location.

Can frontends cache region routing information to improve performance?

Yes, frontends maintain a local cache of region routes through the PartitionRuleManager's routes field, which holds a RwLock<HashMap<RegionNumber, RegionRoute>>. This cache updates asynchronously when metasrv publishes route changes, allowing frontend nodes to resolve region leaders from local memory without querying the metadata service on every request.

What determines how data is partitioned across datanodes?

GreptimeDB uses partition rules defined at table creation time to distribute rows across regions based on primary key ranges or hash values. Metasrv stores these partition definitions and assigns each region to specific datanodes. When data grows, administrators can trigger region splits, after which metasrv updates the region routes and the partition manager propagates the new topology to frontends.

Is the frontend required for all client interactions, or can clients connect directly to datanodes?

While clients can theoretically connect directly to datanodes, production deployments route all traffic through frontends because only frontends possess the PartitionRuleManager logic necessary to resolve which datanode holds specific data. Direct datanode connections bypass the routing layer and require clients to manually handle region mapping, making frontends essential for distributed query execution.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}