How the GreptimeDB Client Module Enables External Application Integration

The GreptimeDB client module provides a high-level Rust API that enables external applications to communicate with a GreptimeDB cluster over gRPC and Arrow Flight, handling connection pooling, load balancing, and authentication transparently.

The client module in the greptimeteam/greptimedb repository serves as the primary entry point for external programs written in Rust to interact with a distributed GreptimeDB cluster. It abstracts the complexity of gRPC channel management, Arrow Flight protocol negotiation, and request routing behind a thin, ergonomic API that supports SQL execution, DDL operations, and streaming data ingestion.

Core Client Architecture

At the heart of the module lies the Client struct defined in src/client/src/client.rs. This struct holds a shared ChannelManager (sourced from the common_grpc crate), a list of peer addresses, and a load-balancing strategy. When instantiated, the Client creates the underlying gRPC channels and provides factory methods for building concrete service clients including region, flow, prometheus gateway, and health-check endpoints.

The ChannelManager reuses tonic Channel instances across requests and applies global configuration for compression, TLS settings, and maximum message sizes. This ensures that external applications do not exhaust system resources by creating redundant TCP connections to the same database nodes.

Load Balancing Strategy

For distributing requests across cluster nodes, the client currently implements a Random load balancer located in src/client/src/load_balance.rs. The Inner::get_peer method selects a target node for each request, allowing the Client to spread load across the provided peer list without requiring external load balancers.

Service-Specific Request Wrappers

The module exposes specialized wrappers that translate high-level operations into protocol-specific RPCs:

• RegionRequester (in src/client/src/region.rs) implements the Datanode trait and sends region-level RPCs via gRPC. It handles partition-level data operations directly against storage nodes.
• FlowRequester (in src/client/src/flow.rs) follows a similar pattern to manage flow-node computations and stream processing tasks.
• Database (in src/client/src/database.rs) provides the highest-level API, constructing GreptimeRequest payloads that encapsulate SQL queries, DDL commands, and bulk inserts. It manages Arrow Flight Ticket creation and do_get streaming to return FlightMessage sequences that decode into query results.

Authentication and Request Metadata

When interacting with secured clusters, the Database struct maintains a FlightContext containing authentication headers. Before sending a request, the client injects these credentials into the request metadata, ensuring that every SQL query or DDL operation carries the necessary auth tokens without manual header manipulation by the developer.

Meta-Client Integration

For internal cluster coordination, the module provides NodeClients in src/client/src/client_manager.rs. This component maintains a cache of per-node Client instances used by the meta-client crate. By lazily initializing and reusing these connections, the meta layer can address individual data nodes efficiently without rebuilding gRPC channels for every metadata operation.

Practical Usage Examples

The following example demonstrates how an external application instantiates a client, configures authentication, and executes SQL queries against a GreptimeDB cluster:

use greptimedb::client::{Client, Database};
use greptimedb::api::v1::CreateTableExpr;
use greptimedb::api::v1::auth_header::{AuthScheme, Basic};

// 1. Initialize client with cluster endpoints
let client = Client::with_urls(&["127.0.0.1:4001", "127.0.0.1:4002"]);

// 2. Create a database handle targeting a specific catalog and schema
let mut db = Database::new("greptime", "public", client.clone());

// Configure basic authentication
db.set_auth(AuthScheme::Basic(Basic {
    username: "admin".into(),
    password: "secret".into(),
}));

// 3. Execute SQL and process results
let output = db.sql("SELECT ts, value FROM my_table LIMIT 10").await?;
println!("Rows returned: {}", output.row_count());

// 4. Create tables via the DDL API
let create_expr = CreateTableExpr {
    // table definition fields...
    ..Default::default()
};
let _ = db.create(create_expr).await?;

// 5. Verify cluster health
client.health_check().await?;

The Client::with_urls constructor automatically initializes the ChannelManager with default configuration, while Database::sql internally handles Arrow Flight ticket generation and response stream decoding.

Summary

• The Client struct in src/client/src/client.rs centralizes channel management and service client creation for gRPC and Arrow Flight communication.
• ChannelManager reuses tonic channels and configures transport-level settings including compression and TLS.
• RegionRequester and FlowRequester provide low-level RPC interfaces for storage and flow nodes, respectively.
• The Database wrapper in src/client/src/database.rs offers high-level methods for SQL execution, DDL, and streaming inserts over Arrow Flight.
• NodeClients enables the meta-client layer to maintain persistent connections to individual data nodes without connection churn.
• Built-in health_check and Prometheus gateway methods support operational monitoring of cluster nodes.

Frequently Asked Questions

What transport protocols does the GreptimeDB client module use?

The client module primarily uses gRPC for administrative and metadata operations, and Arrow Flight for high-performance data ingestion and query result streaming. These protocols are implemented via the tonic and Arrow Rust libraries, with the ChannelManager handling connection lifecycle across both transports.

How does the client module handle load balancing?

The module employs a Random load-balancing strategy defined in src/client/src/load_balance.rs. The Inner::get_peer method randomly selects a peer from the configured address list for each request, distributing traffic across the cluster without requiring external load balancers or client-side discovery services.

Can external applications use the client module from languages other than Rust?

While the client module is written in Rust and provides native Rust APIs, external applications in other languages must use language-specific bindings or the standard PostgreSQL/MySQL wire protocols that GreptimeDB also exposes. The Rust client is optimized for high-performance native applications that need direct Arrow Flight integration.

How is authentication configured in the client module?

Authentication is configured through the Database struct's set_auth method, which accepts an AuthScheme enum variant such as Basic authentication. The client stores these credentials in a FlightContext and automatically injects them into the metadata of every outgoing GreptimeRequest, ensuring secure communication without manual header construction.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}