How PromQL Integration Works Alongside SQL in GreptimeDB: Architecture Deep Dive

GreptimeDB unifies PromQL integration with SQL by parsing both languages into a shared QueryStatement enum, translating PromQL metrics queries into DataFusion logical plans via a specialized PromPlanner, and executing both on the same vectorized engine.

GreptimeDB is an open-source time-series database built on Apache DataFusion that offers native PromQL integration alongside standard SQL. Unlike systems that treat PromQL as an external wrapper, GreptimeDB embeds PromQL directly into its query pipeline, allowing both languages to share the same catalog, optimizer, and execution runtime. This article examines the source code in greptimeteam/greptimedb to explain how the database achieves seamless dual-language support through a unified logical planning layer.

The Unified Query Statement Architecture

At the entry point of every query, GreptimeDB uses the QueryLanguageParser (src/query/src/parser.rs) to normalize both SQL and PromQL into a single internal representation. This design eliminates the need for separate query engines and ensures consistent handling of authentication, resource limits, and session context.

The QueryStatement Enum

The parser returns a QueryStatement enum that abstracts the language differences:

• QueryStatement::Sql(Statement) – Contains a DataFusion AST for standard SQL queries
• QueryStatement::Promql(EvalStmt, Option<String>) – Contains a PromQL AST (promql_parser::parser::EvalStmt) for metric queries

This unification happens in QueryLanguageParser::parse_sql and QueryLanguageParser::parse_promql, which HTTP/gRPC handlers call when receiving requests at endpoints like /v1/promql (src/servers/src/http/prometheus.rs).

Logical Planning for Dual Languages

Once parsed, both statement types flow into the DfLogicalPlanner (src/query/src/planner.rs). The planner inspects the QueryStatement variant and routes PromQL queries to a specialized translator while handling SQL through the standard DataFusion path.

Planning Pipeline Branching

Inside DfLogicalPlanner::plan, a match statement distinguishes the execution paths:

• SQL Path: Calls DfLogicalPlanner::plan_sql (lines 287–299) to build logical plans using the standard DataFusion planner
• PromQL Path: Delegates to PromPlanner::stmt_to_plan (src/query/src/promql/planner.rs) for metric-specific translation

Both paths ultimately produce DataFusion LogicalPlan objects, ensuring that optimization rules and physical planning work identically regardless of the input language.

PromPlanner Implementation Details

The PromPlanner (src/query/src/promql/planner.rs) implements the core translation logic from PromQL AST nodes to DataFusion relational operators. Key steps include:

1. Context Creation: PromPlannerContext::from_eval_stmt extracts start/end timestamps and step intervals from the PromQL query (lines 65–71)
2. Metric Resolution: Resolves the metric name from the __name__ label matcher using catalog::table_source::DfTableSourceProvider
3. Table Scan Generation: Produces TableScan nodes that reference the underlying metric tables

Translating PromQL to DataFusion Operators

The PromQL-to-DataFusion translation handles metric-specific semantics through specialized expression mapping and custom logical extensions.

Label Matcher Conversion

PromQL label matchers (Matcher, Matchers) translate into DataFusion filter expressions via PromPlanner::matchers_to_expr (lines 4031–4045). The conversion produces standard DataFusion expressions such as:

• col(label).eq(literal) for equality matchers
• regexp_match(col(label), pattern) for regex matchers

These expressions become standard Filter nodes in the logical plan, allowing DataFusion's predicate push-down optimizations to apply to PromQL queries.

PromQL-Specific Functions

Functions unique to PromQL—such as rate, histogram_quantile, absent, and resets—map to custom logical plan extensions in the promql::extension_plan module (src/query/src/promql/extension_plan/*.rs). These include:

• SeriesDivide – Handles time-series alignment and step boundaries
• HistogramFold – Processes histogram bucket calculations
• InstantManipulate – Manages lookback windows and timestamp alignment

These extensions compile into the same physical operators as standard SQL functions, enabling cross-language optimization.

Hybrid TQL: Embedding PromQL Inside SQL

GreptimeDB supports TQL (Time-series Query Language), a hybrid syntax that embeds PromQL expressions within SQL WITH clauses. This allows analysts to combine PromQL's metric aggregation with SQL's joining and windowing capabilities.

TQL Parsing and Planning

The hybrid parser in src/sql/src/parsers/with_tql_parser.rs extracts PromQL snippets from TQL EVAL statements. For example:

WITH t AS (TQL EVAL (0, 600, '10s') sum(rate(cpu_usage[1m])))
SELECT service, avg(val) FROM t GROUP BY service

The parser extracts the PromQL part and processes it through QueryLanguageParser::parse_promql, while the surrounding SQL flows through the standard parser. The DfLogicalPlanner::plan_query_with_hybrid_ctes method (lines 350–384) coordinates planning by:

1. Detecting hybrid CTEs via has_hybrid_ctes
2. Routing PromQL sub-queries through PromPlanner
3. Assembling the final logical plan with SQL CTE references pointing to PromQL-derived sub-plans

Shared Execution Runtime

After logical planning, both SQL and PromQL queries converge on the same execution path. The DataFusion physical planner generates vectorized execution plans that run on a shared thread pool. This means:

• Memory management uses identical allocation pools for both languages
• Push-down optimizations apply to filter predicates regardless of whether they originated in SQL WHERE clauses or PromQL label matchers
• Caching layers store results from both query types using the same eviction policies

The execution uniformity ensures that hybrid queries—where SQL joins the results of PromQL aggregations—operate without data serialization barriers or context switches between engines.

Summary

GreptimeDB implements PromQL integration through a layered architecture that preserves language-specific semantics while sharing underlying infrastructure:

• Unified parsing via QueryLanguageParser produces a QueryStatement enum supporting both SQL and PromQL variants
• Specialized planning through PromPlanner translates PromQL AST nodes into standard DataFusion logical operators
• Metric-specific extensions in promql::extension_plan implement functions like rate and histogram_quantile as compatible logical nodes
• Hybrid TQL support enables PromQL embedding within SQL CTEs via with_tql_parser.rs
• Shared execution on the DataFusion engine provides consistent performance characteristics across both query languages

Frequently Asked Questions

Can I mix PromQL and SQL in the same query?

Yes. GreptimeDB supports TQL (Time-series Query Language), which allows you to embed PromQL expressions inside SQL WITH clauses using the TQL EVAL syntax. The hybrid parser extracts the PromQL portion, processes it through PromPlanner, and integrates the resulting sub-plan with the surrounding SQL query, enabling joins and aggregations across both languages.

How does GreptimeDB handle PromQL functions like rate or histogram_quantile?

These functions map to custom logical plan extensions in the promql::extension_plan module (src/query/src/promql/extension_plan/*.rs). The PromPlanner translates PromQL function calls into specialized operators such as SeriesDivide and HistogramFold, which DataFusion treats as standard logical nodes during optimization and physical planning.

What execution engine runs PromQL queries?

PromQL queries execute on the same DataFusion vectorized engine that processes SQL. After PromPlanner converts PromQL AST into a LogicalPlan, the standard DataFusion physical planner takes over. This shared runtime enables consistent memory management, predicate push-down, and caching across both query languages.

Where is the PromQL parsing logic located in the codebase?

The entry point for PromQL parsing is QueryLanguageParser::parse_promql in src/query/src/parser.rs. This method parses the PromQL string into an EvalStmt from the promql_parser crate, wrapping it in the QueryStatement::Promql enum variant for downstream processing by the logical planner.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}