How the Metric Engine Works for Observability Data in GreptimeDB

The Metric Engine is a specialized storage layer that multiplexes many logical metric tables onto a single physical wide table, enabling high-cardinality observability workloads while reusing GreptimeDB's proven Mito storage backbone.

The Metric Engine serves as the primary interface for ingesting and querying time-series metrics in GreptimeDB (greptimeteam/greptimedb). Unlike traditional table-per-metric approaches, this engine implements a thin wrapper architecture that translates logical table operations into efficient physical storage patterns, specifically optimized for Prometheus-style metrics and other observability data sources.

Core Architecture and Concepts

The Metric Engine introduces a logical-to-physical mapping abstraction that separates user-visible table schemas from the underlying storage layout.

Logical Regions and Physical Region Groups

A Logical Region represents a user-visible metric table created with ENGINE = metric. Rather than allocating dedicated storage files, logical regions share a Physical Region Group consisting of two underlying Mito regions:

• Data Region: Stores the actual time-series rows in a wide table format
• Metadata Region: Maintains schema definitions and logical-to-physical mappings as key-value pairs (e.g., keys formatted as __table_<table_id>)

This multiplexing approach is implemented in src/metric-engine/src/engine.rs, where the MetricEngine::create_regions function handles the translation of CREATE TABLE statements into logical region registrations against existing physical tables.

Internal Columns and TSID

Every physical metric table contains two reserved internal columns defined in src/metric-engine/src/data_region.rs:

• __tsid: A 64-bit hash of the tag set (labels) that identifies unique time series
• __table_id: The identifier linking rows to their logical table origin

These columns enable efficient grouping and filtering during query execution while maintaining logical isolation between multiplexed tables.

Data Flow and Processing

The Metric Engine intercepts storage requests and transforms them to match the physical schema before delegating to the underlying Mito engine.

Write Path and Row Modification

When processing INSERT operations, MetricEngine::handle_request in src/metric-engine/src/engine.rs delegates to MetricEngineInner::put_region. The write flow follows these steps:

1. Row Transformation: RowModifier::modify_row (in src/metric-engine/src/row_modifier.rs) computes the tsid by hashing tag values and rewrites column IDs to match the physical schema
2. Internal Column Injection: The modifier appends __tsid and __table_id values to each row
3. Mito Delegation: The transformed request is forwarded to the DataRegion (src/metric-engine/src/data_region.rs), which issues the actual RegionRequest::Put to the Mito engine

This transformation ensures that logically separate tables can coexist in the same physical wide table without collision.

Read Path and Column Projection

For SELECT queries, the engine resolves the physical region ID via the metadata region, then delegates the scan to Mito. During result processing:

1. The scan returns rows containing internal columns (__tsid, __table_id)
2. The engine strips these columns before returning results to the client
3. Column projection ensures only requested logical columns are materialized

The handle_query implementation in src/metric-engine/src/engine.rs manages this delegation and projection logic.

Schema Evolution

The Metric Engine supports limited schema alterations, specifically column addition. When ALTER TABLE ADD COLUMN executes:

1. MetricEngine::alter_regions validates the request
2. DataRegion::add_columns (in src/metric-engine/src/data_region.rs) generates a RegionAlterRequest
3. The request adds columns to the physical schema with optional inverted or skipping indexes
4. Metadata region entries update the logical table schema mappings

This approach maintains consistency across all logical tables sharing the physical region.

Region ID Management and Utilities

The Metric Engine implements a region ID grouping scheme to address multiple physical regions through a single logical identifier. Utility functions in src/metric-engine/src/utils.rs handle the bit-manipulation:

/// Convert a logical region id to its data region id.
pub fn to_data_region_id(logical_id: RegionId) -> RegionId {
    // High-order bits encode the group; low-order bits differentiate members
}

/// Convert to metadata region id.
pub fn to_metadata_region_id(logical_id: RegionId) -> RegionId { 
    // Bitmask operations to derive metadata region address
}

This grouping allows the engine to treat the data and metadata regions as a unified storage unit while maintaining separate physical storage structures.

Optimizations for Observability Workloads

The Metric Engine implements several optimizations specifically targeting high-cardinality metric ingestion:

• Tag Hashing (tsid): Reduces high-cardinality label dimensions to a 64-bit identifier, enabling efficient time-series grouping and aggregation without full tag comparisons
• Wide Table Physical Layout: Minimizes file I/O overhead by storing multiple logical tables in a single physical structure, reducing the file descriptor and compaction overhead that would occur with table-per-metric designs
• Inverted and Skipping Indexes: Optional per-column indexes (specified via INDEX clauses during table creation) accelerate tag-based filtering by leveraging Mito's existing index infrastructure

These features enable the engine to process millions of time-series rows per second while maintaining full SQL query capabilities, including SELECT, GROUP BY, and COUNT operations.

Implementation Examples

The following Rust code demonstrates end-to-end metric engine usage, adapted from the integration tests in tests-integration/tests/sql.rs:

// 1️⃣ Create a physical metric table (the shared wide table)
let sql = r#"
CREATE TABLE phy (
    ts TIMESTAMP TIME INDEX,
    val DOUBLE
) ENGINE = metric WITH ("physical_metric_table" = "");
"#;
client.execute(sql).await?;

// 2️⃣ Create two logical metric tables that map onto `phy`
let create_logical = |name: &str, pk: &str| async move {
    let sql = format!(
        "CREATE TABLE {name} (
            ts TIMESTAMP TIME INDEX,
            val DOUBLE,
            {pk} STRING PRIMARY KEY
        ) ENGINE = metric WITH (\"on_physical_table\" = \"phy\");"
    );
    client.execute(&sql).await
};

create_logical("t1", "host").await?;
create_logical("t2", "job").await?;

// 3️⃣ Insert rows into a logical table – engine hashes tags into `__tsid`
let insert = r#"
INSERT INTO t1 (ts, val, host) VALUES
    ('2024-01-01 00:00:00', 1.23, 'host_a'),
    ('2024-01-01 00:01:00', 3.45, 'host_b');
"#;
client.execute(insert).await?;

// 4️⃣ Query – the engine strips internal columns before returning
let rows = client
    .query("SELECT host, val FROM t1 ORDER BY ts")
    .await?
    .collect::<Vec<_>>()
    .await;
assert_eq!(rows.len(), 2);

These SQL operations work identically via GreptimeDB's HTTP and MySQL-compatible endpoints, as validated in tests-integration/tests/http.rs.

Summary

• The Metric Engine multiplexes many logical metric tables onto a single physical wide table, reducing storage overhead while maintaining logical isolation.
• Logical regions map to physical region groups (data + metadata regions) via the Mito engine, with internal columns __tsid and __table_id enabling time-series identification.
• Write operations transform rows through RowModifier::modify_row to compute tag hashes and adjust column IDs before delegating to the underlying data region.
• Read operations resolve physical region IDs via the metadata region, execute scans through Mito, and strip internal columns before returning results.
• Schema evolution supports column addition through DataRegion::add_columns, maintaining consistency across all logical tables sharing the physical region.

Frequently Asked Questions

How does the Metric Engine handle high-cardinality time series?

The Metric Engine addresses high-cardinality workloads by hashing tag sets into a 64-bit __tsid identifier during ingestion. This transformation occurs in RowModifier::modify_row within src/metric-engine/src/row_modifier.rs, reducing complex label dimensions to a single integer that enables efficient grouping and aggregation without expensive string comparisons.

What is the difference between a logical region and a physical region in the Metric Engine?

A logical region represents a user-visible metric table created with ENGINE = metric, while a physical region group consists of two underlying Mito regions: a data region storing actual rows and a metadata region storing schema mappings. The logical region has no dedicated storage files; instead, it shares the physical wide table with other logical tables, using __table_id to isolate rows as implemented in src/metric-engine/src/data_region.rs.

Can I alter the schema of an existing metric table?

The Metric Engine supports limited schema alterations, specifically column addition. When executing ALTER TABLE ADD COLUMN, the MetricEngine::alter_regions method delegates to DataRegion::add_columns in src/metric-engine/src/data_region.rs, which generates a RegionAlterRequest to modify the physical schema. This operation affects all logical tables sharing the physical region, maintaining consistency across the multiplexed storage layout.

How does the Metric Engine optimize query performance for observability data?

The engine implements several optimizations targeting metric workloads: tag hashing (tsid) reduces high-cardinality dimensions to 64-bit identifiers for efficient aggregation; inverted and skipping indexes specified during table creation accelerate tag-based filtering; and the wide table physical layout minimizes file I/O overhead by storing multiple logical tables in a single physical structure. These features, combined with Mito's compaction and indexing infrastructure, enable ingestion of millions of rows per second while supporting full SQL analytics.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}