Understanding the Data View Feature in INFINI Console for Elasticsearch Time-Series Analysis

The Data View feature (also called index-pattern in the UI) is a logical configuration that groups Elasticsearch indices, designates a time-filter field for time-series queries, and maps custom formatters to specific fields so raw values display as human-readable strings like "0.5 GB" instead of bytes.

The data view feature sits at the heart of the INFINI Console (infinilabs/console) open-source project, acting as the bridge between raw Elasticsearch indices and user-friendly visualizations. By storing view definitions as documents in a system Elasticsearch cluster, the console enables consistent, reusable configurations for time-series exploration across any index pattern.

What Is the Data View Feature?

A Data View is a persisted JSON document that defines which Elasticsearch indices belong together and how the console should interpret their fields. Stored in the system cluster using the template at config/setup/common/view.tpl, each view contains:

• title: The index pattern (e.g., logs-* or metrics-2024.*)
• viewName: A human-readable identifier
• timeFieldName: The field used for time-range filtering (e.g., @timestamp)
• fields: An array of field metadata
• fieldFormatMap: A mapping of field names to formatter configurations

According to the source code in config/setup/common/view.tpl (lines 5-9), the template explicitly defines timeFieldName and fieldFormatMap as core properties when a view is indexed via PUT $[[SETUP_INDEX_PREFIX]]view/.

How Data Views Support Time-Series Field Formatting

Time-series data in Elasticsearch often contains raw metrics—bytes, milliseconds, or ratios—that are unreadable without formatting. The data view feature solves this through two complementary mechanisms.

Defining the Time Filter Field

The timeFieldName property designates which field serves as the global time filter. When a user selects a time range in the UI, visualization components retrieve the data view, extract timeFieldName, and construct Elasticsearch range queries targeting that specific field. This ensures all panels in a dashboard respect the same temporal boundary.

Mapping Field Formatters

The fieldFormatMap is a JSON object where each key is a fully-qualified field name and each value specifies an id (formatter type) and optional params. For example:

{
  "system.memory.used.bytes": {
    "id": "bytes",
    "params": {}
  },
  "cpu.usage": {
    "id": "percent",
    "params": {"pattern": "0.00%"}
  }
}

When rendering charts or tables, the console looks up each metric field in fieldFormatMap and invokes the corresponding formatter implementation from web/src/utils/format.js.

Technical Implementation in the Codebase

The data view feature spans the backend storage layer, REST API, and frontend formatting utilities.

Data View Storage Template

The structural definition resides in config/setup/common/view.tpl. This template is used during setup to create the mapping and initial documents in the system index. Lines 5-9 establish the schema for timeFieldName and fieldFormatMap, ensuring these fields are indexed and searchable when views are persisted.

API Layer for Data Retrieval

The HTTP handlers in modules/elastic/api/view.go serve data view definitions to the frontend. The functions HandleGetViewListAction and HandleGetViewAction (lines 337-339) extract the fieldFormatMap from the stored document and return it verbatim in the JSON response:

// Simplified representation of the API response structure
response := map[string]interface{}{
    "view": map[string]interface{}{
        "title":          view.Title,
        "timeFieldName":  view.TimeFieldName,
        "fieldFormatMap": view.FieldFormatMap, // Returned as-is for frontend use
    },
}

This ensures the UI receives the exact formatter configuration needed to render values correctly.

Formatter Utilities

The actual transformation logic lives in web/src/utils/format.js. The formatter.bytes function (lines 9-21) converts raw byte integers into human-readable strings:

// From web/src/utils/format.js
export const formatter = {
  bytes: (value) => {
    if (value === null || value === undefined) return '-';
    const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
    if (value === 0) return '0 Bytes';
    const i = parseInt(Math.floor(Math.log(value) / Math.log(1024)));
    return Math.round(value / Math.pow(1024, i), 2) + ' ' + sizes[i];
  }
};

When a visualization component detects that a field has id: "bytes" in the fieldFormatMap, it routes the raw Elasticsearch value through this function before rendering.

End-to-End Workflow

The data view feature operates through a five-stage pipeline:

1. Creation: The user submits a POST request to the view API with title, timeFieldName, and optional fieldFormatMap. The backend uses config/setup/common/view.tpl to index this document into the system cluster.

2. Storage: The view persists in the system index with the mapping defined in the template, ensuring timeFieldName and fieldFormatMap are preserved as structured JSON.

3. Retrieval: When loading a dashboard or discovery screen, the frontend calls HandleGetViewAction (lines 337-339 in modules/elastic/api/view.go), which returns the complete view definition including the formatter map.

4. Query Construction: Visualization components extract timeFieldName to build Elasticsearch range queries that respect the global time picker, ensuring all panels query the same temporal window.

5. Rendering: As results arrive, the UI checks fieldFormatMap for each metric field. If a formatter is defined (e.g., id: "bytes"), the raw value passes through the corresponding function in web/src/utils/format.js before display.

Summary

• The data view feature (index-pattern) is a persisted configuration that groups Elasticsearch indices and defines time-series behavior.
• It stores timeFieldName to enable global time filtering across all visualizations.
• The fieldFormatMap property maps specific fields to formatter implementations (bytes, percent, number) for human-readable display.
• View definitions reside in the system cluster using the schema defined in config/setup/common/view.tpl.
• The API layer in modules/elastic/api/view.go serves these definitions to the frontend, which applies formatters from web/src/utils/format.js at render time.

Frequently Asked Questions

What is the difference between a data view and an index pattern in INFINI Console?

Data view and index pattern refer to the same feature. The codebase and API use "data view" as the canonical term, while the UI and locale files (such as web/src/locales/en-US/explore.js) often display "index-pattern" to users familiar with Kibana terminology. Both terms describe the logical grouping of indices and their associated formatting rules.

How does the console know which formatter to apply to a specific field?

When rendering results, the console looks up the field name in the fieldFormatMap stored within the data view definition. If an entry exists, the id property (e.g., "bytes", "number", or "percent") determines which function to invoke from web/src/utils/format.js. The raw Elasticsearch value is then passed through that formatter function before being displayed in charts or tables.

Can I use wildcards when defining a data view title?

Yes. The title property in a data view supports wildcard patterns such as logs-* or metrics-2024.*. This allows a single data view to match multiple indices that share a common naming convention. The UI explicitly mentions this capability in the Create View dialog, as documented in the locale file web/src/locales/en-US/explore.js, enabling users to aggregate data across time-series indices without creating separate views for each index.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}