How the Dashboard and Visualization System Works in INFINI Console's Insight Module

The Insight module stores dashboards and visualizations as Elasticsearch documents, linking them via ID arrays and resolving relationships server-side using terms queries when serving API requests.

INFINI Console is an open-source management platform for Elasticsearch clusters. Its Insight module provides a lightweight, extensible framework for building custom dashboards and visualizations. This article examines the architecture, data models, and API implementation that power the dashboard and visualization system in INFINI Console's Insight module, based on the source code in the infinilabs/console repository.

Architecture Overview

The Insight dashboard system consists of four primary components that interact through a generic ORM layer backed by Elasticsearch:

Component	Responsibility	Key Source File
Dashboard model	Stores metadata (title, description, tags) and an array of visualization IDs	model/insight/dashboard.go
Visualization model	Defines chart configuration (type, series, position) referencing metric definitions	model/insight/visualization.go
Widget model	Generic UI container for arbitrary configuration objects	model/insight/widget.go
API Controllers	CRUD endpoints with automatic visualization resolution via terms queries	plugin/api/insight/dashboard.go

Core Data Models

Dashboard Model

The Dashboard struct in model/insight/dashboard.go (lines 32-48) serves as the top-level container. It stores an array of visualization IDs rather than embedded documents:

type Dashboard struct {
    ID          string   `json:"id" elastic_meta:"_id"`
    Title       string   `json:"title" elastic_mapping:"title:{type:keyword}"`
    Description string   `json:"description" elastic_mapping:"description:{type:text}"`
    Visualizations []string `json:"visualizations" elastic_mapping:"visualizations:{type:keyword}"`
    // ... additional metadata fields
}

This design enables visualization reuse across multiple dashboards while maintaining loose coupling.

Visualization Model

The Visualization struct in model/insight/visualization.go (lines 32-44) defines individual charts. Each visualization contains a Series array where each SeriesItem references a metric definition from the core insight package:

type Visualization struct {
    ID          string       `json:"id" elastic_meta:"_id"`
    Title       string       `json:"title"`
    Type        string       `json:"type"` // e.g., "line", "bar", "pie"
    Series      []SeriesItem `json:"series"`
    Position    GridPosition `json:"position"`
}

type SeriesItem struct {
    Type    string      `json:"type"`
    Metric  MetricMeta  `json:"metric"` // References core/insight metric definitions
    Options interface{} `json:"options"`
}

Widget Model

The Widget model in model/insight/widget.go (lines 30-36) provides a generic container for UI elements that don't fit the standard visualization pattern:

type Widget struct {
    ID     string          `json:"id" elastic_meta:"_id"`
    Title  string          `json:"title"`
    Config json.RawMessage `json:"config"` // Arbitrary JSON configuration
}

This enables extensibility for custom panels such as markdown blocks or alert summaries.

Data Flow and API Implementation

Creating Visualizations

Clients create visualizations by POSTing to /insight/visualization. The createVisualization handler in plugin/api/insight/visualization.go (lines 43-61) unmarshals the payload and persists it via the ORM:

func (h *APIHandler) createVisualization(w http.ResponseWriter, req *http.Request, ps httprouter.Params) {
    var obj insight.Visualization
    // ... validation ...
    err := orm.Create(&obj)
    // ... response handling ...
}

The ORM stores the document in the visualization index with mappings defined by the struct tags.

Creating Dashboards with Visualization References

Dashboard creation follows a similar pattern at /insight/dashboard. The critical difference is that the visualizations field contains an array of string IDs rather than full objects:

POST /insight/dashboard HTTP/1.1
Content-Type: application/json

{
  "title": "Cluster Overview",
  "cluster_id": "cluster-1",
  "visualizations": ["vis-5678", "vis-9012"],
  "tags": ["ops", "overview"]
}

The createDashboard handler in plugin/api/insight/dashboard.go (lines 40-58) stores this in the dashboard index.

Loading Dashboards with Embedded Visualizations

The most complex operation occurs when retrieving a dashboard. The getDashboard handler (lines 76-92 in dashboard.go) implements a server-side join pattern:

1. Fetch the dashboard document via orm.Get
2. Extract the visualizations ID array
3. Execute a terms query to fetch all referenced visualizations in a single batch:

// Build terms query for visualization IDs
q := orm.Query{Size: len(obj.Visualizations)}
q.Conds = orm.And(orm.Eq("id", obj.Visualizations...))

// Execute search
result, err := orm.Search(insight2.Visualization{}, &q)
if err != nil {
    // handle error
}

// Inject full objects into response
obj.Visualizations = result.Result

This approach minimizes network round-trips while maintaining the flexibility of separate indices.

Searching and Filtering

The search endpoint /insight/dashboard/_search supports full-text queries and term filters. The handler (lines 85-124 in dashboard.go) constructs boolean queries combining query_string for text search and term filters for fields like cluster_id:

GET /insight/dashboard/_search?keyword=overview&cluster_id=cluster-1&size=10&from=0

The raw Elasticsearch response is returned directly to the client (h.Write(w, res.Raw)), preserving aggregations and highlighting.

Persistence and ORM Integration

The Insight module leverages the console's generic ORM layer to abstract Elasticsearch operations. Registration occurs in main.go (lines 152-155):

orm.RegisterSchemaWithIndexName(insight.Visualization{}, "visualization")
orm.RegisterSchemaWithIndexName(insight.Dashboard{}, "dashboard")

This registration:

• Creates dedicated indices (visualization and dashboard) if they don't exist
• Applies mappings derived from struct tags (elastic_mapping and elastic_meta)
• Enables automatic _id handling and optimistic concurrency control

The elastic_mapping tags define field types explicitly, such as keyword for IDs and tags, and text for descriptions, ensuring efficient filtering and full-text search.

Security and Permissions

Access to dashboard operations is governed by permission constants defined in core/security/enum/const.go (lines 86-87):

DashboardRead = "insight.dashboard:read"
DashboardAll  = "insight.dashboard:all"

The API handlers rely on the audit_log.MonitoringInterceptor middleware to enforce these permissions. DashboardRead permits GET operations, while DashboardAll grants full CRUD capabilities. This model allows fine-grained access control, enabling read-only dashboards for operators and full management rights for administrators.

Extensibility Features

The Insight module supports extension through two primary mechanisms:

Series Definition Flexibility Each Visualization contains a Series array where SeriesItem.Type determines the chart rendering method (line, bar, pie, etc.). The SeriesItem.Metric field references metric definitions from core/insight/metric.go. Adding new chart types requires only extending the Type enumeration and implementing client-side rendering logic, without modifying the storage layer.

Widget Configuration The Widget model stores arbitrary JSON configurations via the Config field (json.RawMessage). This enables custom UI panels—such as markdown documentation, alert summaries, or external iframe embeds—without requiring database schema migrations or code changes to the backend.

Summary

• Dashboard and visualization documents are stored as separate entities in dedicated Elasticsearch indices (dashboard and visualization), linked by ID references rather than embedded documents.
• Server-side joins use Elasticsearch terms queries to resolve visualization arrays when loading dashboards, minimizing API round-trips while maintaining data normalization.
• Generic ORM integration handles index creation, mapping generation, and CRUD operations, with models registered in main.go using orm.RegisterSchemaWithIndexName.
• Permission controls enforce read-only or full access via insight.dashboard:read and insight.dashboard:all constants.
• Extensible architecture supports new chart types through the SeriesItem structure and custom widgets via JSON configuration fields.

Frequently Asked Questions

How are dashboard and visualization relationships stored in INFINI Console?

Dashboards store an array of visualization IDs in the visualizations field (defined in model/insight/dashboard.go), while complete visualization documents reside in a separate index. When retrieving a dashboard, the API controller executes a terms query against the visualization index to fetch all referenced documents in a single batch, then injects the full objects into the response.

What database backend powers the Insight module's dashboard system?

The system uses Elasticsearch as the primary data store. The console's generic ORM layer (accessed via orm.Create, orm.Get, orm.Search) persists dashboards and visualizations as JSON documents in dedicated indices named dashboard and visualization. Index mappings are auto-generated from struct tags (elastic_mapping) defined in the model files.

How does the API efficiently load related visualizations when fetching a dashboard?

The getDashboard handler in plugin/api/insight/dashboard.go (lines 76-92) implements an efficient loading pattern: after retrieving the dashboard document, it extracts the visualization ID array and constructs an Elasticsearch terms query ("terms": {"id": [...]}). This query retrieves all referenced visualizations in a single search operation, minimizing network overhead compared to individual document lookups.

What permissions are required to manage dashboards in INFINI Console?

Access is controlled by constants defined in core/security/enum/const.go. The permission insight.dashboard:read grants view-only access to dashboard endpoints, while insight.dashboard:all provides full CRUD capabilities. The API enforces these permissions through the audit_log.MonitoringInterceptor middleware, which validates the current user's privileges before executing handler logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}