# How the Dashboard and Visualization System Works in INFINI Console's Insight Module

> Discover how INFINI Console's Insight module powers dashboards and visualizations, storing data in Elasticsearch and resolving relationships server-side for efficient API requests.

- Repository: [INFINI Labs/console](https://github.com/infinilabs/console)
- Tags: internals
- Published: 2026-03-04

---

**The Insight module stores dashboards and visualizations as Elasticsearch documents, linking them via ID arrays and resolving relationships server-side using terms queries when serving API requests.**

INFINI Console is an open-source management platform for Elasticsearch clusters. Its Insight module provides a lightweight, extensible framework for building custom dashboards and visualizations. This article examines the architecture, data models, and API implementation that power the dashboard and visualization system in INFINI Console's Insight module, based on the source code in the `infinilabs/console` repository.

## Architecture Overview

The Insight dashboard system consists of four primary components that interact through a generic ORM layer backed by Elasticsearch:

| Component | Responsibility | Key Source File |
|-----------|----------------|-----------------|
| **Dashboard model** | Stores metadata (title, description, tags) and an array of visualization IDs | [`model/insight/dashboard.go`](https://github.com/infinilabs/console/blob/main/model/insight/dashboard.go) |
| **Visualization model** | Defines chart configuration (type, series, position) referencing metric definitions | [`model/insight/visualization.go`](https://github.com/infinilabs/console/blob/main/model/insight/visualization.go) |
| **Widget model** | Generic UI container for arbitrary configuration objects | [`model/insight/widget.go`](https://github.com/infinilabs/console/blob/main/model/insight/widget.go) |
| **API Controllers** | CRUD endpoints with automatic visualization resolution via terms queries | [`plugin/api/insight/dashboard.go`](https://github.com/infinilabs/console/blob/main/plugin/api/insight/dashboard.go) |

## Core Data Models

### Dashboard Model

The `Dashboard` struct in [`model/insight/dashboard.go`](https://github.com/infinilabs/console/blob/main/model/insight/dashboard.go) (lines 32-48) serves as the top-level container. It stores an array of visualization IDs rather than embedded documents:

```go
type Dashboard struct {
    ID          string   `json:"id" elastic_meta:"_id"`
    Title       string   `json:"title" elastic_mapping:"title:{type:keyword}"`
    Description string   `json:"description" elastic_mapping:"description:{type:text}"`
    Visualizations []string `json:"visualizations" elastic_mapping:"visualizations:{type:keyword}"`
    // ... additional metadata fields
}

```

This design enables visualization reuse across multiple dashboards while maintaining loose coupling.

### Visualization Model

The `Visualization` struct in [`model/insight/visualization.go`](https://github.com/infinilabs/console/blob/main/model/insight/visualization.go) (lines 32-44) defines individual charts. Each visualization contains a `Series` array where each `SeriesItem` references a metric definition from the core insight package:

```go
type Visualization struct {
    ID          string       `json:"id" elastic_meta:"_id"`
    Title       string       `json:"title"`
    Type        string       `json:"type"` // e.g., "line", "bar", "pie"
    Series      []SeriesItem `json:"series"`
    Position    GridPosition `json:"position"`
}

type SeriesItem struct {
    Type    string      `json:"type"`
    Metric  MetricMeta  `json:"metric"` // References core/insight metric definitions
    Options interface{} `json:"options"`
}

```

### Widget Model

The `Widget` model in [`model/insight/widget.go`](https://github.com/infinilabs/console/blob/main/model/insight/widget.go) (lines 30-36) provides a generic container for UI elements that don't fit the standard visualization pattern:

```go
type Widget struct {
    ID     string          `json:"id" elastic_meta:"_id"`
    Title  string          `json:"title"`
    Config json.RawMessage `json:"config"` // Arbitrary JSON configuration
}

```

This enables extensibility for custom panels such as markdown blocks or alert summaries.

## Data Flow and API Implementation

### Creating Visualizations

Clients create visualizations by POSTing to `/insight/visualization`. The `createVisualization` handler in [`plugin/api/insight/visualization.go`](https://github.com/infinilabs/console/blob/main/plugin/api/insight/visualization.go) (lines 43-61) unmarshals the payload and persists it via the ORM:

```go
func (h *APIHandler) createVisualization(w http.ResponseWriter, req *http.Request, ps httprouter.Params) {
    var obj insight.Visualization
    // ... validation ...
    err := orm.Create(&obj)
    // ... response handling ...
}

```

The ORM stores the document in the `visualization` index with mappings defined by the struct tags.

### Creating Dashboards with Visualization References

Dashboard creation follows a similar pattern at `/insight/dashboard`. The critical difference is that the `visualizations` field contains an array of string IDs rather than full objects:

```http
POST /insight/dashboard HTTP/1.1
Content-Type: application/json

{
  "title": "Cluster Overview",
  "cluster_id": "cluster-1",
  "visualizations": ["vis-5678", "vis-9012"],
  "tags": ["ops", "overview"]
}

```

The `createDashboard` handler in [`plugin/api/insight/dashboard.go`](https://github.com/infinilabs/console/blob/main/plugin/api/insight/dashboard.go) (lines 40-58) stores this in the `dashboard` index.

### Loading Dashboards with Embedded Visualizations

The most complex operation occurs when retrieving a dashboard. The `getDashboard` handler (lines 76-92 in [`dashboard.go`](https://github.com/infinilabs/console/blob/main/dashboard.go)) implements a server-side join pattern:

1. Fetch the dashboard document via `orm.Get`
2. Extract the `visualizations` ID array
3. Execute a **terms query** to fetch all referenced visualizations in a single batch:

```go
// Build terms query for visualization IDs
q := orm.Query{Size: len(obj.Visualizations)}
q.Conds = orm.And(orm.Eq("id", obj.Visualizations...))

// Execute search
result, err := orm.Search(insight2.Visualization{}, &q)
if err != nil {
    // handle error
}

// Inject full objects into response
obj.Visualizations = result.Result

```

This approach minimizes network round-trips while maintaining the flexibility of separate indices.

### Searching and Filtering

The search endpoint `/insight/dashboard/_search` supports full-text queries and term filters. The handler (lines 85-124 in [`dashboard.go`](https://github.com/infinilabs/console/blob/main/dashboard.go)) constructs boolean queries combining `query_string` for text search and `term` filters for fields like `cluster_id`:

```http
GET /insight/dashboard/_search?keyword=overview&cluster_id=cluster-1&size=10&from=0

```

The raw Elasticsearch response is returned directly to the client (`h.Write(w, res.Raw)`), preserving aggregations and highlighting.

## Persistence and ORM Integration

The Insight module leverages the console's generic ORM layer to abstract Elasticsearch operations. Registration occurs in [`main.go`](https://github.com/infinilabs/console/blob/main/main.go) (lines 152-155):

```go
orm.RegisterSchemaWithIndexName(insight.Visualization{}, "visualization")
orm.RegisterSchemaWithIndexName(insight.Dashboard{}, "dashboard")

```

This registration:
- Creates dedicated indices (`visualization` and `dashboard`) if they don't exist
- Applies mappings derived from struct tags (`elastic_mapping` and `elastic_meta`)
- Enables automatic `_id` handling and optimistic concurrency control

The `elastic_mapping` tags define field types explicitly, such as `keyword` for IDs and tags, and `text` for descriptions, ensuring efficient filtering and full-text search.

## Security and Permissions

Access to dashboard operations is governed by permission constants defined in [`core/security/enum/const.go`](https://github.com/infinilabs/console/blob/main/core/security/enum/const.go) (lines 86-87):

```go
DashboardRead = "insight.dashboard:read"
DashboardAll  = "insight.dashboard:all"

```

The API handlers rely on the `audit_log.MonitoringInterceptor` middleware to enforce these permissions. `DashboardRead` permits GET operations, while `DashboardAll` grants full CRUD capabilities. This model allows fine-grained access control, enabling read-only dashboards for operators and full management rights for administrators.

## Extensibility Features

The Insight module supports extension through two primary mechanisms:

**Series Definition Flexibility**
Each `Visualization` contains a `Series` array where `SeriesItem.Type` determines the chart rendering method (line, bar, pie, etc.). The `SeriesItem.Metric` field references metric definitions from [`core/insight/metric.go`](https://github.com/infinilabs/console/blob/main/core/insight/metric.go). Adding new chart types requires only extending the `Type` enumeration and implementing client-side rendering logic, without modifying the storage layer.

**Widget Configuration**
The `Widget` model stores arbitrary JSON configurations via the `Config` field (`json.RawMessage`). This enables custom UI panels—such as markdown documentation, alert summaries, or external iframe embeds—without requiring database schema migrations or code changes to the backend.

## Summary

- **Dashboard and visualization documents** are stored as separate entities in dedicated Elasticsearch indices (`dashboard` and `visualization`), linked by ID references rather than embedded documents.
- **Server-side joins** use Elasticsearch terms queries to resolve visualization arrays when loading dashboards, minimizing API round-trips while maintaining data normalization.
- **Generic ORM integration** handles index creation, mapping generation, and CRUD operations, with models registered in [`main.go`](https://github.com/infinilabs/console/blob/main/main.go) using `orm.RegisterSchemaWithIndexName`.
- **Permission controls** enforce read-only or full access via `insight.dashboard:read` and `insight.dashboard:all` constants.
- **Extensible architecture** supports new chart types through the `SeriesItem` structure and custom widgets via JSON configuration fields.

## Frequently Asked Questions

### How are dashboard and visualization relationships stored in INFINI Console?

Dashboards store an array of visualization IDs in the `visualizations` field (defined in [`model/insight/dashboard.go`](https://github.com/infinilabs/console/blob/main/model/insight/dashboard.go)), while complete visualization documents reside in a separate index. When retrieving a dashboard, the API controller executes a **terms query** against the visualization index to fetch all referenced documents in a single batch, then injects the full objects into the response.

### What database backend powers the Insight module's dashboard system?

The system uses **Elasticsearch** as the primary data store. The console's generic ORM layer (accessed via `orm.Create`, `orm.Get`, `orm.Search`) persists dashboards and visualizations as JSON documents in dedicated indices named `dashboard` and `visualization`. Index mappings are auto-generated from struct tags (`elastic_mapping`) defined in the model files.

### How does the API efficiently load related visualizations when fetching a dashboard?

The `getDashboard` handler in [`plugin/api/insight/dashboard.go`](https://github.com/infinilabs/console/blob/main/plugin/api/insight/dashboard.go) (lines 76-92) implements an efficient loading pattern: after retrieving the dashboard document, it extracts the visualization ID array and constructs an Elasticsearch **terms query** (`"terms": {"id": [...]}`). This query retrieves all referenced visualizations in a single search operation, minimizing network overhead compared to individual document lookups.

### What permissions are required to manage dashboards in INFINI Console?

Access is controlled by constants defined in [`core/security/enum/const.go`](https://github.com/infinilabs/console/blob/main/core/security/enum/const.go). The permission `insight.dashboard:read` grants view-only access to dashboard endpoints, while `insight.dashboard:all` provides full CRUD capabilities. The API enforces these permissions through the `audit_log.MonitoringInterceptor` middleware, which validates the current user's privileges before executing handler logic.