How INFINI Console Handles Index Template Management and Dynamic Cluster Registration

INFINI Console uses fasttemplate-rendered .tpl files for index template management and exposes a POST /instance endpoint for dynamic cluster registration, coupling both systems through a global Elastic client registry.

The infinilabs/console repository implements a decoupled architecture for managing Elasticsearch and OpenSearch infrastructure. Index template management operates as a setup-time process that renders DSL templates and pushes them to target clusters, while dynamic cluster registration allows administrators to add new endpoints on-the-fly via REST API calls.

Index Template Management Architecture

Template Initialization Flow

When the Console system cluster starts, main/main.go invokes elastic2.InitTemplate(false) after verifying system-cluster health. This call triggers the template initialization sequence defined in plugin/setup/setup.go.

The InitTemplate() function sets a guard flag cfg1.InitTemplate to prevent duplicate initialization on subsequent runs. If the flag is already set, the process skips rendering and pushing templates, ensuring idempotent startup behavior.

Template Rendering with fasttemplate

The initializeTemplate handler in plugin/setup/setup.go processes SetupRequest payloads containing a template_type field (e.g., template_ilm, rollup, alerting). Based on this type, the system selects the appropriate .tpl file from config/setup/.

Template rendering uses github.com/valyala/fasttemplate to inject runtime values. Tags like $[[SETUP_INDEX_PREFIX]], $[[SETUP_CLUSTER_ID]], and credential variables are replaced with actual cluster configuration. The rendered output produces valid Elasticsearch DSL for index templates, ILM policies, or rollup configurations.

Template Storage and DSL Structure

Template files reside in config/setup/ with distribution-specific subdirectories:

• ILM Templates: config/setup/opensearch/template_ilm.tpl (with v5 and v6 variants for legacy versions)
• Rollup Templates: config/setup/easysearch/template_rollup.tpl for Easysearch distributions
• Common Templates: config/setup/common/ contains alerting.tpl, insight.tpl, view.tpl, and agent.tpl for alert history, visualization data, UI views, and agent ingest pipelines

The system also renders config/system_config.tpl at startup to generate the system-cluster configuration document consumed by the client factory.

Dynamic Cluster Registration Mechanism

The POST /instance Endpoint

Dynamic cluster registration is handled by the managed-server component in plugin/managed/server/instance.go. The endpoint POST /instance accepts JSON payloads mapping to the model.Instance struct.

The handler registerInstance decodes the request body, populating fields for name, endpoint, authentication credentials, and TLS configuration. This endpoint serves both agent-driven registration and UI-based "Add Cluster" workflows.

Validation and Persistence

Before persistence, the handler validates that the endpoint field is non-empty. It uses uri.Parse to extract host and scheme components, automatically populating missing fields when only a URL is provided.

The validated instance is persisted via orm.Save(nil, obj), which stores the model.Instance document in the internal Elasticsearch-backed datastore. If the instance already exists, the Created timestamp is preserved to maintain historical accuracy.

Global Client Registry Integration

After successful persistence, the new cluster becomes available through the global Elastic client registry. The client factory in infini.sh/framework/core/elastic (referenced via elastic.GetClient) reads persisted instance documents to construct ready-to-use elastic.API clients.

This integration enables any Console plugin—whether index-template initialization, alerting, or insight pipelines—to target the newly registered cluster transparently. The tryConnect function in instance.go performs health checks using client.ClusterHealth to verify reachability before marking the cluster as active.

Integrating Templates with Registered Clusters

Once a cluster is registered, the setup module can push index templates to it on demand. The /setup/init API accepts a SetupRequest with InitializeTemplate set to the desired template type (e.g., template_ilm) and a Cluster object containing connection details.

The initializeTemplate handler renders the appropriate .tpl file using runtime values from the cluster configuration, then pushes the rendered DSL via PUT /_template/<name> using the low-level Elastic client. This decoupled approach allows administrators to register clusters first and provision templates later, or to re-provision templates when upgrading Console versions.

Code Examples

Registering a Cluster via API

curl -X POST http://console.mycompany.com/instance \
     -H "Content-Type: application/json" \
     -d '{
       "name": "prod-es-01",
       "endpoint": "https://es-prod.example.com:9200",
       "basic_auth": {
         "username": "admin",
         "password": "s3cr3t"
       },
       "tls": true,
       "hosts": ["es-prod-01.example.com:9200", "es-prod-02.example.com:9200"]
     }'

The response contains the generated instance ID:

{
  "result": "created",
  "_id": "c7b3e1a5-4d8f-48e6-9e2a-f7a5c2d6b9e0"
}

Initializing Templates via Setup API

curl -X POST http://localhost:8080/setup/init \
     -H "Content-Type: application/json" \
     -d '{
       "InitializeTemplate": "template_ilm",
       "Cluster": {
         "Username": "admin",
         "Password": "s3cr3t",
         "Schema": "https",
         "Endpoint": "https://es-prod.example.com:9200",
         "Hosts": ["es-prod-01.example.com:9200", "es-prod-02.example.com:9200"]
       },
       "BootstrapUsername": "admin"
     }'

This triggers the initializeTemplate handler in plugin/setup/setup.go to render the ILM template and push it to the specified cluster.

Managing Templates via REST

Retrieve an existing template:

curl -X GET "http://localhost:8080/elasticsearch/<cluster-id>/_template/metrics-rollover"

Save or update a template:

curl -X PUT "http://localhost:8080/elasticsearch/<cluster-id>/_template/metrics-rollover" \
     -H "Content-Type: application/json" \
     -d @custom_template.json

These endpoints are implemented in modules/elastic/api/template.go via HandleGetTemplateAction and HandleSaveTemplateAction.

Summary

• Index template management in INFINI Console operates at setup time using fasttemplate to render .tpl files from config/setup/ into valid Elasticsearch DSL, which is then pushed to target clusters via PUT /_template/<name>.

• Dynamic cluster registration exposes a POST /instance endpoint that persists model.Instance documents, validates endpoints, and integrates new clusters into the global Elastic client registry via elastic.GetClient.

• Both systems are decoupled: clusters can be registered at any time, and template initialization can be triggered later via the /setup/init API, allowing flexible provisioning of ILM policies, rollup jobs, and alerting indices across heterogeneous Elasticsearch and OpenSearch distributions.

Frequently Asked Questions

How does INFINI Console prevent duplicate template initialization?

The system uses a guard flag cfg1.InitTemplate set inside the InitTemplate() function in plugin/setup/setup.go. When main/main.go invokes elastic2.InitTemplate(false) at startup, the function checks this flag and skips rendering if templates were already initialized. Administrators can force re-initialization by passing true to the function or by calling the /setup/init API directly.

What template types does INFINI Console support?

INFINI Console supports multiple template categories stored under config/setup/: ILM templates (template_ilm.tpl) for index lifecycle management, rollup templates (template_rollup.tpl) for Easysearch distributions, and common templates for alerting (alerting.tpl), insight (insight.tpl), view storage (view.tpl), and agent ingest pipelines (agent.tpl). Distribution-specific variants exist for OpenSearch, Elasticsearch v5, and v6.

How does dynamic cluster registration handle authentication?

The POST /instance handler in plugin/managed/server/instance.go accepts a JSON payload containing optional basic_auth fields (username and password) and TLS configuration flags. These credentials are stored in the model.Instance document persisted via orm.Save. When subsequent operations request a client for that cluster ID, the global Elastic client factory reads these credentials from the persisted document to establish authenticated connections.

Can templates be pushed to existing clusters without restarting?

Yes. While main/main.go automatically initializes templates for the system cluster at startup, administrators can push templates to any registered cluster at runtime using the /setup/init API. By sending a SetupRequest with InitializeTemplate set to the desired type (e.g., template_ilm) and including the target cluster configuration, the initializeTemplate handler in plugin/setup/setup.go renders the template and executes a PUT /_template/<name> request against the specified cluster without requiring a Console restart.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}