# Understanding the Agent Module for Remote Elasticsearch Cluster Management in Infini Console

> Discover how the agent module in Infini Console securely bridges your connection for remote Elasticsearch cluster management. Monitor and manage nodes without direct access.

- Repository: [INFINI Labs/console](https://github.com/infinilabs/console)
- Tags: deep-dive
- Published: 2026-03-04

---

**The agent module acts as a secure HTTP bridge that enables the Infini Console to discover, monitor, and manage remote Elasticsearch nodes without requiring direct network access to each cluster instance.**

The infinilabs/console repository includes a sophisticated **agent module** that simplifies remote Elasticsearch cluster management by deploying lightweight agents on each host. This module eliminates the need for SSH access or direct Elasticsearch API exposure, instead tunneling all administrative traffic through authenticated Console-to-Agent channels.

## How the Agent Module Enables Remote Cluster Discovery

The foundation of remote management lies in the **agent configuration model** and the enrollment system that links console instances to physical Elasticsearch nodes.

### Agent Configuration and Enrollment Model

In [`modules/agent/model/config.go`](https://github.com/infinilabs/console/blob/main/modules/agent/model/config.go), the console defines the global agent settings through `model.AgentConfig`. This configuration specifies whether the agent feature is enabled, the download URL for the Infini Agent binary, TLS certificates, and the HTTP endpoint details used for communication.

When an operator enrolls a node, the console generates a unique **agent ID** and stores it as metadata within the node settings. This creates a persistent binding between the console's logical view and the physical host running the agent.

### Node Binding via GetEnrolledNodesByAgent

The function `GetEnrolledNodesByAgent` in [`modules/agent/api/elasticsearch.go`](https://github.com/infinilabs/console/blob/main/modules/agent/api/elasticsearch.go) (lines 54-78) queries the console's setting store for all `node_settings` records where `metadata.labels.agent_id` matches a specific instance ID. It returns a map of `BindingItem` structs containing the cluster ID, node UUID, and enrollment status, enabling the console to determine which nodes belong to a particular agent instance.

## Proxy Architecture for Secure Remote Access

All interactions with remote nodes route through a centralized proxy layer that handles authentication, TLS termination, and connection pooling.

### The ProxyAgentRequest Transport Layer

The `server.ProxyAgentRequest` function, implemented in [`plugin/managed/server/websocket_proxy.go`](https://github.com/infinilabs/console/blob/main/plugin/managed/server/websocket_proxy.go), serves as the transport layer for all remote-agent calls. This proxy forwards HTTP requests from the console to the agent's REST endpoint (typically listening on a configurable port like `9000`), handling the underlying WebSocket or HTTP connectivity without exposing the Elasticsearch cluster directly to the console's network.

### Real-time Node Discovery via GetElasticsearchNodesViaAgent

To retrieve live cluster topology, the console invokes `GetElasticsearchNodesViaAgent` in [`modules/agent/api/elasticsearch.go`](https://github.com/infinilabs/console/blob/main/modules/agent/api/elasticsearch.go) (lines 6-21). This function constructs a `util.Request` targeting the agent's `/elasticsearch/node/_discovery` endpoint and proxies it through `server.ProxyAgentRequest`. The response unmarshals into `elastic.DiscoveryResult`, providing fresh node metadata including version, IP addresses, and service status.

## Managing Node State and Health

The agent module maintains an accurate view of cluster health by merging locally cached configuration with real-time agent reports.

### Aggregating Online and Offline Status with refreshNodesInfo

The `refreshNodesInfo` function in [`modules/agent/api/elasticsearch.go`](https://github.com/infinilabs/console/blob/main/modules/agent/api/elasticsearch.go) (lines 99-144) produces a comprehensive node inventory by combining two data sources:
- Live node information from `GetElasticsearchNodesViaAgent` (nodes currently reporting via their agents)
- Direct Elasticsearch client calls for nodes not reported by agents (to detect offline or disconnected instances)

This merge process annotates each node with `online/offline` status, `cluster_id`, and **enrollment** state, ensuring the console displays accurate health indicators even when network partitions occur.

### Handling Enrollment Metadata

When `refreshNodesInfo` processes the discovery results, it cross-references the live data against the enrollment map from `GetEnrolledNodesByAgent`. This allows the UI to distinguish between discovered nodes (visible to the agent) and enrolled nodes (officially managed by the console), preventing unauthorized nodes from appearing in management views.

## Remote Log Access Without SSH

The agent module exposes specialized endpoints for operational data retrieval, eliminating the need for direct filesystem access or SSH tunnels.

### Listing Log Files with GetElasticLogFiles

The `GetElasticLogFiles` helper in [`modules/agent/api/elasticsearch.go`](https://github.com/infinilabs/console/blob/main/modules/agent/api/elasticsearch.go) (lines 32-53) sends a POST request to the agent's `/elasticsearch/logs/_list` endpoint. The function requires the `logs_path` extracted from the node's stored configuration (via `getAgentByNodeID`) and returns a list of available log files on the remote host.

### Streaming Log Content via GetElasticLogFileContent

For log analysis, `GetElasticLogFileContent` (lines 56-80) proxies requests to `/elasticsearch/logs/_read`, accepting parameters for `file_name`, `offset`, and `lines`. It returns paginated log content with a `has_more` flag, enabling the console to stream large log files without overwhelming the network or UI.

## Implementation Examples

### Discovering Enrolled Nodes

```go
// instanceID corresponds to the Agent instance stored in the console
nodes, err := agent.GetEnrolledNodesByAgent(instanceID)
if err != nil {
    log.Fatal(err)
}

for nodeID, bind := range nodes {
    fmt.Printf("Node %s belongs to cluster %s (enrolled=%t)\n",
        nodeID, bind.ClusterID, bind.Enrolled)
}

```

*Source:* `GetEnrolledNodesByAgent` – [`modules/agent/api/elasticsearch.go:54-78`](https://github.com/infinilabs/console/blob/main/modules/agent/api/elasticsearch.go#L54-L78)

### Refreshing Node Information

```go
// instanceEndpoint is the HTTP endpoint of the remote agent (e.g., "http://10.0.2.5:9000")
info, err := agent.refreshNodesInfo(instanceID, instanceEndpoint)
if err != nil {
    log.Fatalf("cannot refresh: %v", err)
}

// info.Nodes contains enriched LocalNodeInfo structs with status and metadata
for id, node := range info.Nodes {
    fmt.Printf("Node %s – status:%s enrolled:%t\n", id, node.Status, node.Enrolled)
}

```

*Source:* `refreshNodesInfo` – [`modules/agent/api/elasticsearch.go:99-144`](https://github.com/infinilabs/console/blob/main/modules/agent/api/elasticsearch.go#L99-L144)

### Retrieving Remote Log Files

```go
// Resolve the agent instance managing this specific nodeUUID
instance, err := agent.getAgentByNodeID(nodeUUID)
if err != nil {
    log.Fatal(err)
}

// Fetch available logs from the node's configured log path
logs, err := agent.GetElasticLogFiles(
    context.Background(),
    instance,
    "/var/log/elasticsearch", // logsPath from node config
)
if err != nil {
    log.Fatal(err)
}
fmt.Println("Available log files:", logs)

```

*Source:* `GetElasticLogFiles` – [`modules/agent/api/elasticsearch.go:32-53`](https://github.com/infinilabs/console/blob/main/modules/agent/api/elasticsearch.go#L32-L53)

### Reading Log Content with Pagination

```go
body := map[string]interface{}{
    "file_name": "elasticsearch.log",
    "logs_path": "/var/log/elasticsearch",
    "offset":    0,
    "lines":     200,
}

content, err := agent.GetElasticLogFileContent(context.Background(), instance, body)
if err != nil {
    log.Fatal(err)
}

fmt.Printf("Retrieved %d lines. Has more? %v\n", 
    len(content["lines"].([]interface{})), 
    content["has_more"])

```

*Source:* `GetElasticLogFileContent` – [`modules/agent/api/elasticsearch.go:56-80`](https://github.com/infinilabs/console/blob/main/modules/agent/api/elasticsearch.go#L56-L80)

## Summary

- The **agent module** creates a secure abstraction layer between the Infini Console and remote Elasticsearch nodes, requiring only HTTP connectivity to lightweight agents rather than direct cluster access.
- **Node discovery** relies on `GetEnrolledNodesByAgent` and `refreshNodesInfo` to maintain accurate mappings between agent instances and cluster topology, tracking both online and offline states.
- **Proxy architecture** via `server.ProxyAgentRequest` in [`plugin/managed/server/websocket_proxy.go`](https://github.com/infinilabs/console/blob/main/plugin/managed/server/websocket_proxy.go) handles all transport concerns, including TLS and authentication, for every remote operation.
- **Log management** functions `GetElasticLogFiles` and `GetElasticLogFileContent` expose remote filesystem access through standardized REST endpoints, eliminating SSH dependencies.
- All agent configuration persists in [`modules/agent/model/config.go`](https://github.com/infinilabs/console/blob/main/modules/agent/model/config.go), enabling centralized control of agent deployment parameters and security certificates.

## Frequently Asked Questions

### What is the primary purpose of the agent module in Infini Console?

The agent module serves as a bridge that enables the Infini Console to manage Elasticsearch nodes deployed across network-isolated environments. By installing a lightweight Infini Agent binary on each host, operators can discover nodes, monitor health, and retrieve logs through HTTP APIs without configuring VPNs, SSH keys, or direct Elasticsearch port exposure.

### How does the agent module handle network security for remote clusters?

Security relies on the `server.ProxyAgentRequest` implementation in [`plugin/managed/server/websocket_proxy.go`](https://github.com/infinilabs/console/blob/main/plugin/managed/server/websocket_proxy.go), which establishes authenticated, encrypted channels between the console and each agent. All traffic tunnels through this proxy layer, meaning the console never connects directly to Elasticsearch nodes; it only communicates with the agent endpoints, which in turn access local resources.

### Can the agent module manage multiple Elasticsearch clusters simultaneously?

Yes. The `GetEnrolledNodesByAgent` function returns `BindingItem` structs that include `ClusterID` fields, allowing a single agent instance to discover and report nodes from multiple distinct Elasticsearch clusters. The `refreshNodesInfo` function aggregates these into a unified view while maintaining cluster boundaries through metadata labels.

### What endpoints does the agent expose for log management?

The agent exposes two primary endpoints for log operations: `/elasticsearch/logs/_list` for retrieving available log files in a directory, and `/elasticsearch/logs/_read` for streaming file content with offset-based pagination. The console accesses these through `GetElasticLogFiles` and `GetElasticLogFileContent`, which proxy requests via the agent's HTTP interface defined in [`modules/agent/api/elasticsearch.go`](https://github.com/infinilabs/console/blob/main/modules/agent/api/elasticsearch.go).