How to Use the HTTP Listener Plugin for Metrics Ingestion in Telegraf

The http_listener_v2 plugin starts an HTTP server inside Telegraf that accepts metrics via POST or PUT requests, parses them using any supported data format, and forwards them to Telegraf's accumulator for further processing.

The http_listener_v2 plugin in the influxdata/telegraf repository provides a flexible, production-ready service input for receiving metrics over HTTP. This plugin supports multiple serialization formats, TLS encryption, basic authentication, and request metadata tagging, making it ideal for cloud-native observability pipelines.

Core Architecture and Source Implementation

The plugin is implemented in plugins/inputs/http_listener_v2/http_listener_v2.go. The core logic centers on the HTTPListenerV2 struct, which holds configuration state, the TCP/Unix listener, and the parser instance.

Plugin Initialization and Lifecycle

The Init function (lines 88-111) builds the TLS configuration, normalizes the service address with appropriate protocol prefixes, and sets the default success HTTP status code to 204 if not explicitly configured.

The Start method (lines 118-188) creates the underlying net.Listener—supporting TCP, TCP with TLS, or Unix domain sockets—sets read/write timeouts, and launches an http.Server that uses the plugin itself as its request handler. The Stop method (lines 188-202) handles graceful shutdown by closing the listener and waiting for active connections to complete.

Request Handling Flow

Incoming requests hit the ServeHTTP method (lines 202-217), which validates the request path against configured Paths, applies static response headers, and performs optional basic authentication before delegating to serveWrite.

The serveWrite function (lines 229-304) implements the core ingestion logic:

• Enforces MaxBodySize (default 500 MiB), returning HTTP 413 if exceeded
• Validates HTTP methods against the Methods allowlist, returning HTTP 405 if rejected
• Extracts payload data from either the request body or query string based on DataSource
• Parses bytes using the configured parser (e.g., Influx line protocol, JSON)
• Adds optional tags extracted from HTTP headers or request paths
• Returns the configured SuccessCode (default 204)

Data collection supports both collectBody and collectQuery paths (lines 326-376), with automatic handling of gzip and snappy compression based on Content-Encoding headers.

Configuration Options

The plugin uses the embedded sample.conf file for its default configuration. Here is a production-ready configuration covering the essential options:

[[inputs.http_listener_v2]]
  ## Network address: supports tcp://:8080, unix:///tmp/telegraf.sock, or TLS-enabled addresses

  service_address = "tcp://:8080"

  ## URL paths to accept. Requests to other paths return 404.

  paths = ["/telegraf", "/metrics"]

  ## HTTP methods allowed (default: POST, PUT)

  methods = ["POST", "PUT"]

  ## Parse data from "body" (default) or "query" parameters

  data_source = "body"

  ## Maximum request body size (0 disables limit)

  max_body_size = "500MB"

  ## HTTP status code returned on success

  http_success_code = 204

  ## Tag name for capturing request path (optional)

  # path_tag = "http_listener_v2_path"

  ## Map HTTP headers to metric tags

  # http_header_tags = {"X-Device-ID" = "device_id", "X-Region" = "region"}

  ## Basic authentication (recommended with TLS)

  # basic_username = "admin"

  # basic_password = "s3cr3t"

  ## TLS configuration for HTTPS

  # tls_cert = "/etc/telegraf/cert.pem"

  # tls_key = "/etc/telegraf/key.pem"

  # tls_min_version = "TLS12"

  ## Data format (required): influx, json, graphite, etc.

  data_format = "influx"

Key configuration fields from the HTTPListenerV2 struct (lines 49-64) include:

• ServiceAddress: Binding address with optional protocol prefix
• Paths: Slice of accepted URL paths for metric submission
• DataSource: Either body or query to indicate where metric data resides
• HTTPHeaderTags: Map of header names to tag keys for extracting metadata
• BasicUsername/BasicPassword: Constant-time credential validation (lines 400-410)

Sending Metrics to the HTTP Listener

Influx Line Protocol

Send standard InfluxDB line protocol points to the configured endpoint:

curl -i -XPOST 'http://localhost:8080/telegraf' \
  --data-binary 'cpu_load_short,host=server01 value=0.64 1434055562000000000'

JSON Payloads

When using data_format = "json", send JSON objects with appropriate content headers:

curl -i -XPOST 'http://localhost:8080/telegraf' \
  --header "Content-Type: application/json" \
  --data-binary '{"temperature":23.5,"location":"lab"}'

Query String Parameters

Configure data_source = "query" to extract metrics from URL parameters instead of the request body:

curl -i -XGET 'http://localhost:8080/telegraf?host=server01&value=0.42'

Compressed Data

The collectBody function (lines 326-376) automatically handles compressed payloads:

Gzip compressed:

gzip -c <<EOF | curl -i -XPOST 'http://localhost:8080/telegraf' \
  -H "Content-Encoding: gzip" --data-binary @-
cpu_load_short,host=server01 value=12.0 1422568543702900257
EOF

Snappy compressed:

# Requires snappy compression tool

echo 'cpu_load_short,host=server01 value=12.0' | snappy | \
  curl -i -XPOST 'http://localhost:8080/telegraf' \
  -H "Content-Encoding: snappy" --data-binary @-

Unix Domain Sockets

For local-only communication without TCP overhead:

# Configuration: service_address = "unix:///tmp/telegraf.sock"

curl -i --unix-socket /tmp/telegraf.sock -XPOST 'http://localhost/telegraf' \
  --data-binary 'cpu_load_short,host=server01 value=12.0 1422568543702900257'

Security and Advanced Features

TLS and Mutual Authentication

The plugin embeds common_tls.ServerConfig (lines 65-67) for HTTPS support. Configure certificate files and optionally enforce client certificate validation:

tls_cert = "/etc/telegraf/server.crt"
tls_key = "/etc/telegraf/server.key"
tls_min_version = "TLS13"
tls_allowed_cacerts = ["/etc/telegraf/ca.crt"]

Basic Authentication

Enable constant-time credential comparison by setting basic_username and basic_password:

curl -i -XPOST 'http://localhost:8080/telegraf' \
  -u admin:s3cr3t \
  --data-binary 'cpu_load_short,host=server01 value=12.0'

Request Metadata Enrichment

Path tagging: Set path_tag = "http_listener_v2_path" to automatically add the request URL path as a metric tag.

Header extraction: Use http_header_tags to promulgate HTTP headers into metric tags:

http_header_tags = {"X-Request-ID" = "request_id", "X-Cluster" = "cluster"}

This extracts header values and attaches them to every metric parsed from that request.

High-Throughput Handling

The implementation is fully concurrent; the test suite includes TestWriteHTTPHighTraffic, which validates processing of 25,000 points from 10 parallel writers without data loss. The underlying net.Listener and goroutine-per-request model in Start ensure horizontal scalability within a single instance.

Summary

• The http_listener_v2 plugin creates an HTTP endpoint inside Telegraf via the HTTPListenerV2 struct in plugins/inputs/http_listener_v2/http_listener_v2.go
• It supports any Telegraf data format (Influx line protocol, JSON, etc.) with automatic gzip/snappy decompression
• Configuration includes service_address, paths, methods, data_source, and optional TLS via common_tls.ServerConfig
• Security features include basic authentication (constant-time comparison) and mutual TLS
• Request metadata (paths, headers) can be attached as tags using path_tag and http_header_tags

Frequently Asked Questions

What data formats does the HTTP listener plugin support?

The plugin supports any data format available in Telegraf's parser registry. Set the data_format option to influx, json, graphite, csv, or others. The serveWrite method passes raw bytes to the configured parser, making the plugin agnostic to the specific serialization format as long as the correct parser is specified.

How do I secure the HTTP listener plugin with TLS?

Configure the tls_cert and tls_key options with valid certificate file paths. The plugin uses the common_tls.ServerConfig embedded struct (lines 65-67) to build TLS configurations. For production deployments, combine TLS with basic_username and basic_password to prevent unauthorized metric submission. You can also enable mutual TLS by setting tls_allowed_cacerts to require client certificate verification.

Can I use Unix sockets instead of TCP for local communication?

Yes. Set service_address = "unix:///path/to/socket.sock" and optionally configure socket_mode for file permissions. The Start method (lines 118-188) checks for the unix scheme and creates a Unix domain listener instead of TCP. This reduces network overhead for local collectors and provides filesystem-level access control via standard Unix permissions.

How do I extract query parameters instead of request body data?

Set data_source = "query" in your configuration. The collectQuery function (lines 363-376) extracts the raw query string, URL-decodes it, and passes the result to the parser. This is useful for simple metric injection via GET requests or when integrating with webhook systems that place data in URL parameters rather than request bodies.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}