How Telegraf Service Inputs Work and How They Differ from Regular Inputs

Service inputs in Telegraf are long-running plugins that implement Start() and Stop() lifecycle methods to manage persistent connections, while regular inputs rely on the Gather() method being invoked repeatedly on every collection interval.

Telegraf, the open-source metrics collection agent from InfluxData, supports two distinct architectural patterns for input plugins within the influxdata/telegraf repository. Understanding the distinction between service inputs and regular inputs is critical for developers building custom plugins and engineers optimizing data collection pipelines. The fundamental difference lies in execution semantics: regular inputs are stateless and polled periodically, whereas service inputs maintain background goroutines and manage their own event loops.

What Defines a Service Input

A service input extends the basic Input interface by implementing the ServiceInput interface defined in input.go. This interface adds two critical lifecycle hooks:

type ServiceInput interface {
    // Start the ServiceInput. The Accumulator may be retained and used
    // until Stop is called.
    Start(acc telegraf.Accumulator) error

    // Stop the ServiceInput. Called when Telegraf shuts down.
    Stop()
}

Source: [input.go lines 11-14](https://github.com/influxdata/telegraf/blob/master/input.go#L11-L14)

The Start method is invoked exactly once during plugin initialization. It receives an Accumulator that the plugin can retain and use asynchronously to emit metrics until Stop is called. This design allows service inputs to maintain long-lived connections, listen on sockets, or stream data from external APIs without being invoked on a timer.

How Telegraf Detects and Manages Service Inputs

Telegraf determines whether an input is a service plugin using type assertions at runtime. In models/running_input.go, the agent checks if the plugin implements telegraf.ServiceInput before deciding which execution path to follow:

plugin, ok := r.Input.(telegraf.ServiceInput)   // line 146

The agent tracks service inputs separately to ensure proper lifecycle management. When Telegraf initializes, it calls Start for each service input (lines 254+). During shutdown, it guarantees that Stop is invoked to clean up resources (line 193).

Source: [models/running_input.go lines 146, 193, 254](https://github.com/influxdata/telegraf/blob/master/models/running_input.go)

Regular inputs skip this logic entirely. Telegraf invokes their Gather method synchronously on every collection tick without checking for Start or Stop implementations.

Critical Differences Between Service Inputs and Regular Inputs

The architectural distinction creates fundamentally different behaviors for plugin developers:

Feature	Regular Input	Service Input
Primary Method	Gather(acc telegraf.Accumulator) error – called every collection interval	Start(acc telegraf.Accumulator) error – called once, runs continuously
Shutdown Behavior	No explicit hook; Telegraf stops calling Gather	Stop() – guaranteed call on Telegraf shutdown
State Management	Stateless or short-lived per-gather; opens/closes connections each call	Maintains long-lived connections, background goroutines, or socket listeners
Typical Use Cases	System metrics (CPU, memory, disk), simple HTTP polling	Log streaming, Kafka consumers, socket listeners, Docker event monitoring
Interface Requirements	Implements Input (Description, SampleConfig, Gather)	Implements Input plus Start and Stop

Lifecycle Execution Flow

Telegraf executes service inputs through a specific orchestration sequence:

1. Plugin Discovery: Telegraf loads plugins via the registry and wraps them in RunningInput structs.
2. Type Assertion: In models/running_input.go, the code asserts r.Input.(telegraf.ServiceInput) to detect service capabilities.
3. Initialization: If the assertion succeeds and !r.started, Telegraf calls Start, passing the accumulator. The plugin may spawn goroutines.
4. Runtime: The plugin runs independently, emitting metrics asynchronously until Telegraf receives a shutdown signal.
5. Cleanup: Telegraf iterates running inputs and invokes Stop() for each service input, allowing graceful termination.

The shim mechanism for external plugins replicates this logic. In plugins/common/shim/input.go (lines 40-48), the shim performs the same type assertion to determine whether to call Start and Stop for plugins loaded via the execd driver.

Implementation Examples

Service Input Skeleton

This minimal implementation demonstrates the persistent nature of service inputs:

type MyService struct {
    done chan struct{}
}

func (s *MyService) Description() string { return "Long-running service example" }
func (s *MyService) SampleConfig() string { return `` }

// Start launches a background goroutine that emits metrics
func (s *MyService) Start(acc telegraf.Accumulator) error {
    s.done = make(chan struct{})
    go func() {
        ticker := time.NewTicker(10 * time.Second)
        defer ticker.Stop()
        for {
            select {
            case <-ticker.C:
                acc.AddGauge("service_metric", 
                    map[string]string{"host": "localhost"}, 
                    map[string]interface{}{"value": 42})
            case <-s.done:
                return
            }
        }
    }()
    return nil
}

// Stop signals the goroutine to exit
func (s *MyService) Stop() {
    close(s.done)
}

The accumulator passed to Start is retained and reused across multiple collection cycles without Telegraf re-invoking the method.

Regular Input Comparison

In contrast, a regular input is invoked repeatedly with no persistent state between calls:

type MyPoller struct{}

func (p *MyPoller) Description() string { return "Simple polling input" }
func (p *MyPoller) SampleConfig() string { return `` }

// Gather is called every interval (e.g., 10s) by Telegraf
func (p *MyPoller) Gather(acc telegraf.Accumulator) error {
    // Connection opened and closed within this method
    resp, err := http.Get("http://localhost/metrics")
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    
    acc.AddCounter("poll_total", nil, map[string]interface{}{"value": 1})
    return nil
}

Telegraf manages the scheduling, calling Gather synchronously on each tick, making this pattern unsuitable for streaming or event-driven data sources.

Summary

• Service inputs implement Start() and Stop() in input.go to run continuously, while regular inputs rely on periodic Gather() calls.
• Telegraf detects service inputs via type assertions in models/running_input.go (lines 146, 193, 254) and manages their lifecycle separately from polled inputs.
• Service inputs retain the Accumulator passed to Start to emit metrics asynchronously, ideal for streaming data and persistent connections.
• The shims in plugins/common/shim/input.go replicate this detection logic for externally executed plugins.

Frequently Asked Questions

When should I use a service input instead of a regular input?

Use a service input when your plugin needs to maintain persistent connections, listen for events, or stream data continuously (e.g., Docker logs, Kafka consumers, or HTTP webhooks). Use a regular input when performing simple, stateless polling where opening a new connection on each interval is acceptable (e.g., CPU metrics, disk usage, or HTTP health checks).

What happens if Start() returns an error?

If Start() returns a non-nil error, Telegraf logs the failure and prevents the plugin from running. According to the implementation in models/running_input.go, the error propagates up during the agent startup phase, potentially preventing Telegraf from starting if the plugin is critical to the configuration.

Can a single plugin implement both ServiceInput and regular input patterns?

While a struct can technically implement both Gather() and Start()/Stop(), Telegraf treats it exclusively as a service input if the ServiceInput interface is satisfied. The type assertion in running_input.go checks for ServiceInput first, and if detected, Telegraf calls Start and never invokes Gather automatically. You should design your plugin for one pattern or the other based on your data source requirements.

How do I ensure proper resource cleanup in a service input?

Always implement the Stop() method to close channels, stop goroutines, and release network connections. Telegraf guarantees that Stop is called during shutdown (as shown in models/running_input.go line 193), making it the proper location for deferred cleanup operations that would typically go in a defer statement in short-lived functions.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}