How to Handle Partial Write Errors in Telegraf Output Plugins

To handle partial write errors in Telegraf output plugins, detect the "partial write" substring in server error responses, log the diagnostic message, and return nil to discard the failed points without triggering a retry.

When building or extending output plugins for the influxdata/telegraf repository, you must account for partial write errors—failures where the destination system accepts some metrics but rejects others due to schema conflicts or type mismatches. The built-in InfluxDB output plugin demonstrates the canonical strategy for managing these unrecoverable errors.

How Partial Write Errors Occur in Output Plugins

A partial write error happens when a time-series database accepts a subset of the batch but rejects individual points. In InfluxDB, this commonly occurs with field type conflicts (e.g., writing a float to a field previously defined as an integer). Unlike transient network failures, these errors indicate data-level incompatibilities that will fail identically on every retry attempt.

The Canonical Pattern: InfluxDB Output Plugin

The implementation in plugins/outputs/influxdb/http.go provides the authoritative reference for handling partial write errors. The httpClient.writeBatch method parses server responses and classifies errors to determine whether retry logic should execute.

Detecting Partial Write Errors in HTTP Responses

After sending a batch, the plugin decodes the response body into a writeResponseError struct and inspects the Err field for specific substrings. The code checks against the constant errStringPartialWrite to identify non-recoverable failures:

// plugins/outputs/influxdb/http.go
if strings.Contains(desc, errStringPartialWrite) {
    // "partial write" covers cases like field-type conflicts.
    // The points are not recoverable; we log and discard them.
    c.log.Errorf(
        "When writing to [%s]: received error %v; discarding points",
        c.URL(), desc,
    )
    return nil  // no retry – move on to the next batch
}

The desc variable contains the raw error message from the server. By matching "partial write", the plugin distinguishes schema violations from retriable errors like timeouts or connection resets.

Why Partial Write Errors Are Not Retried

Returning nil from the error handler signals to Telegraf's agent that the write operation completed successfully from a retry perspective. This behavior is critical because:

• The server has already rejected the points due to immutable schema constraints
• Retrying would flood the server with identical invalid data
• Valid points in the batch have already been persisted; only the offending points need dropping

The plugin logs the failure at the Errorf level to alert operators of potential schema mismatches without crashing the agent or blocking subsequent batches.

Implementing Partial Write Handling in Custom Plugins

When developing a custom output plugin for proprietary time-series stores, replicate the pattern from the InfluxDB implementation. Parse the error response, identify partial write conditions, and explicitly drop the metrics.

Example Implementation for Custom HTTP Outputs

func (p *MyOutput) writeBatch(respBody io.ReadCloser) error {
    // Decode the response error payload (JSON with an "error" field)
    var r struct{ Err string `json:"error"` }
    if err := json.NewDecoder(respBody).Decode(&r); err != nil {
        return err
    }

    // Detect a partial write condition
    if strings.Contains(r.Err, "partial write") {
        p.logger.Errorf("Partial write error: %s – dropping points", r.Err)
        // Points have already been dropped by the server; do not retry.
        return nil
    }

    // Handle other error types with standard retry logic
    return fmt.Errorf("write failed: %s", r.Err)
}

Key implementation requirements:

• Parse server responses into a structured format to extract error messages
• Match against "partial write" (or your target system's equivalent substring)
• Log sufficient context including endpoint URLs and full error descriptions
• Return nil to prevent automatic retries while allowing the agent to continue processing

Summary

• Partial write errors indicate schema or type conflicts that cannot be resolved by resending data
• Detection relies on parsing server error responses for the substring "partial write" as implemented in plugins/outputs/influxdb/http.go
• Recovery strategy requires logging the error and returning nil to discard points without retry
• Custom plugins should follow the InfluxDB output pattern of classifying errors and explicitly handling non-retriable failures

Frequently Asked Questions

What causes a partial write error in InfluxDB?

A partial write error occurs when some metrics in a batch violate the target database's schema, such as field type conflicts (attempting to write a string to a field previously defined as a float) or timestamp precision mismatches. The server accepts valid points while rejecting only the offending metrics, returning a "partial write" error message.

Should I retry metrics that failed with a partial write error?

No. According to the Telegraf source code in plugins/outputs/influxdb/http.go, partial write errors are not recoverable through retry logic. The server has already persisted the valid points and permanently rejected the invalid ones based on schema constraints. Retrying identical data would only generate duplicate logs and unnecessary network traffic.

How do I implement partial write detection in a custom Telegraf output plugin?

Implement error string matching in your write method. Decode the HTTP response body, check if the error message contains "partial write" (or your target system's equivalent), log the condition with logger.Errorf(), and return nil to signal successful handling without retry. See the implementation in plugins/outputs/influxdb/http.go for the reference pattern using strings.Contains(desc, errStringPartialWrite).

Where can I find test examples for partial write error handling?

The unit tests in plugins/outputs/influxdb/http_test.go demonstrate how to verify partial write behavior, including mock server responses that trigger the errStringPartialWrite code path and assert that the plugin returns nil rather than an error.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}