# How Telegraf's Secret Store System Protects Sensitive Data: A Deep Dive into the Source Code

> Explore Telegraf's secret store system. Learn how it protects sensitive data using locked memory enclaves, preventing disk swaps and cryptographically wiping RAM.

- Repository: [InfluxData/telegraf](https://github.com/influxdata/telegraf)
- Tags: deep-dive
- Published: 2026-05-14

---

**Telegraf isolates sensitive credentials in locked memory enclaves using the memguard library, ensuring passwords and tokens are never swapped to disk and are cryptographically wiped from RAM immediately after use.**

Telegraf, the open-source agent from InfluxData, handles passwords, API tokens, and other confidential configuration values through a hardened **secret store system** designed to minimize exposure in memory. The implementation in the `influxdata/telegraf` repository uses a three-layer architecture that separates public-facing APIs from protected byte containers and dynamic resolution logic.

## Three-Layer Architecture of the Secret Store System

The secret store system is built from three cooperating layers that handle declaration, storage, and resolution of sensitive values.

### The Secret Struct (Public Interface)

Plugins interact with secrets through the **`Secret`** struct defined in [`config/secret.go`](https://github.com/influxdata/telegraf/blob/main/config/secret.go). This façade tracks whether a value is empty, holds a reference to the actual byte container, and maintains a list of **unlinked** references such as `@{storeID:key}` that require later resolution.

When Telegraf parses TOML configuration files, raw secret bytes are immediately transferred from the parser into this type, preventing accidental logging or exposure in error messages.

### Container Implementations (Protected vs. Unprotected)

Two concrete containers implement the internal `secretContainer` interface, selected at runtime based on the global protection mode:

- **`protectedSecretContainer`** ([`config/secret_protected.go`](https://github.com/influxdata/telegraf/blob/main/config/secret_protected.go)): Uses the third-party **memguard** library to store secrets inside a **locked enclave** that is never swapped to disk and is automatically wiped when destroyed.
- **`unprotectedSecretContainer`** ([`config/secret_unprotected.go`](https://github.com/influxdata/telegraf/blob/main/config/secret_unprotected.go)): Stores bytes in a plain Go slice when protection is deliberately disabled, typically for testing scenarios.

By default, Telegraf runs in protected mode, utilizing the memguard-based implementation.

### Secret-Store Linking and Resolution

When configuration parsing encounters placeholders like `@{mystore:mykey}`, Telegraf stores these in `Secret.unlinked`. After all plugins instantiate, the configuration engine—triggered by [`config/plugin_selector.go`](https://github.com/influxdata/telegraf/blob/main/config/plugin_selector.go)—resolves each placeholder against registered secret stores.

Dynamic resolvers implement `telegraf.ResolveFunc` and are invoked lazily. Static values are replaced directly in the container, while dynamic secrets remain encrypted until explicitly accessed.

## Memory Protection Mechanisms in Detail

The Telegraf secret store system implements several cryptographic defenses against memory-based attacks.

### Locked Memory Enclaves with memguard

In [`config/secret_protected.go`](https://github.com/influxdata/telegraf/blob/main/config/secret_protected.go), the `protectedSecretImpl.Container` method creates a secure memory region using `memguard.NewEnclave(secret)`. This API allocates **page-aligned memory** that is locked into RAM, preventing the operating system from paging the secret to disk swap space.

All read and write operations occur through `memguard.LockedBuffer` instances, which provide scoped access to the decrypted bytes while maintaining the underlying protection.

### Automatic Zero-Wipe on Destruction

When a secret is no longer needed, the `protectedSecretContainer.Destroy` method explicitly wipes memory. The implementation calls `lb.Destroy()` on the locked buffer and `c.Destroy()` on the enclave, both of which overwrite the underlying bytes with cryptographically secure random data before freeing the memory.

This ensures that secret values disappear from physical RAM and cannot be recovered through cold-boot attacks or memory forensics.

### Constant-Time Comparison for Timing Attack Resistance

The `protectedSecretContainer.Equals` method uses `lockedBuffer.EqualTo` from memguard to perform timing-attack-resistant comparisons. Unlike standard byte comparison that short-circuits on mismatch (leaking information through execution time), this implementation compares all bytes in constant time regardless of match position.

### Dynamic Resolution and Lazy Decryption

Dynamic secrets remain in their encrypted form until a plugin explicitly calls `Secret.Get()` (defined in [`config/secret.go`](https://github.com/influxdata/telegraf/blob/main/config/secret.go) lines 87-106). At that moment, the resolver decrypts the value into a temporary locked buffer, the plugin uses the data, and the buffer is immediately destroyed. This **just-in-time decryption** minimizes the window during which clear-text credentials exist in memory.

## Working with Secrets in Plugin Development

Developers can integrate with the secret store system using the standard Telegraf configuration patterns.

Declare a secret field in your plugin's configuration struct:

```go
type MyPlugin struct {
    Token config.Secret `toml:"token"`
    APIKey config.Secret `toml:"api_key"`
}

```

Access the secret inside the `Gather` method, ensuring immediate destruction after use:

```go
func (p *MyPlugin) Gather(acc telegraf.Accumulator) error {
    // Decrypt and retrieve the secret value
    buf, err := p.Token.Get()
    if err != nil {
        return err
    }
    defer buf.Destroy() // Cryptographically wipe memory on scope exit

    token := string(buf.Bytes())
    // ... use token to authenticate with external API ...
    return nil
}

```

For dynamic resolution from external vaults, implement resolver linking in the `Init` method:

```go
func (p *MyPlugin) Init() error {
    resolvers := map[string]telegraf.ResolveFunc{
        "@{vault:api_key}": func() ([]byte, bool, error) {
            secret, err := vaultClient.Get("api_key")
            return []byte(secret), false, err // false indicates static resolution
        },
    }
    return p.Token.Link(resolvers)
}

```

## Enabling and Disabling Protection

The global `selectedImpl` variable controls which container implementation is active. The API exposes `EnableSecretProtection()` and `DisableSecretProtection()` in [`config/secret.go`](https://github.com/influxdata/telegraf/blob/main/config/secret.go) (lines 48-54), allowing operators to toggle modes at startup.

Production deployments should always run with protection enabled. The unprotected mode is reserved for debugging sessions where memguard's memory locking might conflict with development tools or sandboxed environments.

## Summary

- The **Secret struct** provides a type-safe interface that prevents accidental string conversion and logging of sensitive values.
- **memguard integration** creates locked memory enclaves that resist swapping and cold-boot attacks.
- **Zero-wipe destruction** ensures secrets are overwritten in RAM immediately after use through explicit `Destroy()` calls.
- **Constant-time comparison** protects against timing side-channels when validating credentials.
- **Lazy resolution** keeps dynamic secrets encrypted until the moment of use, minimizing exposure windows.

## Frequently Asked Questions

### How does Telegraf prevent secrets from appearing in core dumps?

The `protectedSecretContainer` implementation uses `memguard.NewEnclave()` to allocate memory that is explicitly **mlock**'d (locked into physical RAM) and marked with **mprotect** flags that exclude it from core dump files. Additionally, the zero-wipe mechanism in `Destroy()` overwrites the bytes before the process releases the memory, ensuring that even if a dump occurs during active use, the enclave contents are not recoverable.

### What is the difference between protected and unprotected secret containers?

The **protected container** ([`config/secret_protected.go`](https://github.com/influxdata/telegraf/blob/main/config/secret_protected.go)) utilizes the memguard library for locked, non-swappable memory with automatic wiping, while the **unprotected container** ([`config/secret_unprotected.go`](https://github.com/influxdata/telegraf/blob/main/config/secret_unprotected.go)) stores secrets in standard Go slices without memory locking or cryptographic erasure. The unprotected variant exists primarily for testing environments where memguard's system calls might be restricted or when debugging requires readable memory traces.

### How do I reference a secret from a vault in my Telegraf configuration?

Use the placeholder syntax `@{storeID:key}` in your TOML configuration file, where `storeID` matches a registered secret store and `key` is the specific credential identifier. During the linking phase, Telegraf resolves these placeholders through the resolver functions defined in [`cmd/telegraf/cmd_secretstore.go`](https://github.com/influxdata/telegraf/blob/main/cmd/telegraf/cmd_secretstore.go), fetching the actual values from HashiCorp Vault, AWS Secrets Manager, or other supported backends without exposing them in configuration files.

### Can I disable secret protection for debugging purposes?

Yes. You can call `DisableSecretProtection()` at runtime to switch to the `unprotectedSecretContainer` implementation, which stores secrets in plain Go slices. However, this should only be used in development or testing environments, as it removes all memory isolation guarantees and allows secrets to appear in swap files and core dumps.