How to Use Vault for Managing API Keys in Flowsint Enrichers

Flowsint stores third-party API keys in an encrypted Vault rather than source code, using AES-256-GCM encryption with per-user keys derived from a master environment variable, and injects secrets into enrichers at runtime via the VaultProtocol interface.

Flowsint provides a built-in Vault system to securely manage API credentials for third-party enrichment services. By storing secrets outside of your codebase and injecting them at runtime, you eliminate the risk of leaking sensitive tokens in version control. This guide explains how to configure the Vault, store API keys, and retrieve them inside your enrichers according to the reconurge/flowsint source code.

Vault Architecture and Encryption

The Vault system consists of three primary components that work together to provide secure secret management.

Core Components

The Vault class in flowsint-core/src/flowsint_core/core/vault.py implements the VaultProtocol interface and handles encryption, decryption, and database persistence. It stores secrets as Key rows containing ciphertext, IV, and salt.

The VaultService in flowsint-core/src/flowsint_core/core/services/vault_service.py acts as a factory that creates user-specific Vault instances. When you call VaultService.for_user(owner_id), it returns a configured Vault instance bound to that user's encryption key.

Enrichers receive a VaultProtocol implementation via dependency injection at runtime. The framework automatically instantiates the Vault and passes it to enrichers that declare secret requirements in their parameter schemas.

Encryption Standards

The Vault encrypts all secrets using AES-256-GCM with a per-user data key. This user-specific key is derived from a master key defined in your environment, ensuring that even database administrators cannot read secrets without the master key.

Configuring the Master Encryption Key

Before storing any secrets, you must configure the master encryption key that protects all user-specific keys.

Set the MASTER_VAULT_KEY_V1 environment variable to a base64-encoded 32-byte key:

export MASTER_VAULT_KEY_V1=base64:YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXoxMjM0NTY=

The Vault class reads this variable during initialization to derive user-specific encryption keys. Without this environment variable, the Vault cannot encrypt or decrypt secrets and will raise a configuration error.

Storing API Keys in the Vault

Administrators add secrets programmatically using the Vault.set_secret() method. This encrypts the plaintext and stores the resulting ciphertext in the database.

from uuid import UUID
from flowsint_core.core.services.vault_service import create_vault_service
from flowsint_core.core.db import get_db

with get_db() as db:
    vault_service = create_vault_service(db)
    owner_id = UUID("123e4567-e89b-12d3-a456-426614174000")
    vault = vault_service.for_user(owner_id)
    
    # Store the secret with a human-readable reference

    new_key = vault.set_secret("WHoxy_API_KEY", "my-super-secret-whoxy-token")
    print(f"Secret stored with ID {new_key.id}")

The vault_ref parameter (e.g., "WHoxy_API_KEY") serves as the human-readable identifier that enrichers use to retrieve the secret later.

Declaring Secret Requirements in Enrichers

Enrichers must declare their API key dependencies explicitly in their parameter schema using the "type": "vaultSecret" declaration.

@classmethod
def get_params_schema(cls):
    """Declare required parameters for this enricher."""
    return [
        {
            "name": "WHoxy_API_KEY",
            "type": "vaultSecret",
            "description": "Whoxy domain search engine API key.",
            "required": True,
        },
    ]

This declaration tells the Flowsint framework that this enricher requires access to the Vault and expects to find a secret named WHoxy_API_KEY.

Retrieving Secrets at Runtime

When the framework executes an enricher, it automatically injects a VaultProtocol implementation via the vault parameter. Enrichers retrieve plaintext secrets using vault.get_secret().

# Inside an enricher implementation

api_key = self.vault.get_secret("WHoxy_API_KEY")

# api_key now contains the decrypted token ready for HTTP requests

The get_secret() method first attempts a UUID lookup if you pass a UUID string. If that fails, it falls back to a name lookup against the vault_ref field. This dual-lookup mechanism supports both hardcoded names and user-specified vault entry IDs.

Handling User-Provided Vault IDs

For flexibility, enrichers can accept a specific vault entry ID in their parameters and implement fallback logic:

vault_id = params.get("WHoxy_API_KEY")
secret = self.vault.get_secret(vault_id)  # Try UUID first

if secret is None:
    secret = self.vault.get_secret("WHoxy_API_KEY")  # Fallback to name

This pattern allows users to override which specific Vault entry to use while maintaining a default fallback.

Integration Testing and Validation

The integration tests in flowsint-enrichers/tests/test_vault_integration.py verify that secret resolution, fallback logic, and error handling work correctly. When a required secret is missing, the enricher raises a clear error directing the user to the Vault UI rather than failing with a cryptic decryption error.

Summary

• Encryption: Flowsint uses AES-256-GCM with per-user keys derived from MASTER_VAULT_KEY_V1 to encrypt all secrets at rest.
• Storage: Use Vault.set_secret(vault_ref, plaintext) in flowsint-core/src/flowsint_core/core/vault.py to store API keys.
• Declaration: Define "type": "vaultSecret" in your enricher's get_params_schema() to declare dependencies.
• Injection: The framework automatically provides a VaultProtocol instance to enrichers at runtime.
• Retrieval: Call vault.get_secret() with either a UUID or the original vault_ref name to obtain plaintext secrets.

Frequently Asked Questions

What encryption algorithm does Flowsint Vault use?

Flowsint Vault uses AES-256-GCM encryption. Each user has a unique data encryption key derived from the master key defined in MASTER_VAULT_KEY_V1, ensuring that secrets remain secure even if the database is compromised.

How do I declare that my enricher needs a specific API key?

Add a parameter definition with "type": "vaultSecret" to your enricher's get_params_schema() method. Specify the name field as the lookup key (e.g., "WHoxy_API_KEY") and set "required": True if the enricher cannot function without it.

What happens if a secret is missing when the enricher runs?

If a required secret is not found in the Vault, the enricher raises a clear error message directing the user to add the missing key through the Vault UI. This prevents silent failures and provides actionable remediation steps.

Can I reference a specific Vault entry by ID rather than name?

Yes. The Vault.get_secret() method accepts either a UUID string or a name. If you provide a UUID, it performs a direct lookup; if that fails or you provide a string name, it falls back to searching by the vault_ref field.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}