How ai-memory's Privacy Strip Boundary Sanitizes Untrusted Hook Payloads

ai-memory treats every lifecycle hook payload as untrusted text and sanitizes it through a dedicated privacy strip boundary that uses regex-based redaction, an allow-list mechanism, and an opaque type system to prevent secrets from ever reaching durable storage.

The akitaonrails/ai-memory repository implements a defense-in-depth approach to data privacy by ensuring that sensitive information never crosses the persistence boundary. At the core of this system is the privacy strip boundary—a strictly enforced sanitization layer that processes all incoming hook payloads before they become durable observations. This architectural guarantee ensures that bearer tokens, API keys, and private keys are redacted irreversibly at the point of ingestion.

Core Architecture of the Privacy Strip Boundary

The Sanitizer Engine in sanitize.rs

The privacy strip boundary centers on a single Sanitizer instance constructed at startup. Defined in crates/ai-memory-core/src/sanitize.rs at lines 29-33, the Sanitizer compiles a static list of built-in regex patterns (BUILTIN_PATTERN_STRS) along with any operator-provided extra patterns.

The built-in detection patterns (lines 48-78) cover high-risk credential formats:

• Bearer tokens and API-key prefixes
• PEM-encoded blocks (SSH keys, certificates)
• URL-embedded credentials
• Common secret naming conventions

The scrubbing logic resides in Sanitizer::scrub(&self, input: &str) -> String (lines 196-215). This method iterates through the compiled regex set and replaces each match with the literal string "[REDACTED]". When a match occurs, the sanitizer logs which specific pattern triggered the redaction for audit purposes.

The Opaque Type Wrapper Sanitized

To enforce the privacy strip boundary at the type level, ai-memory uses Sanitized<T>, defined in sanitize.rs lines 28-33. This wrapper is deliberately opaque—once data enters the sanitized state, the inner value cannot be accessed directly. Instead, developers must explicitly call inner() for read-only access or into_inner() to consume the wrapper.

This type system guarantee ensures that no raw, unsanitized string can accidentally flow into the storage layer. The compiler enforces that every NewObservation must pass through Sanitized::new before persistence.

How Hook Payloads Flow Through the Sanitization Boundary

From HTTP POST to NewObservation in router.rs

When a lifecycle hook POSTs to the /hook endpoint, the request enters through crates/ai-memory-hooks/src/router.rs. The router deserializes the JSON payload into local types defined in payload.rs, then immediately constructs a NewObservation struct (defined in crates/ai-memory-core/src/observation.rs).

At this boundary, the raw payload is still considered untrusted. The router does not manipulate the observation directly; instead, it passes control to the sanitization layer.

Applying the Privacy Strip Before Persistence

The critical enforcement point occurs in Sanitized::new (sanitize.rs lines 42-52). This constructor accepts the raw NewObservation and applies three transformations:

1. Title scrubbing: Runs sanitizer.scrub() on obs.title
2. Body scrubbing: Runs sanitizer.scrub() on obs.body
3. Size enforcement: Truncates the body to OBSERVATION_BODY_MAX_BYTES using truncate_utf8_bytes_head_tail

Only after these steps does the system create a Sanitized<NewObservation>. The rest of the codebase receives this sanitized type, guaranteeing that durable storage only contains redacted text. Because this transformation occurs before any persistence operation, the original secrets are irretrievable—there is no decryption key or backdoor to recover redacted content.

Configuring the Privacy Strip Without Code Changes

Operators can extend the privacy strip boundary via SanitizeConfig without modifying the source code. The configuration accepts two parameters:

• extra_patterns: A vector of custom regex strings to supplement the built-in set
• allowlist: Substrings that should never be redacted, even if they match a pattern

This design allows teams to add organization-specific secret formats (like internal canary tokens) while ensuring that public identifiers or demo keys remain readable in logs.

Code Example: Sanitizing Hook Payloads in Practice

The following Rust example demonstrates the complete flow from raw hook payload to sanitized storage:

use ai_memory_core::{Sanitizer, SanitizeConfig, Sanitized, NewObservation};

// 1. Build a sanitizer (normally done once at startup)
let cfg = SanitizeConfig {
    extra_patterns: vec![r"CANARY-\d+".to_string()], // custom pattern
    allowlist: vec!["PROJECT_TOKEN_PUBLIC".to_string()], // keep this substring
};
let sanitizer = Sanitizer::new(&cfg).expect("invalid extra regex");

// 2. Directly scrub an arbitrary string
let raw = "Authorization: Bearer abcdef0123456789ABCDEF0123456789";
let safe = sanitizer.scrub(raw);
assert!(safe.contains("[REDACTED]") && !safe.contains("abcdef"));

// 3. Sanitizing a hook observation
let raw_obs = NewObservation {
    session_id: Default::default(),
    workspace_id: Default::default(),
    project_id: Default::default(),
    kind: ai_memory_core::ObservationKind::UserPrompt,
    extension: None,
    source_event: None,
    title: "OPENAI_API_KEY=sk-FAKE1234567890".into(),
    body: "see /home/user/.ssh/id_ed25519".into(),
    importance: 5,
};
let sanitized_obs = Sanitized::new(raw_obs, &sanitizer);
let stored = sanitized_obs.into_inner(); // now safe to hand to the store
assert!(stored.title.contains("[REDACTED]"));
assert!(stored.body.contains("[REDACTED]"));

Summary

• Irreversible redaction: The privacy strip boundary in sanitize.rs applies regex-based scrubbing before any data reaches durable storage, ensuring secrets cannot be recovered.
• Type-system enforcement: The Sanitized<T> wrapper (lines 28-33) prevents accidental leakage by requiring explicit unwrapping to access inner data.
• Configurable detection: Operators can extend the built-in patterns via SanitizeConfig without recompiling, using extra_patterns and allowlist options.
• Hook integration: The router in router.rs immediately sanitizes all NewObservation instances through Sanitized::new, enforcing the boundary at the entry point.
• Audit logging: The scrub method logs which pattern triggered each redaction, supporting security monitoring and compliance requirements.

Frequently Asked Questions

What built-in patterns does ai-memory's privacy strip detect?

According to the source code in sanitize.rs lines 48-78, the Sanitizer includes regex patterns for bearer tokens, API-key prefixes, PEM-encoded cryptographic blocks, SSH key paths, and URL-embedded credentials. These patterns cover common secret formats found in environment variables and configuration files.

How does the Sanitized type prevent accidental data leakage?

Sanitized<T> acts as an opaque newtype wrapper that separates trusted (scrubbed) data from untrusted input. Because the inner value has no public fields, developers cannot accidentally log or serialize the raw payload. The type requires calling inner() or into_inner(), making the sanitization boundary explicit in the code and preventing unsanitized strings from reaching storage APIs.

Can operators customize the privacy strip boundary without modifying source code?

Yes. The SanitizeConfig structure allows operators to pass extra_patterns (custom regexes) and an allowlist (protected substrings) during sanitizer initialization. This configuration happens at startup, enabling teams to adapt the privacy strip to organization-specific secrets without forking the repository or changing the compiled code.

At what point in the request lifecycle does sanitization occur?

Sanitization occurs immediately after deserialization and before any persistence logic. In crates/ai-memory-hooks/src/router.rs, the hook handler creates a NewObservation and immediately passes it to Sanitized::new. This ensures that if the subsequent storage operation fails or retries, the system never holds an unsanitized copy of the payload in memory for longer than necessary.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}