# How Hook Payloads Are Sanitized Before Storage in ai-memory: A Security Deep Dive

> Learn how akitaonrails ai-memory sanitizes hook payloads using a three-stage pipeline to scrub secrets, enforce size, and protect SQLite storage from untrusted data.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: security-guide
- Published: 2026-09-11

---

**Hook payloads in ai-memory undergo a mandatory three-stage sanitization pipeline using the `Sanitizer` type to scrub secrets via regex patterns, enforce size bounds, and guarantee that no untrusted data reaches SQLite storage.**

The `akitaonrails/ai-memory` repository implements a zero-trust architecture for handling incoming webhook data. Every hook payload is treated as untrusted and must pass through rigorous sanitization before persistence. This article examines the specific Rust implementation that prevents secret leakage and ensures deterministic, bounded storage.

## The Three-Stage Sanitization Pipeline

The system enforces sanitization at three critical integration points to provide defense-in-depth.

### Stage 1: Router Ingress Scrubbing

In [`crates/ai-memory-hooks/src/router.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/router.rs) (around line 2625), the hook-ingress router creates a `Sanitized` wrapper around the raw observation immediately upon receipt. This first line of defense calls `Sanitizer::scrub` on the title, body, and metadata fields before any further processing occurs.

```rust
// Simplified excerpt from router.rs
let raw_obs = incoming_observation;                         // untrusted JSON
let sanitized = Sanitized::new(raw_obs, &state.sanitizer); // scrubs title/body
store.write_observation(sanitized);

```

### Stage 2: Work-Stream Event Processing

The work-stream processor in [`crates/ai-memory-hooks/src/workstream.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/workstream.rs) (line 750) handles nested event structures. It iterates over each event in the payload—including checkpoints and tool calls—applying `sanitize_events` which invokes `sanitizer.scrub` on every string field.

```rust
fn sanitize_events(sanitizer: &Sanitizer, events: &mut [Event]) -> Result<()> {
    for ev in events {
        ev.content = sanitizer.scrub(&ev.content);          // body text
        if let Some(meta) = &mut ev.metadata {
            *meta = sanitizer.scrub(meta);                  // user metadata
        }
    }
    Ok(())
}

```

### Stage 3: Storage Layer Verification

Finally, the `Writer` in [`crates/ai-memory-store/src/writer.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/writer.rs) (line 946) receives only the already-sanitized `Sanitized` observation. This guarantees that the SQLite backend never processes raw, potentially secret-containing text.

```rust
// Writer receives only sanitized data
writer.send(StoreCommand::InsertObservation {
    observation: sanitized_observation,
});

```

## Core Sanitization Implementation

The `Sanitizer` type defined in [`crates/ai-memory-core/src/sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/sanitize.rs) provides the central logic for secret removal.

### The scrub Method and SanitizerInner

The `SanitizerInner` struct implements the `scrub(&self, input: &str) -> String` method, which returns a cleaned string suitable for storage. This method applies:

- **Built-in secret patterns**: Compiled regular expressions matching common secrets including API keys, JWTs, OAuth tokens, and GCP keys.
- **Operator-provided patterns**: Custom regexes defined in the `[sanitize]` configuration section.
- **Allow-lists**: Optional patterns that prevent false-positive redaction of permitted strings.
- **Size bounding**: Strings exceeding the configured limit (default 2000 bytes) are truncated before regex processing to prevent denial-of-service.

### Using the Sanitizer API

```rust
use ai_memory_core::Sanitizer;

// Initialize with built-in secret detection patterns
let sanitizer = Sanitizer::builtin();

// Example payload containing a JWT
let raw = "User token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...";
let safe = sanitizer.scrub(raw);

assert!(safe.contains("[REDACTED]"));   // Secret replaced

```

## Security and Performance Guarantees

This architecture provides three critical guarantees:

1. **Secret Containment**: Sensitive data appearing in prompts, tool outputs, or user metadata never leaks to the database or wiki interface.
2. **Deterministic Output**: The same sanitizer instance produces identical outputs for identical inputs, ensuring consistent storage behavior.
3. **Bounded Storage**: The 2000-byte default truncation prevents attackers from exhausting storage with arbitrarily large payloads.

## Summary

- Hook payloads in ai-memory are treated as **untrusted** by default and wrapped in a `Sanitized` type immediately upon ingress.
- The `Sanitizer::scrub` method in [`crates/ai-memory-core/src/sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/sanitize.rs) applies regex-based secret detection, allow-lists, and size bounding.
- Sanitization occurs at three layers: the **router** (initial scrubbing), the **work-stream** (event-level cleaning), and the **writer** (storage verification).
- Built-in patterns detect **JWTs, API keys, OAuth tokens, and cloud provider credentials**, with support for operator-defined custom rules.
- **Size limits** (default 2000 bytes) truncate oversized strings before processing to prevent DoS attacks.

## Frequently Asked Questions

### What types of secrets does ai-memory detect automatically?

The built-in sanitizer recognizes common high-entropy patterns including JSON Web Tokens (JWTs), API keys for major providers, OAuth tokens, Google Cloud Platform keys, and other credential formats. These patterns are compiled into the `Sanitizer` at initialization and applied via the `scrub` method.

### Can operators customize the sanitization behavior?

Yes. Operators can extend the built-in patterns through the `[sanitize]` section of the configuration file, adding custom regular expressions for domain-specific secrets. Additionally, allow-lists can be configured to prevent false-positive redaction of specific permitted strings or patterns.

### How does ai-memory handle oversized hook payloads?

Before regex processing, the sanitizer enforces a **bounding** limit of 2000 bytes by default. Any string exceeding this length is truncated. This prevents denial-of-service attacks that attempt to store arbitrarily large payloads or exhaust processing resources on massive regex operations.

### Is metadata subjected to the same sanitization as content?

Yes. The work-stream processor explicitly scrubs both `content` fields and `metadata` fields. As shown in [`workstream.rs`](https://github.com/akitaonrails/ai-memory/blob/main/workstream.rs), the `sanitize_events` function applies `sanitizer.scrub` to the content body and iterates through any user-provided metadata, ensuring no secret-bearing fields escape processing.