# How ai-memory's Sanitize Boundary Prevents Secrets from Entering the Store

> Discover how ai-memory's Sanitize Boundary prevents secrets from entering storage. Learn how its compile-time guarantee and wrapper scrub credentials with regex patterns, ensuring secure data persistence.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: security-guide
- Published: 2026-08-30

---

**ai-memory enforces a compile-time guarantee that no raw text can be persisted to storage by requiring all writes to pass through a `Sanitized<T>` wrapper that scrubs credentials using configurable regex patterns.**

The `akitaonrails/ai-memory` codebase implements a defense-in-depth approach to secret management. Rather than relying on post-hoc scanning or manual review, the system embeds privacy controls directly into the type system and data lifecycle. This article explains how the sanitize boundary works, where it is enforced, and how you can configure it for your deployment.

## Core Components of the Sanitize Boundary

The protection layer consists of two tightly-coupled mechanisms: a stateful `Sanitizer` that performs pattern-based redaction, and a generic `Sanitized<T>` wrapper that makes the boundary unbypassable.

### The `Sanitizer` State Machine

Defined in [[`crates/ai-memory-core/src/sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/sanitize.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/sanitize.rs), the `Sanitizer` struct compiles regex patterns at initialization and applies them uniformly across the application.

**Built-in detection patterns** cover common secret formats:

- Bearer tokens and API key prefixes (`sk-`, `Bearer `, etc.)
- PEM-encoded cryptographic material
- URL-embedded credentials (`user:pass@host`)
- Private key file paths and environment variable assignments

These patterns are compiled once during `Sanitizer::new()` or `Sanitizer::builtin()` and shared via `Arc`, ensuring consistent behavior without runtime recompilation.

**Configuration options** extend or relax the default rules:

```rust
// From sanitize.rs lines 55-62
pub struct SanitizeConfig {
    /// Additional regex patterns to match beyond built-ins
    pub extra_patterns: Vec<String>,
    /// Substrings that must never be redacted even if they match patterns
    pub allowlist: Vec<String>,
}

```

The core scrubbing logic in `scrub()` (lines 199-210) iterates all compiled patterns, replacing matches with the literal string `[REDACTED]` unless the match contains an allowlist entry:

```rust
// Conceptual implementation
pub fn scrub(&self, input: &str) -> String {
    let mut output = input.to_string();
    for regex in &self.compiled {
        output = regex.replace_all(&output, "[REDACTED]").to_string();
    }
    output
}

```

### The `Sanitized<T>` Compile-Time Boundary

The `Sanitized<T>` wrapper provides the enforcement mechanism. Located in the same [[`sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/sanitize.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/sanitize.rs) file (lines 42-50), this type has only one valid constructor:

```rust
// Lines 42-44: Safe constructor is the only way to build this type
impl<T> Sanitized<T> {
    pub fn new(value: T, sanitizer: &Sanitizer) -> Self
    where
        T: Sanitize, // Requires implementors to define scrubbing behavior
    { /* ... */ }
}

```

For observations, the implementation `impl Sanitized<NewObservation>` performs three operations:

1. Scrubs the `title` field using `sanitizer.scrub()`
2. Scrubs the `body` field using `sanitizer.scrub()`
3. Enforces `OBSERVATION_BODY_MAX_BYTES = 16 KiB` to bound storage consumption

The result: **the compiler rejects any attempt to persist unsanitized data**. The store writer in [[`crates/ai-memory-store/src/writer.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/writer.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/writer.rs) (lines 660-680) accepts only `Sanitized<NewObservation>` values, making the boundary architectural rather than advisory.

## Where the Boundary Is Enforced

The sanitize boundary applies at every text ingress point across the codebase.

### Lifecycle Hooks Entry Point

In [[`crates/ai-memory-hooks/src/router.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/router.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/router.rs) (lines 2490-2505), incoming JSON payloads from external systems are immediately wrapped:

```rust
// Router converts HTTP payload to internal representation
let raw_observation: NewObservation = parse_json(body)?;
let sanitized = Sanitized::new(raw_observation, &self.sanitizer);
// Only `sanitized` can be passed to the store
store.insert_observation(sanitized)?;

```

This ensures that malicious or accidentally-leaked credentials in hook payloads never reach durable storage.

### Wiki Content Processing

The wiki subsystem in [[`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs) applies sanitization at multiple points:

- Lines 166-180: Page titles and bodies are scrubbed before markdown file creation
- Lines 1107-1172: Front-matter metadata undergoes the same processing

This protects against credential leakage through documentation that might contain copied terminal output or configuration examples.

### MCP API Hand-Off

The Model Context Protocol server in [[`crates/ai-memory-mcp/src/server.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/server.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/server.rs) (lines 761-805) sanitizes:

- Client-provided observation names
- Feedback reason strings
- Custom metadata fields

This prevents LLM-generated content or external tool outputs from contaminating the observation store.

## Practical Configuration and Usage

### Building a Custom Sanitizer

```rust
use ai_memory_core::{Sanitizer, SanitizeConfig, Sanitized, NewObservation};

// Step 1: Define custom patterns and allowlist exceptions
let cfg = SanitizeConfig {
    extra_patterns: vec![
        r"CANARY-\d{6}".to_string(),      // Internal canary tokens
        r"proj_[a-z0-9]{32}".to_string(), // Project-specific API format
    ],
    allowlist: vec![
        "PROJECT_TOKEN_PUBLIC".to_string(), // Safe to log this literal
    ],
};

// Step 2: Compile sanitizer (fails fast on invalid regex)
let sanitizer = Sanitizer::new(&cfg).expect("valid regex patterns");

// Step 3: Create observation with potential secrets
let raw = NewObservation {
    session_id: SessionId::new(),
    workspace_id: WorkspaceId::new(),
    project_id: ProjectId::new(),
    kind: ObservationKind::UserPrompt,
    extension: None,
    source_event: None,
    title: "OPENAI_API_KEY=sk-secret123abc".into(),
    body: "Error connecting to db: postgresql://admin:hunter2@prod.internal/ai_db".into(),
    importance: 5,
};

// Step 4: Apply boundary - this is required, not optional
let safe = Sanitized::new(raw, &sanitizer);

// Step 5: Extract and persist (only possible after sanitization)
let inner: NewObservation = safe.into_inner();
writer.insert_observation(inner).await?;

```

### Result Inspection

After processing, sensitive values are replaced:

| Field | Original | After `Scrub()` |
|-------|----------|-----------------|
| `title` | `OPENAI_API_KEY=sk-secret123abc` | `OPENAI_API_KEY=[REDACTED]` |
| `body` | `postgresql://admin:hunter2@...` | `postgresql://[REDACTED]@...` |

The `[REDACTED]` literal is chosen specifically to be:

- Human-readable in logs and debugging
- Unlikely to collide with legitimate data
- Consistent across all redaction sites for searchability

## Architecture Benefits

**Type system enforcement** eliminates entire categories of bugs. Developers cannot accidentally call `writer.insert_observation(raw_value)` because the method signature requires `Sanitized<NewObservation>`.

**Global consistency** emerges from the `Arc<Sanitizer>` sharing pattern. All components—hooks, wiki, MCP—use identical regex compilation and matching logic.

**Configurable without recompilation** via the `[sanitize]` section in [`config.toml`](https://github.com/akitaonrails/ai-memory/blob/main/config.toml). Operators can tune detection without modifying source code or rebuilding binaries.

## Summary

- **`Sanitizer`** compiles regex patterns at startup and provides the `scrub()` method for credential redaction
- **`Sanitized<T>`** creates a compile-time boundary that prevents unsanitized data from reaching storage APIs
- **The boundary applies at every ingress point**: lifecycle hooks, wiki writes, and MCP endpoints
- **Configuration** supports custom patterns via `extra_patterns` and safe-words via `allowlist` without code changes
- **The store writer** in [`ai-memory-store/src/writer.rs`](https://github.com/akitaonrails/ai-memory/blob/main/ai-memory-store/src/writer.rs) only accepts sanitized observations, making enforcement automatic

## Frequently Asked Questions

### What secret patterns does ai-memory detect by default?

The built-in patterns cover Bearer tokens, OpenAI/API key prefixes (`sk-`, `pk-`), PEM blocks, URL credentials, private key paths, and common environment variable assignments. The full list is defined in [[`sanitize.rs`](https://github.com/akitaonrails/ai-memory/blob/main/sanitize.rs) lines 48-78](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-core/src/sanitize.rs#L48-L78).

### Can I add custom patterns without modifying the source code?

Yes. Add an `[sanitize]` section to your [`config.toml`](https://github.com/akitaonrails/ai-memory/blob/main/config.toml) with `extra_patterns` containing valid regex strings. The sanitizer will compile these alongside built-ins at startup. Invalid patterns cause immediate failure with a descriptive error.

### What happens if legitimate data accidentally matches a secret pattern?

Use the `allowlist` configuration option to specify substrings that must never be redacted. For example, `allowlist = ["PUBLIC_TOKEN_EXAMPLE"]` preserves that literal even if it matches a regex. Apply this sparingly and document each exception.

### Is there any way to bypass the sanitize boundary?

No—not without modifying the source code to change [`writer.rs`](https://github.com/akitaonrails/ai-memory/blob/main/writer.rs) method signatures. The `Sanitized<T>` type has no public constructors other than `::new()`, and that constructor requires a `Sanitizer` reference. The Rust compiler enforces this guarantee.

### Does sanitization affect performance?

The `Sanitizer` is built once and shared via `Arc`, so regex compilation cost is paid at startup. The `scrub()` method runs linearly over input text against compiled patterns. For typical observation sizes (under 16 KiB), overhead is negligible compared to SQLite I/O.