How ai-memory Sanitizes Incoming Hook Payloads: A Complete Technical Guide

ai-memory sanitizes every incoming hook payload using a dedicated Sanitizer struct that strips secrets, removes control characters, truncates oversized inputs, and enforces type safety at the storage boundary.

The ai-memory project, authored by Akita On Rails, is a Rust-based memory system for AI agents that must handle untrusted data from external hook sources. Sanitization is not an afterthought—it's enforced at multiple architectural layers to prevent secret leakage and injection attacks.

The Sanitization Pipeline Architecture

Incoming hook payloads traverse a strict pipeline before reaching persistent storage. The design follows a fail-safe default principle: data is assumed malicious until proven otherwise.

Layer 1: Payload Reception in the Router

The hook router (crates/ai-memory-hooks/src/router.rs) is the first line of defense. When a webhook request arrives, the router immediately wraps raw JSON in a Sanitized container:

// From crates/ai-memory-hooks/src/router.rs, line 2493
let sanitized = Sanitized::new(raw_obs, &state.sanitizer);

This pattern ensures that no raw string operations occur on unsanitized data. The RouterState holds a Sanitizer instance that defaults to the built-in rules, though operators can substitute custom implementations.

Layer 2: Core Sanitization Rules

The Sanitizer implementation lives in crates/ai-memory-core/src/sanitize.rs. It compiles regex patterns at initialization for performance, then applies this scrubbing sequence:

1. Secret detection — Matches API keys, tokens, and password patterns against a builtin deny-list
2. Control character stripping — Removes zero-width spaces, non-printable Unicode, and ASCII control sequences
3. Length bounding — Truncates payloads exceeding 2,000 characters

The central method is straightforward:

// scrub(&self, text: &str) -> String
let clean = sanitizer.scrub(&user_input);

Layer 3: Field-Level Sanitization in Event Processing

Not all fields receive identical treatment. The workstream implementation (crates/ai-memory-hooks/src/workstream.rs, lines 646-698) defines sanitize_events, which applies scrubbing selectively to prompt, reply, and result fields while preserving structural metadata.

The MCP HTTP server (crates/ai-memory-mcp/src/server.rs) adds domain-specific helpers for client-provided strings:

// crates/ai-memory-mcp/src/server.rs, lines 758-807
sanitize_client_name(&client_name)
sanitize_feedback_reason(&reason)

These helpers ensure that identifier strings and human-readable labels meet length and character constraints appropriate to their database columns.

Type Safety at the Storage Boundary

The database writer (crates/ai-memory-store/src/writer.rs) completes the security model. Line 661 documents a critical invariant: "the type system at the store boundary, so an unsanitized observation … would be a violation."

This means the Sanitized wrapper type must be consumed to extract its inner value. The compiler rejects direct storage of raw strings, making sanitization unavoidable rather than conventional.

Extending the Sanitizer

Operators requiring custom rules can replace the builtin sanitizer without forking the codebase:

// crates/ai-memory-wiki/src/lib.rs, lines 166-170
Wiki::with_sanitizer(my_custom_sanitizer)

Custom implementations must satisfy the same scrub interface but may:

• Add industry-specific secret patterns (e.g., medical record numbers)
• Implement allow-lists for known-safe content
• Adjust length limits for specialized deployments

Practical Examples

Basic Hook Handler Implementation

use ai_memory_core::Sanitizer;
use ai_memory_hooks::router::RouterState;

// RouterState::default() provides the builtin sanitizer
let state = RouterState::default();

// Raw JSON from HTTP request body
let raw_obs = r#"{"prompt": "My key is sk-abc123", "user": "alice"}"#;

// Sanitized wrapper scrubs all fields
let sanitized = Sanitized::new(&raw_obs, &state.sanitizer);

// Inner value is safe for storage
store_observation(sanitized.inner())?;

Manual String Sanitization for Testing

use ai_memory_core::Sanitizer;

let s = Sanitizer::builtin();

let raw = "API key: sk-live-51Nx...   \u{200B}\n<script>alert('xss')</script>";
let clean = s.scrub(&raw);

assert!(!clean.contains("sk-live"));        // secret removed
assert!(!clean.contains("<script>"));       // HTML stripped
assert!(!clean.contains("\u{200B}"));       // zero-width space gone
assert!(clean.len() <= 2000);               // length bounded

Key Files Reference

File	Responsibility	Critical Lines
crates/ai-memory-core/src/sanitize.rs	Sanitizer struct, regex rules, scrub method	struct definition ~132
crates/ai-memory-hooks/src/router.rs	Creates Sanitized wrappers from HTTP payloads	2493-2504
crates/ai-memory-hooks/src/workstream.rs	Event-specific field scrubbing	646-698
crates/ai-memory-mcp/src/server.rs	Client name and feedback sanitization	758-807
crates/ai-memory-wiki/src/lib.rs	Sanitizer customization API	166-170
crates/ai-memory-store/src/writer.rs	Type-safe storage boundary	661

Summary

• Untrusted by default: Every hook payload is wrapped in Sanitized before processing
• Defense in depth: Sanitization occurs at router, workstream, and server layers
• Bounded output: Secrets, control characters, and oversized inputs are removed or truncated
• Compile-time enforcement: The storage layer's type system prevents unsanitized data persistence
• Operator extensible: Custom sanitizers plug in via Wiki::with_sanitizer

Frequently Asked Questions

What happens if a payload exceeds the 2,000 character limit?

The Sanitizer::scrub method truncates the input to 2,000 characters after removing rejected patterns. This occurs silently; no error is raised, but data beyond the boundary is discarded to prevent denial-of-service via memory exhaustion.

Can the builtin sanitizer be disabled entirely?

Technically yes, by providing a no-op implementation via Wiki::with_sanitizer. However, the storage writer's type system still requires passing data through the Sanitized wrapper type, so complete bypass is impossible without code modification.

How does ai-memory detect secrets without knowing my specific key formats?

The builtin sanitizer uses regex patterns matching common secret conventions: sk- prefixes for Stripe keys, ghp_ for GitHub tokens, generic api_key, password, and token field names. Custom sanitizers can add organization-specific patterns for higher precision.

Is there a performance cost to sanitization every request?

Sanitizer construction compiles regex patterns once at startup. The scrub method itself operates in linear time relative to input length with bounded memory allocation due to the 2,000-character cap. For high-throughput deployments, benchmark against your typical payload sizes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}