Understanding the `Sanitized<NewObservation>` Type in ai‑memory's Security Model

Sanitized<NewObservation> is the typed safety gate that guarantees every observation submitted to ai‑memory has been stripped of secrets before it is written to durable storage, leveraging Rust's type system to prevent accidental credential leaks at compile time.

The ai‑memory project implements a privacy‑first architecture for persistent AI session management. At the core of its security model lies the Sanitized<T> generic wrapper, specifically instantiated as Sanitized<NewObservation>, which transforms runtime sanitization into a compile‑time invariant. This pattern ensures that raw observations containing API keys, tokens, or private keys can never reach the SQLite store without first passing through a mandatory privacy strip.

The Typed Boundary Pattern

The Sanitized<T> wrapper functions as a typed boundary that marks values which have already undergone privacy transformation. According to the akitaonrails/ai‑memory source code, the only public constructor for Sanitized<NewObservation> is Sanitized::new, implemented in crates/ai-memory-core/src/sanitize.rs at lines 29‑33.

Because the constructor is the sole entry point for creating this type, the compiler enforces that no code path can persist an unsanitized observation. Attempting to pass a raw NewObservation directly to storage functions results in a type mismatch error, making the security check impossible to bypass through programmer error.

Enforced Sanitization and Size Limits

The Sanitized::new method enforces two critical security constraints defined in crates/ai-memory-core/src/sanitize.rs (lines 42‑51):

• Secret Redaction: The method receives a raw NewObservation and applies the Sanitizer, which redacts credentials, tokens, private keys, and other sensitive patterns from both the title and body fields.
• Universal Size Ceiling: The observation is truncated to respect OBSERVATION_BODY_MAX_BYTES, preserving only the head and tail of large payloads to prevent denial‑of‑service attacks through storage exhaustion.

Once the Sanitizer completes its transformation, the resulting observation is wrapped in the Sanitized container, cryptographically sealing it as safe for persistence.

Immutability and Trust Guarantees

After creation, Sanitized<NewObservation> provides an immutable guarantee that the contained data holds no raw secrets. The inner value is accessible only through two deliberately simple getters:

• inner() – Returns a reference to the sanitized observation.
• into_inner() – Consumes the wrapper and returns the owned sanitized observation.

This design means that once a Sanitized<NewObservation> exists, downstream components in crates/ai-memory-store/src/ops.rs can trust that the observation is safe to write to the database without additional checks. The rest of the system operates under the invariant that privacy‑strip before storage = immutable guarantee, removing the need for defensive re‑sanitization at every persistence layer.

Implementation Across the Ingestion Pipeline

The security model relies on specific file locations to enforce sanitization at the system boundaries:

File	Security Role
crates/ai-memory-core/src/sanitize.rs	Defines Sanitized<T>, the Sanitizer implementation, and the Sanitized::new constructor for NewObservation.
crates/ai-memory-core/src/ids.rs	Provides identifier types (SessionId, WorkspaceId, ProjectId) used within the observation structure.
crates/ai-memory-hooks/src/router.rs	Entry point that receives raw hook payloads, creates NewObservation instances, and routes them through the sanitizer before storage.
crates/ai-memory-store/src/ops.rs	Final persistence layer that accepts only Sanitized<NewObservation> types for SQLite storage operations.

Practical Usage Example

The following pattern demonstrates how the type system enforces security:

use ai_memory_core::{NewObservation, Sanitized, Sanitizer};

// 1️⃣ Build a sanitizer from the default built‑in patterns
let sanitizer = Sanitizer::builtin();

// 2️⃣ Create a raw observation that may contain secrets
let raw_obs = NewObservation {
    session_id: SessionId::new(),
    workspace_id: WorkspaceId::new(),
    project_id: ProjectId::new(),
    kind: ObservationKind::UserPrompt,
    extension: None,
    source_event: None,
    title: "OPENAI_API_KEY=sk-REALKEY123".into(),
    body: "Read /home/user/.ssh/id_rsa".into(),
    importance: 5,
};

// 3️⃣ Convert to sanitized observation – privacy strip runs automatically
let sanitized_obs = Sanitized::new(raw_obs, &sanitizer);

// 4️⃣ The inner observation is now safe to store
let safe_obs = sanitized_obs.into_inner();
assert!(safe_obs.title.contains("[REDACTED]"));
assert!(safe_obs.body.contains("[REDACTED]"));

In step 3, Sanitized::new is the only legal way to produce the sanitized type. The Sanitizer automatically redacts recognized secret patterns and truncates oversized payloads. The resulting safe_obs contains no raw credentials and is ready for durable storage.

Summary

• Sanitized<NewObservation> serves as a compile‑time proof that an observation has been scrubbed of secrets and bounded by size limits.
• Single constructor enforcement via Sanitized::new in sanitize.rs prevents unsanitized data from reaching storage.
• Immutable trust flows through the system once the wrapper is created, eliminating the need for redundant security checks in downstream components.
• Pipeline integration ensures sanitization occurs at entry points (router) before data reaches persistence (store operations).

Frequently Asked Questions

What prevents developers from accidentally storing unsanitized observations?

The Rust type system enforces this at compile time. Storage functions in crates/ai-memory-store/src/ops.rs accept only Sanitized<NewObservation> types, not raw NewObservation instances. Because Sanitized::new is the only constructor defined in crates/ai-memory-core/src/sanitize.rs, the compiler rejects any code path that attempts to bypass the privacy strip, making accidental credential leaks structurally impossible.

How does the Sanitizer detect which content to redact?

The Sanitizer implementation applies built‑in regex patterns that recognize common secret formats including API keys (e.g., sk-*), private key headers (e.g., BEGIN RSA PRIVATE KEY), and access tokens. When Sanitized::new processes the observation, it scans both the title and body fields and replaces matching patterns with [REDACTED] markers before the data enters the wrapper.

Can the original unsanitized observation be retrieved after wrapping?

No. Once a NewObservation is wrapped by Sanitized::new, the original raw data is consumed and cannot be recovered. The API exposes only inner() and into_inner() methods, both of which return references or owned values of the already‑sanitized observation, ensuring that secret data never leaks back into the application runtime after the privacy strip completes.

Where does the sanitization occur in the request lifecycle?

Sanitization happens at the ingestion boundary in crates/ai-memory-hooks/src/router.rs, where raw webhook payloads first enter the system. The router creates NewObservation instances from external input and immediately transforms them through Sanitized::new before passing them to the storage layer. This ensures that secrets are stripped at the system perimeter, long before reaching the SQLite persistence logic in crates/ai-memory-store/src/ops.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}