How ai-memory Stores AI Agent Session Observations: SQLite Persistence Architecture

ai-memory persists every AI agent session observation in a durable SQLite database using a single-writer actor pattern, with data flowing through a type-safe sanitization boundary that strips private information before storage.

The akitaonrails/ai-memory repository implements a Rust-based persistence layer designed specifically for AI agent telemetry. Understanding how ai-memory stores AI agent session observations reveals a carefully engineered three-layer architecture that balances durability, privacy, and concurrency safety.

The Three-Layer Storage Architecture

The storage system processes every observation through distinct architectural boundaries before committing to disk.

Domain Model Definition

At the core lies the NewObservation struct, which describes the raw data that must be persisted. This domain model resides in crates/ai-memory-core/src/observation.rs and serves as the canonical representation of an observation before it enters the storage pipeline. The struct captures essential metadata including the session identifier, observation kind, content body, and importance weighting.

Sanitization Boundary

Before any data reaches the SQLite store, the hook runtime executes a privacy-scrubbing pass. The system wraps the raw NewObservation in a Sanitized<NewObservation> type, leveraging Rust's type system to guarantee that only sanitized rows can be written. This critical boundary is documented in the writer implementation at crates/ai-memory-store/src/writer.rs (lines 66-71), ensuring that sensitive information never touches the persistent storage layer.

Single-Writer SQLite Actor

All write operations flow through the Store::insert_observation async method. This method dispatches a WriteCmd::InsertObservation command to a dedicated writer thread running a single-writer actor. The writer executes individual SQLite transactions, inserting rows into the observation table while updating necessary indexes. This actor pattern, implemented in crates/ai-memory-store/src/writer.rs (lines 65-84), eliminates concurrency conflicts by serializing all mutations through a single thread.

SQLite Schema and Data Model

The observation table schema, defined by the Observation struct at lines 12-36 of crates/ai-memory-core/src/observation.rs, structures data to support multi-tenant AI agent workloads:

• id – Primary key (ObservationId) generated by the store
• session_id – Foreign key linking the observation to a specific agent session
• workspace_id and project_id – Scoping identifiers enabling multi-workspace and multi-project isolation
• kind – ObservationKind enum value (e.g., session-start, user-prompt, tool-use)
• extension and source_event – Optional namespaced metadata for extensibility
• title and body – Human-readable summary and sanitized content
• importance – Integer 1-10 weight used by downstream consolidation pipelines
• created_at – Wall-clock timestamp (jiff::Timestamp) for temporal queries

The writer executes parameterized SQL statements similar to the following pattern to ensure atomic, consistent inserts:

INSERT INTO observation (
    id, session_id, workspace_id, project_id,
    kind, extension, source_event, title, body,
    importance, created_at
) VALUES (?,?,?,?,?,?,?,?,?,?,?);

Writing Observations to the Store

Client code interacts with the storage layer through the public Store façade provided in crates/ai-memory-store/src/lib.rs. The following example demonstrates ingesting a sanitized observation from a hook runtime:

use ai_memory_core::{NewObservation, ObservationKind};
use ai_memory_store::Store;

// Build the raw observation (the hook already sanitised it)
let raw_obs = NewObservation {
    session_id,
    workspace_id,
    project_id,
    kind: ObservationKind::UserPrompt,
    extension: None,
    source_event: None,
    title: "Ask for help".into(),
    body: "How does ai‑memory store observations?".into(),
    importance: 5,
};

// The hook runtime provides `Sanitized<NewObservation>`.
// The Store API consumes the sanitized version.
let obs_id = store
    .insert_observation(sanitized_obs)
    .await
    .expect("failed to store observation");
println!("Stored observation id: {}", obs_id);

The insert_observation method guarantees durability by channeling the write through the single-writer actor, ensuring that the SQLite transaction commits successfully before returning the generated ObservationId.

Reading and Querying Observations

Retrieval operations are handled by the Reader component, which supports pagination and filtering without blocking the writer thread. The ObservationPage struct, located at line 734 of crates/ai-memory-store/src/reader.rs, provides cursor-based access to observation history:

use ai_memory_store::{Store, ObservationOrder};

let page = ObservationPage {
    session_id,
    limit: 20,
    offset: 0,
    order: ObservationOrder::Desc,
    ..Default::default()
};

let result = store
    .list_observations(page)
    .await
    .expect("failed to read observations");

for record in result.records {
    println!(
        "[{}] {} – {}",
        record.kind.as_str(),
        record.title,
        record.body
    );
}

The reader API supports filtering by kind, importance thresholds, and time ranges, enabling efficient retrieval of specific observation types for analysis or UI rendering.

Summary

• SQLite backend: ai-memory uses a durable SQLite database with a strict schema for observation storage.
• Single-writer actor: All writes serialize through a dedicated thread to prevent concurrency issues and ensure atomic transactions.
• Type-safe sanitization: The Sanitized<NewObservation> wrapper enforces privacy scrubbing at compile time before data persists.
• Structured schema: Observations link to sessions, workspaces, and projects with importance-weighted metadata for consolidation pipelines.
• Async API: The Store façade provides insert_observation for writes and list_observations for paginated reads.

Frequently Asked Questions

What database does ai-memory use for storing observations?

ai-memory uses SQLite as its persistent storage engine. The implementation leverages a single-writer actor pattern to manage concurrent access, ensuring that all observations are written atomically to the observation table while maintaining ACID compliance.

How does ai-memory handle concurrent writes to the observation store?

The system implements a single-writer actor architecture. All write operations are funneled through a dedicated writer thread that executes SQLite transactions sequentially. This design prevents race conditions and database lock contention that would occur with multiple concurrent writers, while allowing read operations to proceed in parallel through the Reader component.

What is the purpose of the Sanitized wrapper in ai-memory?

The Sanitized<T> type acts as a compile-time guarantee that sensitive data has been scrubbed before persistence. Located in the storage pipeline between the hook runtime and the SQLite writer (as noted in writer.rs lines 66-71), this wrapper ensures that the type system rejects any attempt to store unsanitized observations, enforcing privacy by default.

How are observations organized within the SQLite database?

Observations are organized in a relational schema centered on the observation table with columns for id, session_id, workspace_id, project_id, and kind. This structure enables multi-tenant isolation across different workspaces and projects while supporting efficient queries filtered by session, observation type, importance level, or creation timestamp.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}