What Is a Decay Tombstone in ai‑memory’s Eviction Process?

A decay tombstone is a soft‑delete marker that preserves a wiki page’s database record after its content file is removed, enabling safe ancestry tracking and delayed permanent deletion during the memory‑forget sweep.

In the akitaonrails/ai‑memory repository, the eviction system relies on this tombstone mechanism to balance storage constraints with data integrity. When the retention engine identifies a page as “cold” due to a low salience score, it initiates a two‑phase deletion process that leaves a temporary trace in the SQLite pages table.

How the Decay Tombstone Mechanism Works

The eviction flow splits deletion into distinct soft‑ and hard‑delete phases to prevent broken backlinks and allow for audit trails.

Phase 1: Soft Deletion Creates the Tombstone

When a page’s relevance drops below the configured threshold, the system deletes the markdown file from the wiki filesystem but does not remove the corresponding row in the database. Instead, the soft_delete_for_decay_if_latest operation in crates/ai‑memory‑store/src/ops.rs converts the active record into a tombstone by setting is_latest = false and populating the superseded_at column with the current timestamp source.

This lightweight update preserves the row’s primary key and ancestry references without touching the filesystem, ensuring that existing backlinks remain valid during a configurable grace period.

Phase 2: Hard Deletion by the Forget Sweep

After the number of days specified in hard_delete_after_days elapses, the periodic memory_forget_sweep job invokes hard_delete_decayed_page_chain to recursively purge the tombstone and its entire version history. This operation, found in crates/ai‑memory‑store/src/ops.rs, permanently deletes the database rows and reclaims disk space source.

The sweep logic is orchestrated from crates/ai‑memory‑wiki/src/wiki.rs, which coordinates the transition from soft to hard deletion states source.

Design Rationale for Tombstone‑Based Eviction

The ai‑memory architecture uses tombstones to satisfy two critical operational requirements:

• Ancestry Preservation: By retaining the row with is_latest = false, the system maintains the version chain (parent‑child relationships) so that historical references do not dangle during the grace period.
• Configurable Retention: The hard_delete_after_days setting provides a safety buffer, allowing administrators to recover recently evicted content or audit deletions before the hard_delete_decayed_page_chain operation makes them irreversible.

This approach is documented in the project’s design specifications, which describe the tombstone as a temporary “deleted‑but‑still‑recorded” state necessary for safe eviction source.

Code Example: Tombstone Lifecycle

The following Rust snippets demonstrate the core operations that manage decay tombstones during the eviction pipeline:

use ai_memory_store::ops::{self, DecayParams};
use ai_memory_store::models::PageId;

// 1. Create a decay tombstone (soft delete)
// Called when salience score falls below threshold
fn evict_page(conn: &mut SqliteConnection, page_id: PageId) -> Result<(), StoreError> {
    let params = DecayParams::default();
    ops::soft_delete_for_decay_if_latest(
        conn,
        &params,
        page_id,
        true, // is_latest flag
    )?;
    // Row now has is_latest = false and superseded_at = now()
    Ok(())
}

// 2. Permanent removal by the background sweep
// Executed after hard_delete_after_days have passed
fn purge_tombstone(conn: &mut SqliteConnection, page_id: PageId) -> Result<(), StoreError> {
    ops::hard_delete_decayed_page_chain(
        conn,
        page_id,
        i64::MAX, // Ignore age checks, force deletion
    )?;
    Ok(())
}

These functions are typically invoked by the automated sweep job defined in crates/ai‑memory‑consolidate/src/sweep.rs, rather than by application code directly.

Summary

• A decay tombstone is a database state marked by is_latest = false and a superseded_at timestamp, created when a wiki page’s markdown file is evicted.
• The soft_delete_for_decay_if_latest function creates the tombstone without filesystem operations, preserving the ancestry chain.
• The hard_delete_decayed_page_chain function permanently removes the tombstone after the configurable hard_delete_after_days grace period expires.
• This two‑phase eviction process ensures backlink integrity and provides an audit window before irreversible deletion.

Frequently Asked Questions

What triggers the creation of a decay tombstone?

The retention engine triggers tombstone creation when a page’s salience score drops below the configured eviction threshold during the memory‑forget sweep. At that point, the system deletes the page’s markdown file and calls soft_delete_for_decay_if_latest to mark the database row as superseded source.

How long does a decay tombstone persist before hard deletion?

A tombstone persists for the number of days specified by the hard_delete_after_days configuration parameter. After this grace period, the memory_forget_sweep background job invokes hard_delete_decayed_page_chain to permanently purge the record and its ancestry chain source.

What is the difference between soft_delete_for_decay_if_latest and hard_delete_decayed_page_chain?

soft_delete_for_decay_if_latest performs a metadata update that sets is_latest = false and records the superseded_at timestamp, creating the tombstone while preserving the row. In contrast, hard_delete_decayed_page_chain executes a destructive SQL delete that recursively removes the tombstone and all its historical versions, freeing the primary keys and storage source.

Why does ai‑memory retain the database row after deleting the file?

Retaining the row prevents immediate breakage of backlinks and preserves the version ancestry chain. This design allows the system to maintain referential integrity during the grace period and provides an audit trail for recently evicted content before the hard_delete_decayed_page_chain operation makes the deletion permanent source.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}