How to Identify Decay Candidates and Trigger Forget Sweeps in ai-memory

Decay candidates are identified by calculating a retention score for each wiki page and filtering for scores below the configurable cold threshold, while forget sweeps are triggered manually via CLI, automatically by hourly scheduled jobs, or adaptively when tombstone counts exceed budget limits.

The akitaonrails/ai-memory repository implements an intelligent content lifecycle management system for AI memory stores. Understanding how to identify decay candidates and trigger forget sweeps allows you to reclaim storage from stale wiki pages while preserving high-value content through configurable retention policies.

How Decay Candidates Are Identified

Querying the Pages Table

The identification process begins in crates/ai-memory-store/src/reader.rs where the Reader::decay_candidates method executes a SELECT query against the pages table. This query retrieves every page for the current workspace and project, including the pinned flag, salience, updated_at, last_accessed_at, and access_count fields.

Computing Retention Scores

Each page receives a retention score calculated by the pure function retention_score in crates/ai-memory-store/src/decay.rs. This function applies an exponential decay formula defined by DecayParams, incorporating the page's age, access count, days since last access, and optional salience values. The default parameters create approximately a 35-day half-life for the time-decay term.

Applying the Cold Threshold Filter

The system filters candidates by comparing scores against params.cold_threshold. A page becomes a decay candidate only if retention_score < params.cold_threshold and the page is not pinned (pinned == false). Pinned pages, such as slots or manually protected entries, remain immune to decay regardless of their score. The method returns a vector of DecayCandidate structs containing the page ID, path, computed score, and pinned status.

What Triggers Forget Sweeps

Manual CLI Command

The primary manual trigger is the ai-memory forget-sweep command implemented in crates/ai-memory-cli/src/commands/forget_sweep.rs. This command explicitly runs the sweep for the current project and prints statistics about identified candidates and completed deletions.

Scheduled Background Jobs

The auto-improve scheduler in crates/ai-memory-consolidate/src/auto_improve_schedule.rs launches periodic retention jobs. By default, this calls store.run_forget_sweep() every hour, enabling automated maintenance without manual intervention.

Admin HTTP Endpoint

For remote administration, the MCP server exposes the /admin/forget-sweep endpoint in crates/ai-memory-mcp/src/routes.rs. This admin-only route forwards requests to the store method, allowing integration with external automation systems.

Adaptive Tombstone Triggers

When ops::soft_delete_for_decay_if_latest in crates/ai-memory-store/src/ops.rs processes a decay candidate, it may schedule an immediate sweep if the number of pending tombstones exceeds a configurable budget. This adaptive approach prevents unbounded growth of soft-deleted records.

The Sweep Execution Process

During a sweep, the system executes three atomic phases within a single writer transaction. First, store.reader.decay_candidates collects eligible pages. Second, ops::soft_delete_for_decay_if_latest marks each candidate by setting the superseded_at column, creating a decay tombstone while preserving ancestry for potential rollback. Finally, after hard_delete_after_days (default 180 days), ops::hard_delete_decayed_page_chain purges old tombstones and their underlying markdown files.

Configuration and Usage Examples

Running a Manual Sweep

cargo run --bin ai-memory -- forget-sweep --project my-proj

Inspecting Candidates Programmatically

use ai_memory_store::{Store, DecayParams};

#[tokio::main]
async fn main() {
    let store = Store::open("data/ai-memory.db").await.unwrap();
    let ws = store.workspace_id("default").await.unwrap();
    let proj = store.project_id("my-proj").await.unwrap();

    let candidates = store.reader.decay_candidates(ws, proj).await.unwrap();
    for c in candidates {
        println!("{} – score {:.3}", c.path, c.retention_score);
    }
}

Tuning Decay Parameters

# .ai-memory.toml

[decay]
lambda = 0.03
cold_threshold = 0.15
hard_delete_after_days = 90

Key Implementation Files

• crates/ai-memory-store/src/decay.rs – Defines DecayParams and retention scoring formulas
• crates/ai-memory-store/src/reader.rs – Implements decay_candidates and decay_tombstones_before queries
• crates/ai-memory-store/src/ops.rs – Contains soft_delete_for_decay_if_latest and hard_delete_decayed_page_chain
• crates/ai-memory-cli/src/commands/forget_sweep.rs – CLI entry point
• crates/ai-memory-consolidate/src/auto_improve_schedule.rs – Hourly sweep scheduler
• crates/ai-memory-mcp/src/routes.rs – Admin HTTP endpoint

Summary

• Decay candidates are identified by calculating retention scores using exponential decay formulas in crates/ai-memory-store/src/decay.rs
• Pages with scores below cold_threshold that are not pinned become eligible candidates
• Forget sweeps trigger via CLI commands, hourly scheduled jobs, admin HTTP endpoints, or adaptive tombstone limits
• Sweeps execute as atomic transactions combining soft deletes and hard deletes after 180 days
• Configure decay aggressiveness through .ai-memory.toml settings like lambda and cold_threshold

Frequently Asked Questions

What prevents important pages from being marked as decay candidates?

The pinned flag in the pages table protects specific entries from decay regardless of their retention score. Additionally, high salience values (set via user feedback like "Helpful") scale the time-decay term to keep pages above the cold threshold.

How often does the automatic forget sweep run?

By default, the auto-improve scheduler in crates/ai-memory-consolidate/src/auto_improve_schedule.rs triggers a sweep every hour. This interval is configurable through the scheduler settings.

What is the difference between soft delete and hard delete in the sweep process?

Soft delete marks pages by setting the superseded_at column, creating decay tombstones that preserve page history for potential rollback. Hard delete permanently removes pages and their markdown files after exceeding hard_delete_after_days (default 180 days).

Can I adjust how aggressively ai-memory removes old content?

Yes. Modify the DecayParams in your .ai-memory.toml configuration file. Decrease cold_threshold or increase lambda to accelerate decay, or reduce hard_delete_after_days to shorten the grace period before permanent deletion.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}