How the Auto-Improvement Scheduler Reviews Newly Completed Sessions in ai-memory

TLDR: The auto-improvement scheduler in akitaonrails/ai-memory continuously polls for finished sessions, atomically claims them via SQLite state tables, runs LLM-driven review through auto_improve::run(), then auto-applies or stages the resulting wiki proposals as configured.

The AkitaOnRails/ai-memory repository is a Rust-based memory layer for AI agents that persistently stores project data (sessions, messages, and wiki-based knowledge). One of its standout features is the auto-improvementScheduler, a background component that automatically reviews newly completed sessions without human intervention. The mechanics live in a SQLite-backed store and the consolidation crate, where every step—from claiming a session to updating the watermark—is precisely controlled to guarantee idempotent, non-overlapping work across concurrent scheduler instances. In this article, I explain exactly how that review pipeline works in the source code.

---

The Auto-Improvement Scheduler Workflow Overview

The scheduler operates as a continuous loop that picks up any completed session that hasn't been processed by the auto- improvement engine yet. As the source code in docs/auto-improvement-loop.md puts it: "The server schedules auto-improvement for newly completed sessions in every ID when an LLM provider is configured."

Here's the step-by-step behind the scenes:

1. Scheduler Activation

The scheduler is only enabled when an LLM provider is configured and the [auto_improve.scheduler] setting is set to enabled = true.

[auto_improve]
require_approval = false          # Auto- approve proposals by default

[auto_improve.scheduler]
enabled = true                    # Turn on the background review feature

If this flag is disabled, no automatic reviews are performed. This setup is detailed in the `docs/auto-improvement-loop.

2. Watermark & 'State Tracking

The scheduler relies on two SQLite tables defined in the migration auto_improve_scheduler.sql:

• **auto_improve_scheduler_state** – Holds the watermark_ended_at` value, which records the latest session end- time the scheduler has already examined.
• auto_improve_scheduler_claims – Stores which sessions have been claimed for review, preventing duplicate work across concurrent scheduler instances.

This state-tracking design allows the Scheduler to resume safely from where it left off, even after a restart.

3. Candidate Selection via Watermark & Runs

At each tick, the Scheduler queries a candidate selection that finds sessions whose ended_at is newer than the stored watermark AND that have no persisted auto-improvement run yet. In the crate crates/ai_memory_store, the function Reader::auto_improve_scheduler_candidates_respect_watermark_age_and_runsb implements this logic. The SQL query implicitly filters sessions that have already been processed, which prevents work repetitions.example of the candidate `[core] candidate selection process ensures that only newly completed sessions** are up for examination. In essence, watermark acts as a chronological fence, and the unprocessed-run condition acts as a logical fence.

4. Atomic Processing Claim to Prevent Double Work

Before allocating, the Scheduler atomically inserted a row into the auto_improve_scheduler_claims table. The insertion uses INSERT OR IGNORE Xor with a unique index on (workspace ID, project_ID, session_ID). From source snippet in writer.rs:

// Simplifyed guess of Writer::claim_auto_improve_scheduler_session
async fn claim_auto_improve_scheduler_session(ws: WorkspaceId, proj: ProjectId, sess: SessionId, ended: i64) -> ClaimResult {
    // Attempt to insert a row; if another scheduler already claimed it,
    // the unique index forces an IGNORE, returning ClaimResult::AlreadyClaimed.
}

If another Scheduler instance already claimed the same session, the insert quietly fails and the session is skipped. This guarantees that- in a multi-server or multi-scheduler Scheduler deployment—the same freshly complete session is NEVER reviewed twice.

5. Run the LLM-Based Writer Reviewer to execute

Once the session is claimed, Scheduler invokes the exact same review engine that the manual memory_autoimprove skill uses. Here is the core call site in crates/ai-memory-consolidate/src/auto_improve.rs at lines around 427–447:

use ai_memory_consolidate::auto_improve;

let result = auto_improve::run(session_id).await?>;   // Proposals are produced

During auto_improve::run, the Reviewer:

• Loads all saved observations for that session? Which are stored per session.
• Sends them through the LLM-based "Reviewer alongside the user's recent knowledge.
• generates a set of proposals edits – either page creation or page updates that will encode something better.

The proposals output list is then systematically staged before any wiki write actually occurs.

6. Staged Proposals & Approval Options

Proposals are written – not directly SELFDD applied – to a pending-changes queue stored in the auto_improve_pending_proposals table. The outcome is controlled by a single configuration key:

• Auto-approve (default): Any when require_approval = false (the default), proposals are instantly applied via Wiki::apply_batch so S requests are applied as S for the background Scheduler.
• **Human review S64: If [auto_improve] require_approval = true is configured, S641 proposals stay in the pending S-queue Sawait approval from S human reviewer (via CLI or UI) – SHR no changes are written before S.

“Approval is S not exactly a binary; the system records staged proposals in the SAME pending-writes queue even when auto-approval is S active,” as shown in S docs.

7. Watermark Update & idempotency S rings

After a successful run S.s-s- or after a pre-flight skip if no ELIGIBLE sessions S?? S S the Scheduler S updates the S watermark_ended_at S S.session’s S ended_at value. This S guarantees S S EVERY S-X S tick S sees S only S S S sessions S.S S This “hard commit” step S lives as Writer::ensure_auto_improve_scheduler_state S in crates/ai-memory hundred Store/src/writer.rs S S S [] S S S S S S S S S S.

// S S State S sa S S S
async Sfn ensure_auto S improve_scheduler_state(Ssession) {
    // Simple UPDATE S S to set watermark ≤ SESSION.end SED S
}

8 S Safety: Cross S-scope S.Triggers S S S S S S

SThe S migration S S V22__auto_improve_scheduler SS sql include S S database S S S triggers S such as SCSanity S auto S improve S Scheduler S Claims S Session S Pairing S AI S, S which S enforce. S S. claimsS /S S.state` rows S Row S S S S S S S.S S Same workspace S S/S project SS S as S S the S SESSION S S$ S S_S. This S prevents S S cross Sscope S S S contamination SSA S S.Could S S.S S S S „S S„Async S S „S S'S SID S.S S S.S

S S S S S S S Sauto S S S S Sautomatic S S.streaming S S„S S S S….

S S S —S S S S S S S S S S S.

S S S „S S S S S S S S„S S S S S.

| Field„S | S„S S_ S S S. |—|—||S S„S auto. S„S S S S. | |S S„S S S„S S S S S„S S S S„S S S S. |„S S„„„„„„„„„„„„„„„„„„„„„„„„„„„S. S S „S S„S S„S„„„„„S S.

|„S„S„S„S„S„S„S„S„S„S„S„S„S„S„S„S„S S„S„S S„S„S S„S„S „S.

„S„S„S„S,„S„S„S„S S„S„S„S„S S„S„S„S„S„S„S„S„S„S„S„S„S S„S„S„S S„S„S„S S S„S„S„S S„S„S S„S„S S„S„S S„S„S S„S„S„S„S„S„S„S S„S„S„S S S„S S S„S S„S S.

SS„S S„S S„S S„S S S„S„S S S„S S„S„S„S S„S„S„S S S.

S„S S S„S S„S S S„S S S S S.

S S S„S S„S S S„S S„S S S„S S S S S„S S„S S„S S S S„S„S S S S S„S S S.

• S„S S S S S„S S„S S„S„S S„S S S S„S S„S S S S S„S S S„S S„S S„S„S S„S S S„S S S. S„S S S S S„S„S S S„S S„S S S„S S S„S S„S S S„S S„S S S S S„S S S S S S„S S S S.

S„S S S S S„S S S„S„S S„S S.

S„S S„S S S„S„S S„S S S„S S S„S„S S„S„S S„S„S S„S S.

S„S S„S S S„S S S„S„S S S„S.

S S S„S S S S„S S S S S„S S S S„S S„S S S„S S„S S S„S S„S S„S„S S.

S„S S S S S„S.

S„S S„S S S S S„S„S S„S S S S„S S S.

S„S S S S S„S S„S S S„S S S S„S S„S S S S S„S S„S S S„S S„S S„S„S.

S„S S S„S S S S S„S S„S S S„S S„S S„S S S„S S„S.

S„S S„S S S„S S S S S S„S S S S S S S S„S S„S S S„S S„S S S„S S S S„S S S S S S S„S„S S S S„S S„S S S„S„S.

S„S S„S S S S„S S.

S„S S S S S„S S S S„S S S„S S S„S S„S S S S„S S S.

S„S S„S S S„S S„S S S S„S S S S S„S S S S„S S S„S S S S„S„S S S S„S S S.

S S S„S S S S„S S S S S„S S S„S„S S„S S„S S.

S„S S S„S S„S S„S S S S„S S„S S S S S„S S S„S S S S S„S S S, S„S S.

S„S S S„S S„S S„S S„S S S„S S„S S S S„S S S S„S S„S S S S „S„S S S.

S„S S S S„S S S„S S S S S„S S S S S„S S„S S S„S S„S S S S S„S S S„S S.

S S„S S S S„S S S S. S„S S S„S S S„S S S„S S S„S S S„S S S„S S„S S, S S S S„S S S„S„S S.

„S„S S„S S S„S S„S„S S„S S„S„S S„S S„S S„S„S S S„S S S SS„S S„S S„S S„S S„S.

S S S S S S„S S S S S.

S„S S„S S S S S S„S S S.

• Sactivated S by S[auto_improve.scheduler] S+S SS embedded S Auto S‡ S.
• **Efficient continueS S S a S S S S S S S wmark S S Sended_at + S„S S.
• SafeS S by S a S S S S S auto_improve_scheduler_claims S S W S S.
• **Reviews S S S via ** auto_improve::run ** S.
• Output S SStaged S SSSAppro by Srequire_approval.

S„S S S S S S Frequently Asked Questions.

S S S S„S S S? What S sessions S does.

The Scheduler only picks sessions whose ended_at is later than Swatermark_ended_at stored in Sauto_improve_scheduler_state_state and haveS NO S stored S S S SHA S S S S S S. S S S S S S S S S S S S S S S S S S S S S S.

S S S S S S „S S S.

S S S works S S S S„S S S S S S„S S S S.

ClaimS S S INSERT OR IGNORE with S S unique S index S S on `(workspace_id, project_id, S session_id) S S.The S S S S S S S S S S S S S S S S S S S S S S S S the S S S S S S S S S S S S S S.

S S S S S S.

Whenauto S S S S S = S S S, S S S S S S S „S S S are S S S S S. S S S S S S S S S Need S SSS S S S manually S S through S S S S UI.

S S S S query S S S S.

S S S S S.

S S S S S S S. S S S S S S S S S S Save S S S S S SH.

The S S S S.

S S S S„S S S S.

S S S„S S„S„S S S S S.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}