# How ai-memory's Auto-Improvement Feature Works with LLM Providers: A Technical Deep Dive

> Explore how ai-memory's auto-improvement feature automatically refines LLM provider performance. Learn about the deterministic pipeline, wiki edits, and optional human approval for enhanced AI.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: deep-dive
- Published: 2026-09-12

---

**ai-memory's auto-improvement feature runs a deterministic background pipeline that converts completed sessions into validated wiki edits by orchestrating calls to configured LLM providers, with optional human approval gates.**

The `akitaonrails/ai-memory` repository implements a self-improving knowledge base through an LLM-driven consolidation loop. This **auto-improvement** system automatically reviews finished conversation sessions, generates structured proposals for wiki updates, and applies them only after passing rigorous validation checks. Understanding this architecture is essential for operators configuring production deployments with OpenAI, Anthropic, or other supported providers.

## The Auto-Improvement Pipeline Architecture

The feature operates as a deterministic workflow orchestrated by the `ai-memory-consolidate` crate. When the server initializes via `ai-memory serve`, it launches a periodic scheduler that claims eligible sessions and processes them through a read-only review phase before any writes occur.

### Scheduler Tick and Session Discovery

The pipeline begins with `run_auto_improve_scheduler_tick`, defined in [`crates/ai-memory-consolidate/src/auto_improve_schedule.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/auto_improve_schedule.rs). This background task periodically scans for sessions that meet three criteria:

- The session has completed and reached the minimum age threshold (`min_session_age_secs`)
- The session has not been previously reviewed by the auto-improvement loop
- An LLM provider is actively configured in the server configuration

The scheduler respects rate limits through `max_sessions_per_tick`, ensuring the system does not overwhelm the configured LLM provider or local storage.

### Read-Only LLM Review

Once a session is claimed, the system invokes `run_auto_improve_review` from [`crates/ai-memory-consolidate/src/auto_improve.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-consolidate/src/auto_improve.rs). This function loads the session's observations, projects them into a bounded prompt context, and calls the LLM via `complete_structured_with_operation_id` (implemented in [`crates/ai-memory-llm/src/provider.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-llm/src/provider.rs)).

The LLM returns structured proposals containing:
- Target wiki page paths
- Operation types (create or patch)
- Confidence scores
- Evidence quotes from the source session

This phase is strictly **read-only**; no modifications touch the durable wiki store until validation completes.

## Validation and Safety Mechanisms

Before any proposal reaches the wiki, it must survive a multi-layer validation gate designed to prevent hallucinated or low-quality edits from corrupting the knowledge base.

### Schema and Confidence Validation

Each proposal undergoes automated checks against:

- **Schema correctness**: Ensuring the structured output matches the expected proposal format
- **Confidence floor**: Validating against `DEFAULT_AUTO_IMPROVE_MIN_CONFIDENCE` (configurable via [`ai-memory.toml`](https://github.com/akitaonrails/ai-memory/blob/main/ai-memory.toml))
- **Size caps**: Enforcing maximum token limits for both input context and generated edits
- **External evaluation**: Optional gating through external evaluators if configured

Invalid proposals are rejected immediately and logged for debugging, while accepted proposals enter the staging phase.

### Staging Pending Writes

Approved proposals are written to `_pending/auto-improve/` as human-readable side-car files. According to the implementation in [`auto_improve.rs`](https://github.com/akitaonrails/ai-memory/blob/main/auto_improve.rs), these side-cars contain:

```toml

# Example side-car structure

target_path = "concepts/architecture.md"
operation = "patch"
confidence = 0.94
evidence = ["quote from session...", "supporting context..."]
body = "### New Section\nProposed content..."

```

This staging approach ensures that all pending modifications are inspectable before final application, supporting both automated and manual review workflows.

## Configuration and Approval Modes

The `ai-memory` server exposes granular control over the auto-improvement behavior through the `[auto_improve]` section of [`ai-memory.toml`](https://github.com/akitaonrails/ai-memory/blob/main/ai-memory.toml), allowing operators to balance automation with oversight.

### Automatic vs. Manual Approval

By default, staged proposals auto-approve through the standard wiki write path, instantly updating durable pages. Operators requiring human oversight can disable this behavior:

```toml
[auto_improve]
require_approval = true

```

When enabled, proposals remain in `_pending/auto-improve/` until explicitly approved via the pending-writes UI or CLI:

```bash
ai-memory pending-writes list
ai-memory pending-writes approve <proposal-id>

```

### Tuning Limits and the Experience Pass

Advanced configuration options include:

- `max_input_tokens`: Limits context window usage per review
- `max_edits_per_proposal`: Caps the number of distinct wiki modifications per session
- `rejection_buffer_size`: Controls memory of failed proposals to avoid repetition

Additionally, an optional **experience pass** (configured under `[auto_improve.experience]`) enables cross-session synthesis. This secondary LLM-driven process identifies higher-level patterns across multiple completed reviews, generating meta-insights that improve future proposal quality.

## CLI and Programmatic Usage

The auto-improvement system exposes both command-line interfaces for operators and Rust APIs for custom integrations.

### Manual Review Commands

Trigger one-off reviews without waiting for the scheduler:

```bash

# Review the most recent unprocessed session

ai-memory auto-improve

# Force review of a specific session

ai-memory auto-improve --session-id <uuid>

```

### Rust Integration

Embed the review logic directly in Rust applications:

```rust
use ai_memory_consolidate::auto_improve::{run_auto_improve_review, AutoImproveReviewConfig};

let config = AutoImproveReviewConfig::default();
let report = run_auto_improve_review(
    &reader, 
    &llm_provider, 
    session_id, 
    &config
).await?;

```

The `report` object contains details of generated proposals, validation results, and staging paths. Database schema migrations for tracking runs are maintained in [`crates/ai-memory-store/migrations/V42__auto_improve_proposal_author.sql`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/migrations/V42__auto_improve_proposal_author.sql).

## Summary

- **ai-memory's auto-improvement** requires an active LLM provider configuration to function; without it, the scheduler remains dormant.
- The pipeline follows a strict **read-then-write** architecture: sessions are reviewed, proposals are validated, and only then are changes staged or applied.
- Safety mechanisms include **confidence floors**, schema validation, and configurable **manual approval gates** to prevent low-quality automated edits.
- Operators control behavior through [`ai-memory.toml`](https://github.com/akitaonrails/ai-memory/blob/main/ai-memory.toml) settings under `[auto_improve]` and `[auto_improve.scheduler]`, including the optional **experience pass** for cross-session learning.
- All pending modifications are inspectable as side-car files in `_pending/auto-improve/` before final application.

## Frequently Asked Questions

### What triggers the ai-memory auto-improvement loop?

The loop triggers when `ai-memory serve` starts and initializes `run_auto_improve_scheduler_tick`. This background task periodically scans for completed sessions that meet the `min_session_age_secs` threshold and have not been previously reviewed. The system only activates if an LLM provider is configured in the server settings.

### Which LLM providers does ai-memory support for auto-improvement?

ai-memory supports any provider implementing the abstraction in [`crates/ai-memory-llm/src/provider.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-llm/src/provider.rs), including OpenAI, Anthropic, and local models. The specific provider is configured server-side, and the auto-improvement system invokes it through `complete_structured_with_operation_id` to generate wiki edit proposals.

### How can I prevent automatic application of AI-generated edits?

Set `require_approval = true` in the `[auto_improve]` section of your [`ai-memory.toml`](https://github.com/akitaonrails/ai-memory/blob/main/ai-memory.toml) configuration. This forces all LLM proposals into the `_pending/auto-improve/` directory as side-cars, requiring manual approval through the CLI (`ai-memory pending-writes approve`) or web UI before they modify the wiki.

### What happens if an LLM proposal fails validation?

Proposals that fail schema checks, fall below the `DEFAULT_AUTO_IMPROVE_MIN_CONFIDENCE` threshold, or exceed size limits are rejected immediately and never staged. The rejection is logged, and the session is marked as reviewed to prevent infinite retry loops. Operators can inspect rejection details in the server logs to tune confidence floors or prompt boundaries.