How ai-memory Handles Embedding Configuration Changes and Stale Vectors

ai-memory detects stale vectors by comparing stored embedding metadata triples (provider, model, dimension) against the current runtime configuration, filters mismatched rows from hybrid search results, and provides automated cleanup through the embed --force CLI command to delete and regenerate vectors.

The akitaonrails/ai-memory repository implements a robust versioning strategy for vector embeddings to ensure hybrid search accuracy when providers or models change. When you modify embedding settings—such as switching from OpenAI to a local model or adjusting vector dimensions—the system must identify and exclude outdated vectors to prevent ranking degradation. This article examines the source code mechanisms that detect, isolate, and remediate stale embeddings through metadata tracking and scheduled back-filling.

Tracking Embedding Configuration with Metadata Triples

The (provider, model, dim) Signature

Every vector stored in ai-memory is accompanied by a configuration triple identifying its provenance. According to docs/ARCHITECTURE.md, each embedding row contains the vector plus the specific provider, model name, and dimensionality used during generation. This metadata allows the system to determine whether a stored vector matches the current configuration defined in ai-memory.toml.

Schema Support in the Pages Table

Schema migration M9 added three columns to the pages table: embedding_provider, embedding_model, and embedding_dim. These columns enable direct SQL queries to identify which pages have vectors matching the active configuration and which contain stale data. The health endpoint (/api/v1/health) leverages this metadata to surface diagnostics about missing or outdated embeddings.

Runtime Detection and Filtering of Stale Vectors

Hybrid Search Filtering

When executing hybrid search queries, ai-memory automatically excludes vectors with mismatched configuration triples. The query layer applies a WHERE provider = ? AND model = ? AND dim = ? clause to the page_embeddings table, ensuring that only vectors generated with the current provider, model, and dimension settings participate in similarity calculations.

Diagnostic Warnings

As documented in docs/design-decisions.md, the storage layer emits a warning the first time it encounters a stale row during a request, then silently ignores subsequent mismatches for the duration of that operation. This prevents query failures while maintaining visibility into configuration drift through application logs.

Cleanup and Re-embedding Strategies

Bulk Deletion of Stale Embeddings

The Writer::delete_stale_page_embeddings function in crates/ai-memory-store/src/writer.rs executes a scoped SQL DELETE statement to remove outdated vectors:

DELETE FROM page_embeddings 
WHERE provider != ? OR model != ? OR dim != ?
-- Scoped to current workspace/project

This deletion runs automatically during the scheduled back-fill pass or when triggered manually via the CLI.

Forcing Vector Regeneration

After deleting stale rows, the system recomputes embeddings using the new configuration. The ai-memory embed --force command initiates this workflow by first invoking the stale deletion routine, then processing all pages lacking current embeddings. This command is available via the CLI entry point in crates/ai-memory-cli/src/commands/embed.rs.

Example usage:

# Delete stale vectors and regenerate with current config

ai-memory embed --force

Or programmatically in Rust:

use ai_memory_cli::commands::embed::EmbedCommand;

let cmd = EmbedCommand {
    force: true,        // Enable stale cleanup
    project: None,
    workspace: None,
    parallelism: 8,
};
cmd.run().await?;

Configuration Updates

To trigger the stale vector workflow, update your ai-memory.toml:

[embedder]
provider = "openai"
model = "text-embedding-3-large"
dim = 3072  # New dimension triggers mismatch detection

Summary

• ai-memory stores embedding metadata triples (provider, model, dimension) alongside every vector to track configuration provenance.
• Runtime filtering automatically excludes stale vectors from hybrid search results by comparing stored triples against the current configuration.
• The delete_stale_page_embeddings method in crates/ai-memory-store/src/writer.rs performs bulk cleanup of mismatched rows using scoped SQL deletions.
• The embed --force command automates deletion and back-filling of vectors after configuration changes.
• Database schema M9 adds columns to the pages table to support health diagnostics and traceability of embedding states via the /api/v1/health endpoint.

Frequently Asked Questions

How does ai-memory know which vectors are stale?

ai-memory compares the stored (provider, model, dim) triple in each embedding row against the current configuration loaded from ai-memory.toml. Rows with mismatched values are considered stale and filtered out during hybrid search queries using a WHERE clause that enforces exact matches on all three metadata fields.

Will stale vectors cause search errors or just reduced accuracy?

Stale vectors trigger a warning log on first encounter but are silently ignored during the request to prevent failures. They do not participate in similarity rankings, ensuring search accuracy remains high while the system awaits re-embedding via the scheduled or manual back-fill process.

How do I remove stale vectors after changing my embedding model?

Run the CLI command ai-memory embed --force or manually invoke Writer::delete_stale_page_embeddings followed by the embed job. This deletes vectors with mismatched configuration triples and regenerates them using the new settings defined in your TOML configuration.

Can I check how many pages have stale embeddings?

Yes. Query the pages table for rows where embedding_provider, embedding_model, or embedding_dim differ from your current configuration, or check the /api/v1/health endpoint which surfaces diagnostics about embedding coverage and staleness counts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}