Core Architectural Invariant of ai-memory Regarding Data Storage: Markdown as the Single Source of Truth

The core architectural invariant of ai-memory regarding data storage is that markdown wiki files serve as the single source of truth, while SQLite operates strictly as a derived index with all writes serialized through a single-writer actor.

The akitaonrails/ai-memory repository implements a Git-backed knowledge management system where persistence architecture prioritizes human-readable documentation over traditional database supremacy. This design establishes a single source of truth model that fundamentally separates authoritative data storage from performance optimization layers. Understanding this invariant is essential for developers extending the system or troubleshooting concurrency issues.

The Single Source of Truth Principle

At the heart of ai-memory's storage layer lies an immutable hierarchy: markdown wiki files are the authoritative source for all knowledge. Every piece of persisted data originates from these version-controlled markdown files, which reside in a standard Git repository. According to the architecture documentation in docs/ARCHITECTURE.md (lines 21-29), the system explicitly designates the markdown wiki as "the source of truth," ensuring that all information remains human-editable and portable.

SQLite functions solely as a derived read-only index that mirrors the wiki contents for fast search and retrieval operations. The database does not hold independent truth; instead, it is reconstructed entirely from the markdown wiki during re-indexing operations. This relationship means the SQLite store can be deleted and regenerated without data loss, provided the markdown files remain intact.

Serialized Writes Through a Single-Writer Actor

To maintain consistency between the authoritative markdown files and the derived SQLite index, all mutations are funneled through a single-writer actor. This architectural constraint guarantees serialized write operations, eliminating "database is locked" race conditions that typically plague concurrent SQLite access. As documented in docs/ARCHITECTURE.md (line 489), the cross-cutting invariants section emphasizes this single-writer SQLite actor as a fundamental system guarantee.

The implementation in crates/ai-memory-store/src/ops.rs (line 4) reinforces this pattern through explicit writer thread constraints, ensuring that only one actor processes write operations at any given time. This serialization point creates a clear sequence: markdown files are persisted first, and the SQLite index updates occur afterward through the actor's queue.

Code Implementation and File References

Wiki Layer

The wiki crate maintains the source of truth contract in crates/ai-memory-wiki/src/wiki.rs (line 1559), where page write operations explicitly treat markdown persistence as the definitive action. When content changes, the system writes to the markdown file system first before notifying downstream indexing components.

Storage Layer

The storage crate in crates/ai-memory-store/src/ops.rs (line 4) implements the single-writer invariant, ensuring SQLite operations never conflict. This design prevents write contention by queuing all database modifications through the dedicated actor thread.

Write Operation Flow Example

The following Rust pseudocode illustrates the invariant in practice, showing how the system prioritizes markdown persistence over database updates:

// Adding a new page flows through the single-writer architecture
// Step 1: Persist to the authoritative markdown source
wiki::write_page(path, markdown_content)?;   // Markdown remains the source of truth

// Step 2: Queue the derived index update through the single-writer actor
store::queue_write(page_id, markdown_content)?; // Serialized SQLite update

This sequence ensures that even if the SQLite update fails or the application crashes, the authoritative data remains safe in the markdown files. The system can always reconstruct the database state from the wiki contents, but never the reverse.

Summary

• Markdown wiki files are the single source of truth for all persisted knowledge in ai-memory, stored as version-controlled plain text.
• SQLite serves strictly as a derived index rebuilt from markdown during re-indexing, holding no independent state.
• All writes serialize through a single-writer actor to prevent race conditions and guarantee consistency between the wiki and the database index.
• The architecture separates authoritative storage (markdown/Git) from performance optimization (SQLite), enabling human-editable documentation to drive the system.

Frequently Asked Questions

Why does ai-memory treat markdown as the source of truth instead of SQLite?

Treating markdown files as the authoritative source ensures all knowledge remains human-readable, version-controlled through Git, and editable with standard tools. This design prevents vendor lock-in to a binary database format and allows the SQLite layer to function purely as a disposable performance cache that can be regenerated from the markdown wiki at any time.

How does the single-writer actor prevent database locking issues?

The single-writer actor enforces serialized access to the SQLite database by funneling all write operations through a single thread. As implemented in crates/ai-memory-store/src/ops.rs, this pattern eliminates concurrent write attempts that would trigger SQLite's "database is locked" errors, ensuring atomic updates between the markdown source and the derived index.

What happens to the SQLite database during re-indexing operations?

During re-indexing, the system rebuilds the entire SQLite database from scratch by parsing the markdown wiki files. Because the database is a derived artifact rather than the source of truth, it can be safely dropped and regenerated without data loss, synchronizing the search index with the current state of the markdown repository.

Can applications write directly to the SQLite database to bypass the markdown layer?

No. Writing directly to SQLite would violate the core architectural invariant. Since the SQLite store is a derived index that mirrors the markdown wiki, direct mutations would create divergent state that would be overwritten during the next re-indexing operation. All changes must flow through the wiki layer to ensure the markdown files remain the definitive record.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}