How ai-memory Ensures Atomicity in File Writes: The Temp-Rename Pattern Explained

ai-memory guarantees atomicity in file writes by implementing a three-step "write to temp, fsync, then rename" algorithm in crates/ai-memory-wiki/src/atomic.rs, ensuring that readers never see partially written files even during system crashes.

The akitaonrails/ai-memory repository implements a crash-safe wiki subsystem where every markdown mutation must survive unexpected power losses or process terminations. By channeling all filesystem operations through a single atomic write path, the system prevents data corruption while maintaining consistency between on-disk files and the SQLite index.

The Three-Step Atomic Write Algorithm

At the heart of ai-memory’s durability guarantees lies the write_atomic function defined in crates/ai-memory-wiki/src/atomic.rs (line 102). This routine implements the standard POSIX atomic file replacement pattern, ensuring atomicity in file writes through a carefully sequenced temp-file dance.

Step 1: Write to a Temporary File

The process begins by creating a sibling temporary file in the same directory as the target path. The full byte slice of content is written to this temporary file rather than the destination directly. This isolation prevents readers from encountering partial data if the writer process crashes mid-operation.

Step 2: fsync for Durability

After writing the bytes, the function calls fsync on the file descriptor. This forces the operating system to flush the data from page cache to persistent storage, ensuring the temporary file contents survive a power failure before any rename occurs.

Step 3: Atomic Rename

Finally, std::fs::rename moves the temporary file onto the target path. On POSIX-compatible filesystems, this rename operation is atomic—observers see either the old file or the new file, never a partially written state. This single syscall provides the foundational guarantee that ai-memory pages are either fully persisted or left untouched.

Integration with the Wiki Subsystem

The atomic write primitive serves as the foundation for higher-level operations. Two key components delegate to this helper:

• GitAdapter::write_atomic in crates/ai-memory-wiki/src/git.rs (lines 512–514) provides a thin wrapper that forwards calls to the underlying atomic routine.
• Wiki::write_page in crates/ai-memory-wiki/src/wiki.rs (lines 680–681) invokes self.git.write_atomic(&abs, raw.as_bytes()) after collecting front-matter and body content, ensuring every page update follows the crash-safe path.

These layers ensure that whether you interact with the raw Git adapter or the high-level Wiki API, all writes respect the atomicity contract.

Concurrency Control and Lock Hierarchy

Atomic file writes solve the crash-safety problem, but atomicity in file writes also requires coordination among concurrent threads. The wiki layer implements a two-tier locking strategy defined in crates/ai-memory-wiki/src/wiki.rs:

• mutation_lock (lines 166–172): A shared read/write lock (Arc<RwLock<()>>) that serializes write operations globally when necessary.
• page_locks (lines 217–224): A per-page async mutex that prevents interleaved renames and SQLite upserts for the same logical page.

This hierarchy ensures that even with multiple concurrent writers, the sequence of filesystem rename and database update remains uninterruptible. The combination of filesystem-level atomic renames and application-level mutexes guarantees that the on-disk markdown and the SQLite store remain perfectly aligned.

Practical Usage Examples

Direct Atomic Write

For low-level operations, you can invoke the atomic write routine directly:

use ai_memory_wiki::write_atomic;
use std::path::Path;

let path = Path::new("/tmp/wiki/project/notes.md");
let content = b"# Title\n\nSome markdown body.";

write_atomic(path, content).expect("atomic write failed");

High-Level Wiki API

Most applications should use the Wiki struct, which handles front-matter serialization and database consistency:

use ai_memory_wiki::{Wiki, WritePageRequest};
use ai_memory_store::WriterHandle;

let wiki = Wiki::new(&std::path::Path::new("/var/lib/ai-memory"), writer)
    .expect("wiki init");

let req = WritePageRequest {
    workspace_id,
    project_id,
    path: "notes/intro.md".into(),
    title: Some("Introduction".into()),
    body: "Welcome to this project".into(),
    ..Default::default()
};

let page_id = wiki.write_page(req).await.expect("write_page failed");
println!("Page persisted with id {page_id}");

Under the hood, write_page executes the admission checks, calls git.write_atomic, and upserts the page record in SQLite within the critical section protected by the lock hierarchy.

Summary

• Atomicity in file writes is achieved through a temp-file pattern: write to sibling, fsync, then atomic rename.
• The core implementation lives in crates/ai-memory-wiki/src/atomic.rs (pub fn write_atomic).
• GitAdapter::write_atomic and Wiki::write_page provide the exclusive write paths for all markdown mutations.
• Concurrency control via mutation_lock and page_locks prevents race conditions between filesystem operations and database updates.
• The design guarantees that observers see either the complete old file or the complete new file, never partial content or desynchronized state.

Frequently Asked Questions

What happens if the system crashes during the temporary file write?

If a crash occurs before the fsync completes or before the rename executes, the temporary file remains on disk but the original target file stays untouched and intact. On restart, the stale temporary file may be overwritten by subsequent write operations, leaving the system in a consistent state with the original data preserved.

Does this atomic write pattern work on all filesystems?

The atomic rename guarantee requires a POSIX-compatible filesystem that implements atomic rename() semantics. Most modern filesystems (ext4, XFS, APFS, NTFS with specific flags) support this behavior, but network filesystems or FAT32 may not provide the same crash safety. ai-memory assumes a POSIX-compliant storage layer for its durability guarantees.

How does ai-memory handle concurrent writes to the same page?

The system uses a per-page async mutex (page_locks) and a shared mutation_lock to serialize access. When Wiki::write_page is called, it acquires the appropriate locks before performing the git atomic write and SQLite upsert, ensuring that concurrent mutations to the same path cannot interleave and corrupt the file or database state.

Where is the atomic write logic located in the codebase?

The low-level atomic write routine resides in crates/ai-memory-wiki/src/atomic.rs at line 102 (pub fn write_atomic). The wrapper used by the Git adapter is in crates/ai-memory-wiki/src/git.rs lines 512–514, while the high-level integration point is in crates/ai-memory-wiki/src/wiki.rs lines 680–681.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}