How ai-memory Ensures Atomic Markdown Writes With Git Versioning

ai-memory guarantees atomic markdown writes by using a temp-file-and-rename pattern with fsync barriers, wrapped in a Git-backed persistence layer that snapshots, stages, and commits changes only after successful database transactions.

The akitaonrails/ai-memory project treats every piece of knowledge as a plain Markdown file inside a dedicated wiki directory that is itself a standard Git repository. To ensure that no write operation can leave the filesystem or the Git history in an inconsistent state—even during power loss or process crashes—the codebase implements a rigorous five-stage atomicity pipeline.

The Atomic Write Pipeline

The atomic markdown writes with git versioning strategy spans three core modules: the low-level filesystem helper, the Git persistence adapter, and the high-level wiki orchestrator. Together they enforce that every page is written exactly once, never partially, and that each successful write becomes an immutable Git commit.

Isolated Temp File Creation

In crates/ai-memory-wiki/src/atomic.rs, the write_atomic function creates a hidden tempfile with the prefix .ai-memory-tmp.* in the same directory as the target Markdown file. The content is written to this tempfile and immediately flushed to stable storage using fsync (tmp.as_file().sync_data()?).

This isolation ensures that a crash during the write cannot corrupt the existing file because the original remains untouched until the rename completes.

Atomic Rename With Retry Logic

Once the tempfile is persisted, GitAdapter::persist (in crates/ai-memory-wiki/src/git.rs) executes an atomic rename operation inside a commit-lock protected critical section (self.writing()). On POSIX systems, this replaces the destination atomically; on Windows, a transient sharing-violation triggers a retry loop (persist_with_retry) to handle locks caused by antivirus software or indexing services.

After the rename succeeds, the new file is again fsync-ed (persisted.sync_data()?) to guarantee the data blocks are on disk before the operation proceeds.

Directory Sync for Durability

Following the rename, write_atomic opens the parent directory and invokes sync_all() (best-effort). This step ensures that the directory entry for the renamed file has reached stable storage, preventing a power loss from losing the new filename even if the file data was already written.

Git Commit Coordination

The Wiki::write_page method in crates/ai-memory-wiki/src/wiki.rs orchestrates the transactional boundary. It first calls persist_tmp_with_rollback_snapshot to capture a snapshot of any existing version, then writes the file atomically via crate::atomic::write_atomic. Only after the SQLite database successfully upserts the page row does GitAdapter::commit_all create a new Git commit containing the updated file.

If the database transaction fails, rollback_or_inconsistent restores the previous file from the snapshot. This ties the filesystem update and the Git commit together: the file is never left in an uncommitted state.

Serialized Access Control

GitAdapter maintains a single-writer lock (commit_lock) and stages only paths explicitly reported via mark_written. All writes to the same (workspace, project, path) tuple acquire a per-path mutex (page_locks) to prevent concurrent modifications that could corrupt the Git index or produce torn files.

Core Implementation Files

Understanding the atomic markdown writes with git versioning mechanics requires familiarity with three primary source files.

atomic.rs

Located at crates/ai-memory-wiki/src/atomic.rs, this module implements the core temp-plus-rename algorithm. The write_atomic function encapsulates the tempfile creation, fsync calls, atomic rename, and parent-directory synchronization in a single, reusable helper.

git.rs

crates/ai-memory-wiki/src/git.rs houses the GitAdapter struct, which provides the persist, write_atomic, and commit_all methods. It manages the commit_lock (a Mutex<Option<Open>>) that serializes Git repository access and the staging area that tracks which paths have been modified since the last commit.

wiki.rs

The Wiki struct in crates/ai-memory-wiki/src/wiki.rs provides the high-level write_page API. It coordinates the snapshot-and-rollback logic, ensuring that the SQLite metadata and the Git filesystem remain consistent even when errors occur mid-transaction.

Practical Usage Example

The following Rust code demonstrates how to create or update a wiki page while leveraging the atomic guarantees:

use ai_memory_wiki::{Wiki, WritePageRequest};

let wiki = Wiki::new(data_dir).await?;
let req = WritePageRequest {
    workspace_id,
    project_id,
    path: "notes/todo.md".into(),
    frontmatter: serde_json::json!({ "title": "TODO" }),
    body: "Write atomic write article".into(),
    ..Default::default()
};
let page_id = wiki.write_page(req).await?;
println!("Page stored as version {}", page_id);

Internally, the write_page flow executes these steps:

1. Snapshot the existing file (if any) for rollback.
2. Write to a tempfile with .ai-memory-tmp.* prefix.
3. fsync the tempfile.
4. Atomically rename the tempfile to the destination.
5. fsync the renamed file and parent directory.
6. Upsert the metadata in SQLite.
7. Commit the change to Git, or rollback if step 6 fails.

Summary

• Temp-file isolation: Writes occur in .ai-memory-tmp.* files that are renamed only after successful fsync, ensuring the original file cannot be corrupted by a partial write.
• Cross-platform atomicity: The rename operation is atomic on POSIX and uses a retry loop (persist_with_retry) on Windows to handle transient locking.
• Directory durability: sync_all() on the parent directory guarantees that the new filename survives a system crash.
• Transactional integrity: Wiki::write_page snapshots the previous version and only commits to Git after the database transaction succeeds, with automatic rollback on failure.
• Serialized concurrency: Per-path mutexes (page_locks) and a global commit_lock prevent race conditions during concurrent writes.

Frequently Asked Questions

What happens if the system crashes during a write operation?

If a crash occurs while writing to the .ai-memory-tmp.* tempfile, the original Markdown file remains untouched because the rename has not yet occurred. The temporary file may be orphaned, but it will not interfere with the existing data. Upon restart, the next write operation will simply create a new tempfile.

How does ai-memory prevent concurrent writes to the same page?

The GitAdapter maintains per-path mutexes (page_locks) that serialize access to specific (workspace, project, path) combinations. Additionally, a global commit_lock ensures that only one thread can stage and commit changes to the Git repository at a time, preventing index corruption.

Why does the code synchronize the parent directory after renaming the file?

Calling sync_all() on the parent directory forces the filesystem to flush the directory entry metadata to stable storage. Without this step, a power loss immediately after a successful file rename could theoretically lose the new filename even though the file data was written, leaving the repository in an inconsistent state.

Can I recover a previous version if the database transaction fails after the file is written?

Yes. The persist_tmp_with_rollback_snapshot method creates a backup of the existing file before the atomic write. If the subsequent SQLite upsert fails, the rollback_or_inconsistent function automatically restores the previous file content from the snapshot, ensuring the filesystem matches the Git state prior to the attempted change.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}