How the ai-memory Wiki Handles Atomic File Writes and Git Versioning

The ai-memory wiki guarantees crash-safe page updates by writing to temporary files before atomically renaming them over target paths, then persists every logical change to a local Git repository via a GitAdapter that lazily initializes, stages all files, and commits with a fixed author.

The wiki subsystem in the akitaonrails/ai-memory repository lives inside the ai-memory-wiki crate and persists Markdown pages to disk while maintaining a full version history. It combines filesystem-level atomic writes with SQLite upserts and Git-based snapshots to ensure that no crash or store failure can leave the database and filesystem in an inconsistent state. This design makes the wiki system both durable and fully auditable.

Atomic File Writes in ai-memory-wiki

The crate implements durable file mutations through a temporary-file dance that eliminates partial-write corruption.

The write_atomic Temp-Rename-Fsync Pattern

In crates/ai-memory-wiki/src/atomic.rs (lines 13-44), the write_atomic function performs a three-step durability ritual: it writes the full payload to a temporary file, calls sync_data on it, renames the temporary file over the target path, and finally calls sync_all on the parent directory. This tmp + rename + fsync sequence ensures that operating-system or process crashes can never leave a partially written Markdown page on disk. The implementation also tracks the inode of the persisted file so that the file-watcher can safely ignore its own writes and avoid recursive mutation loops.

Rollback Safety with Snapshotting

When a page write must be undone, replace_file_with_rollback_snapshot in crates/ai-memory-wiki/src/wiki.rs (lines 63-71) first captures a snapshot of any existing file before delegating to write_atomic. If the subsequent SQLite upsert fails, rollback_installed_files restores the previous contents or removes the newly created file entirely. This guarantees that the filesystem and the database remain consistent even when the store layer returns an error.

Git-Based Versioning in ai-memory-wiki

Every successful disk write is backed by a Git repository that the system manages automatically.

Lazy Repository Initialization via GitAdapter

The Wiki struct embeds a GitAdapter defined in crates/ai-memory-wiki/src/git.rs. During Wiki::new, the adapter lazily runs git init inside the wiki/ directory only if it is not already a repository. This self-healing bootstrap means deployments do not require manual repository setup.

Staging and Commit Workflow

After a page is written to disk and upserted into SQLite, callers can invoke Wiki::commit_all (or rely on auto-commit hooks) to create a permanent checkpoint. The adapter stages all files with an effective git add *, then generates a commit attributed to the fixed author ai-memory <ai-memory@local>. The GitAdapter also exposes helper methods to count commits, fetch recent checkpoints, and reconstruct a file as it existed at a specific revision. Source lines 43-84 in git.rs contain the full implementation of these operations.

Synchronized Page Write Pipeline

Page modifications follow a strict pipeline that coordinates locking, persistence, embedding, and optional webhooks.

Mutation Locking and Concurrency Control

write_page in crates/ai-memory-wiki/src/wiki.rs (lines 18-30) acquires a read lock on mutation_lock while it writes the file and upserts the store row. Because a concurrent project move takes an exclusive lock, the design prevents interleaving writes that could leave stale files behind after a directory relocation. After persistence completes, an optional embedder runs synchronously, and any webhook chain is dispatched asynchronously without blocking the critical section.

The Seven-Step Write Workflow

A single call to write_page executes the following ordered steps:

1. Sanitisation – self.sanitizer.scrub cleans the body and front-matter.
2. Admission webhook – An optional chain may mutate the markdown before it is emitted.
3. Atomic disk write – replace_file_with_rollback_snapshot calls atomic::write_atomic to durably persist the file.
4. Store upsert – WriterHandle::upsert_page writes the page record to SQLite.
5. Rollback on error – If the store fails, rollback_installed_files restores the snapshot.
6. Embedding – If an embedder is configured, the document is embedded synchronously.
7. Git commit – The caller (or an auto-commit hook) invokes Wiki::commit_all, which delegates to GitAdapter::commit_all.

Code Examples

Writing a Page Atomically

use ai_memory_wiki::Wiki;
use ai_memory_wiki::WritePageRequest;

// Assume `wiki` is a `Wiki` instance and `writer` is a `WriterHandle`.
let req = WritePageRequest {
    workspace_id,
    project_id,
    path: PagePath::new("notes/todo.md")?,
    frontmatter: serde_json::json!({}),
    body: String::from("Buy milk"),
    tier: Tier::User,
    pinned: false,
    title: None,
    admission_ctx: None,
    author_id: None,
    actor: ActorContext::default(),
};

let page_id = wiki.write_page(req).await?;

Committing All Pending Changes

// After one or more `write_page` calls:
if let Some(oid) = wiki.commit_all("auto-commit after session")? {
    println!("Created commit {}", oid);
}

Internal Rollback on Store Failure

// Inside `write_page` the rollback happens automatically.
// The core logic is roughly:
let installed = replace_file_with_rollback_snapshot(&abs, emitted.as_bytes())?;
if let Err(e) = writer.upsert_page(new_page).await {
    rollback_installed_files(&[installed])?;
    return Err(e.into());
}

Summary

• Atomic writes are enforced by write_atomic in atomic.rs, which uses tmp + rename + fsync to prevent partial files.
• Rollback safety is provided by replace_file_with_rollback_snapshot and rollback_installed_files in wiki.rs, keeping the filesystem and SQLite consistent.
• Git versioning is managed by the embedded GitAdapter in git.rs, which lazily initializes the repository and commits all staged changes under a fixed author.
• Concurrency control uses a mutation_lock read/write pattern so that project moves cannot interleave with page writes.
• The full pipeline runs from sanitisation through admission webhooks, atomic persistence, store upsert, optional embedding, and finally Git commit.

Frequently Asked Questions

How does ai-memory prevent corrupted files during a crash?

The wiki uses write_atomic in crates/ai-memory-wiki/src/atomic.rs to write each page to a temporary file, fsync it, rename it over the target path, and fsync the parent directory. Because filesystem renames are atomic on POSIX systems, a crash at any point leaves either the old file or the new file intact, never a partially written one.

What happens if the SQLite upsert fails after the file is written?

replace_file_with_rollback_snapshot in crates/ai-memory-wiki/src/wiki.rs snapshots the existing file before overwriting it. If WriterHandle::upsert_page returns an error, rollback_installed_files restores the snapshot or deletes the new file so the disk state matches the database state.

Does the Git repository need to be created manually before running the wiki?

No. The GitAdapter inside Wiki::new lazily initializes the repository in the wiki/ directory by running git init only if one does not already exist. This is implemented in crates/ai-memory-wiki/src/git.rs.

Can multiple page writes and a project move happen at the same time?

No. write_page acquires a read lock on mutation_lock while a project move acquires an exclusive lock. This prevents concurrent writes from leaving stale files behind during directory moves, as detailed in crates/ai-memory-wiki/src/wiki.rs lines 18-30.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}