# How ai-memory Handles Atomic Markdown Writes: The tmp-rename-fsync Pattern Explained

> Discover how ai-memory ensures crash-safe Markdown writes with the tmp-rename-fsync pattern and automatic rollback for database failures. Learn about this robust atomic write protocol.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: deep-dive
- Published: 2026-09-05

---

**ai-memory guarantees crash-safe Markdown file updates using a tmp-rename-fsync atomic write protocol with automatic rollback on database failures.**

The `ai-memory` wiki engine stores every page as a Markdown file inside the `<data_dir>/wiki/` directory tree. When you create or update a page, the system must ensure that **no partially written or torn file ever appears on disk**—even if the process crashes, the database transaction fails, or antivirus software briefly locks files on Windows. This article breaks down the exact mechanism implemented in the akitaonrails/ai-memory source code.

## The Entry Point: Wiki::write_page

All page writes flow through **`Wiki::write_page`** at [[`wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/wiki.rs) lines 1803-1805](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs#L1803-L1805). This high-level method orchestrates sanitization, admission webhooks, atomic file replacement, database upserts, and optional embedding computation.

Here's the complete pipeline:

| Step | Operation |
|------|-----------|
| **Scrub & admission** | Body sanitization, optional webhook mutation, front-matter canonicalization |
| **Emit Markdown** | In-memory `Markdown` struct serialized to UTF-8 |
| **Snapshot & atomic write** | Existing file snapshot saved, new content written atomically |
| **Store upsert** | Page metadata upserted into SQLite; on failure, filesystem rolled back |
| **Embedding** | Vector embedding computed and stored (post-commit) |
| **Post-write webhooks** | Non-blocking hooks dispatched after durability confirmed |

## Core Primitive: atomic::write_atomic

The heart of the system lives in **`atomic::write_atomic`** at [[`atomic.rs`](https://github.com/akitaonrails/ai-memory/blob/main/atomic.rs) lines 1-20](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/atomic.rs#L1-L20). This function implements the classic **tmp-rename-fsync** pattern for atomic file replacement:

```rust
// Example: atomically replace a page (used internally by write_page)
use ai_memory_wiki::atomic::write_atomic;
use std::path::Path;

let path = Path::new("/data_dir/wiki/001/002/notes.md");
let content = b"# Updated title\n\nNew body";

write_atomic(path, content)?;   // guarantees atomicity & fsync

```

The six-step protocol ensures durability:

1. **Create temporary file** in the target directory with prefix `.ai-memory-tmp.`
2. **Write and sync data** to the temporary file via `tmp.as_file().sync_data()?`
3. **Atomic rename** using `tempfile::NamedTempFile::persist` (with Windows retry logic)
4. **Sync target file** after rename via `persisted.sync_data()?`
5. **Best-effort parent directory fsync** via `dir.sync_all()?` to persist directory entry
6. **Return inode/NTFS file index** for the file watcher's self-ignore mechanism

### Windows-Specific Retry Handling

On Windows, transient sharing violations from antivirus or search indexers can block the rename. The system uses **`persist_with_retry`** at [[`atomic.rs`](https://github.com/akitaonrails/ai-memory/blob/main/atomic.rs) lines 20-40](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/atomic.rs#L20-L40), which retries the rename with 10ms delays before giving up.

## Rollback Safety: Database-Filesystem Consistency

Atomic writes alone don't solve the **two-phase commit problem**: what if the file rename succeeds but the SQLite upsert fails? The `ai-memory` system solves this with **snapshot-based rollback**.

### replace_file_with_rollback_snapshot

This helper at [[`wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/wiki.rs) lines 2303-2310](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs#L2303-L2310) captures the previous file state before overwriting:

```rust
// Example: manually rolling back a failed write (illustrates the helper)
use ai_memory_wiki::wiki::{replace_file_with_rollback_snapshot, rollback_or_inconsistent};

let installed = replace_file_with_rollback_snapshot(&path, bytes)?;
if let Err(db_err) = writer.upsert_page(...).await {
    // Restore the previous version so DB and FS stay consistent
    rollback_or_inconsistent(&[installed], &db_err)?;
}

```

The function returns an **`InstalledFile`** struct recording:
- The old file bytes (or `None` if file didn't exist)
- The target path

### rollback_installed_files

On store-write failure, **`rollback_installed_files`** at [[`wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/wiki.rs) lines 2312-2325](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs#L2312-L2325) executes:

- **For existing files**: Write back the saved snapshot bytes
- **For new files**: Remove the newly-created file

This ensures the on-disk state always matches the database state, even after crashes.

## Complete write_page Usage

Here's how client code triggers the full atomic workflow:

```rust
// Example: how write_page performs a safe replace with rollback
use ai_memory_wiki::wiki::Wiki;
use ai_memory_core::WritePageRequest;

let req = WritePageRequest {
    workspace_id,
    project_id,
    path: PagePath::new("notes/todo.md")?,
    frontmatter: serde_json::json!({ "tags": ["todo"] }),
    body: "Fix the atomic write implementation".into(),
    ..Default::default()
};

let page_id = wiki.write_page(req).await?;   // atomic write + DB upsert

```

## File Watcher Integration: Self-Ignoring Writes

The atomic write system returns the **inode (or NTFS file index)** of the new file at [[`atomic.rs`](https://github.com/akitaonrails/ai-memory/blob/main/atomic.rs) lines 95-140](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/atomic.rs#L95-L140). The file watcher uses this to distinguish its own writes from external modifications, preventing infinite update loops.

## Key Source Files

| File | Purpose |
|------|---------|
| [[`crates/ai-memory-wiki/src/atomic.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/atomic.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/atomic.rs) | `write_atomic`, `persist_with_retry`, inode extraction—low-level atomic primitives |
| [[`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs) | `write_page`, `replace_file_with_rollback_snapshot`, rollback orchestration |
| [[`crates/ai-memory-wiki/src/markdown.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/markdown.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/markdown.rs) | Markdown serialization used before atomic write |
| [[`crates/ai-memory-wiki/src/admission.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/admission.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/admission.rs) | Pre-write webhook chain for mutation/rejection |
| [[`crates/ai-memory-wiki/tests/suite/mod.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/tests/suite/mod.rs)](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/tests/suite/mod.rs) | Test coverage for atomic writes, rollback, Windows retry logic |

## Summary

- **Atomic markdown writes** in `ai-memory` use the proven tmp-rename-fsync pattern with six durability steps
- **Windows retry logic** handles transient antivirus locks via `persist_with_retry`
- **Snapshot rollback** ensures database-filesystem consistency when SQLite upserts fail
- **Inode tracking** lets the file watcher ignore its own atomic writes
- **Admission webhooks** run before the atomic commit, allowing safe content mutation

## Frequently Asked Questions

### What happens if the process crashes during an atomic write?

If the crash occurs **before** the rename, the temporary file is abandoned and the original file remains intact. If the crash occurs **after** the rename but **before** the database commit, the next operation will detect the inconsistency and trigger `rollback_or_inconsistent` to restore the previous file state.

### Does ai-memory fsync on every write, and does this hurt performance?

Yes, `ai-memory` calls `sync_data()` on both the temporary file and the persisted file, plus `sync_all()` on the parent directory. These are blocking operations that ensure crash safety. The design prioritizes durability over raw throughput—appropriate for a knowledge base where data integrity matters more than log-style append performance.

### Why does the system need to track inodes after atomic writes?

The file watcher monitors the `<data_dir>/wiki/` tree for external changes. Without inode tracking, the watcher would detect its own atomic renames as external modifications and potentially re-process or re-index the file. By recording the inode of files it writes, the watcher can filter self-generated events.

### Can the atomic write fail on Windows even with retry logic?

In extreme cases—antivirus software with aggressive file locking, or very slow I/O—the 10ms × N retry loop in `persist_with_retry` may exhaust its attempts. The operation returns an error, which bubbles up to `write_page` and triggers the standard rollback path. No partial file state is ever committed.