# Why Markdown in Git Is the Source of Truth in ai-memory > Discover how Markdown in Git acts as the immutable source of truth for ai-memory, with SQLite serving only as a rebuildable search index. Learn more about this efficient data management. - Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory) - Tags: architecture - Published: 2026-09-11 --- **In ai-memory, plain Markdown files versioned in Git serve as the immutable source of truth, while the SQLite database acts solely as a rebuildable search index derived from those files.** The akitaonrails/ai-memory project implements a knowledge management system where Markdown in Git as the source of truth guarantees data durability and human accessibility. Unlike traditional applications that treat databases as primary storage, ai-memory inverts this model: the filesystem holds the canonical content, and the database merely accelerates queries. This architecture ensures that your knowledge remains portable, human-readable, and safe from corruption. ## The Immutable Wiki Architecture According to [`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md), the system stores all knowledge under `/wiki/` as ordinary Markdown files. These files are versioned by Git, creating an immutable audit trail that survives database corruption or deletion. ### Plain Text Authority The Markdown pages are ordinary text files that can be inspected with standard Unix tools, edited in any text editor like Obsidian, or processed by automated scripts. As noted in [`README.md`](https://github.com/akitaonrails/ai-memory/blob/main/README.md), "Your memory is plain markdown. The source of truth is a git-backed wiki of ordinary `.md` files." This design eliminates vendor lock-in and ensures the content remains accessible regardless of the ai-memory application's state. ### Versioned History Every modification creates a Git commit, enabling rollback to previous states and complete auditability. The files can be grepped, diffed, and merged using standard Git workflows, treating knowledge with the same rigor as source code. ## SQLite as the Derived Index While Markdown files hold the truth, [`docs/ARCHITECTURE.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/ARCHITECTURE.md) explains that "SQLite is the derived index for search, sessions, observations, handoffs, audit, embeddings, and the optional managed-workstream continuity ledger." This index is purely functional—it can be destroyed and regenerated without data loss. The implementation in [`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs) handles the writer thread that consumes page changes and populates the searchable index. ### Automatic Index Rebuilding Whenever a page is written or updated, the server’s writer thread records the change in SQLite, populating FTS5 tables, entity relationships, and vector embeddings. However, these structures are ephemeral; the `ai-memory reindex` command can rebuild the entire database from the Markdown source, ensuring the system recovers instantly from corruption. ## Writing to the Source of Truth via the Rust API Applications interact with the canonical store through the `Wiki` struct defined in [`crates/ai-memory-wiki/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/lib.rs). The `write_page` method appends content directly to the filesystem, triggering asynchronous index updates while guaranteeing the Markdown file is written first. ```rust use ai_memory_wiki::Wiki; use ai_memory_core::ids::PagePath; let wiki = Wiki::new(data_dir.clone())?; let path = PagePath::from_str("sessions/2026-09-11T12:00:00Z.md")?; let content = r#" ## Session Summary - Prompt: Explain why markdown is source of truth - Result: See docs/ARCHITECTURE.md "#; wiki.write_page(&path, content)?; ``` This code path ensures the Markdown file exists on disk before any database entries are created, maintaining the invariant that the filesystem is primary. ## Operational Resilience and Recovery The separation of concerns between content and index provides exceptional durability. If the SQLite file becomes corrupted, operators simply regenerate it from the immutable Markdown source. ### CLI Reindexing Workflow To rebuild the derived index from the Markdown source of truth: ```bash # Stop the server to prevent write conflicts ai-memory stop # Regenerate SQLite from the wiki files ai-memory reindex --data-dir ~/.local/share/ai-memory ``` After completion, the system resumes operation with a fresh, consistent database that perfectly reflects the Git-tracked Markdown files. Queries then leverage the rebuilt index: ```bash ai-memory query "source of truth" # Returns ranked results using FTS5 and link-neighbour RRF ``` ## Summary - **Markdown files are canonical**: Stored under `/wiki/`, these Git-tracked files contain the authoritative content. - **SQLite is disposable**: The database acts as a derived index for fast search and entity tracking, rebuildable via `ai-memory reindex`. - **Human-first design**: Plain text storage enables editing with standard tools, version control, and portable backups. - **Write ordering guarantees**: The Rust API in [`crates/ai-memory-wiki/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/lib.rs) ensures filesystem writes complete before index updates. ## Frequently Asked Questions ### What happens if the SQLite database becomes corrupted? If the SQLite file is damaged, no knowledge is lost. Because Markdown in Git is the source of truth, administrators can stop the server and run `ai-memory reindex --data-dir ` to regenerate the entire database from the immutable Markdown files. The system will repopulate FTS5 indices, embeddings, and entity tables exactly as they were. ### Why not use the database as the primary source of truth? The architects of ai-memory prioritized human readability and long-term data liberation. By keeping the source of truth in plain Markdown files, users avoid vendor lock-in and can access their knowledge using any text editor or standard Git tools. The database remains an optimization layer that improves query performance without risking data portability. ### How do I manually edit memory entries? Since the content resides in standard `.md` files under the wiki directory, you can edit them directly with Vim, Obsidian, or any Markdown editor. After saving changes and committing them to Git, the system will detect the modifications and update the derived SQLite index accordingly, or you can force a reindex to ensure consistency. ### Can I use standard Git workflows to manage my ai-memory data? Yes. The Markdown files are fully compatible with Git operations. You can `git diff` changes, `git log` to review history, branch for experimental knowledge organization, or merge concurrent edits. This treats your accumulated knowledge with the same collaborative rigor as software source code.