# ai-memory Data Directory Structure: A Complete Guide to `<data_dir>` Layout and Organization

> Understand the ai-memory data directory structure. Explore the six-folder layout including wiki, db, raw, logs, models, and client projects for efficient organization and data management.

- Repository: [Fabio Akita/ai-memory](https://github.com/akitaonrails/ai-memory)
- Tags: how-to-guide
- Published: 2026-09-01

---

**The ai-memory data directory uses a six-folder layout with `wiki/` as the single source of truth, `db/memory.sqlite` as the searchable index, plus `raw/`, `logs/`, `models/`, and [`client-projects.json`](https://github.com/akitaonrails/ai-memory/blob/main/client-projects.json) for work-streams, logging, future models, and client metadata.**

The `<data_dir>` directory in the [akitaonrails/ai-memory](https://github.com/akitaonrails/ai-memory) repository serves as the central on-disk workspace for all persistent state. Understanding this structure is essential for configuring deployments, debugging synchronization issues, or extending the system with custom tooling. This guide breaks down each component based on the actual source code implementation.

## Top-Level Directory Structure

The ai-memory data directory contains six top-level subdirectories and several configuration files. All paths are resolved relative to a root `data_dir` passed to core components at initialization.

| Sub-path | Purpose | Source Implementation |
|----------|---------|----------------------|
| **`wiki/`** | Markdown source of truth; every page lives here and is version-controlled by git2 | [`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs) |
| **`db/memory.sqlite`** | SQLite WAL database holding searchable index, page versions, sessions, handoffs, and embeddings | [`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs) |
| **`raw/`** | Immutable, sanitized JSONL segments for managed work-streams | [`crates/ai-memory-hooks/src/workstream.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/workstream.rs) |
| **`logs/`** | Daily-rolling tracing logs from server and CLI | [`crates/ai-memory-mcp/src/admin.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/admin.rs) |
| **`models/`** | Reserved folder for bundled embedding models (future local-model support) | [`docs/design-decisions.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/design-decisions.md) |
| **[`client-projects.json`](https://github.com/akitaonrails/ai-memory/blob/main/client-projects.json)** | Private JSON mapping of client-local checkout links | [`crates/ai-memory-mcp/src/admin.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/admin.rs) |

### Configuration Files in `<data_dir>`

Beyond the folder structure, these auxiliary files live at the root:

- **[`config.toml`](https://github.com/akitaonrails/ai-memory/blob/main/config.toml)** — Global configuration overridable by environment variables
- **[`auth.json`](https://github.com/akitaonrails/ai-memory/blob/main/auth.json)** — Stored provider refresh tokens for LLM authentication
- **`auth-token` / `auth-header`** — Files used by the native hook command for authentication

## The `wiki/` Folder: Single Source of Truth

The `wiki/` directory is the **only** mutable source of truth in the entire ai-memory data directory structure. All other components derive their state from here.

According to the architecture, the SQLite database and raw work-stream files must stay synchronized with `wiki/` contents—the system treats them as disposable rebuildable artifacts, not primary data.

```rust
use ai_memory_wiki::Wiki;
use ai_memory_store::WriterHandle;
use std::path::Path;

let data_dir = Path::new("/my/custom/data_dir");
let writer = WriterHandle::new()?;               // single-writer SQLite actor
let wiki = Wiki::new(data_dir, writer)?;         // roots at <data_dir>/wiki

```

The `Wiki::new()` constructor in [`crates/ai-memory-wiki/src/wiki.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-wiki/src/wiki.rs) handles atomic markdown writes and ensures git2-backed version control for every page operation.

## The `db/memory.sqlite` Database: Derived Search Index

The SQLite database at `db/memory.sqlite` uses WAL mode for concurrent readers and a single writer. It contains:

- Full-text search index
- Page version history
- Session and handoff records
- Vector embeddings

```rust
use ai_memory_store::Store;
use std::path::Path;

let data_dir = Path::new("/my/custom/data_dir");
let store = Store::open(data_dir)?;               // opens <data_dir>/db/memory.sqlite

```

The `Store::open()` function in [`crates/ai-memory-store/src/lib.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-store/src/lib.rs) enforces a single-writer actor model. Multiple processes cannot write simultaneously—this prevents corruption and guarantees consistency with the `wiki/` folder.

## The `raw/` Directory: Immutable Work-Stream Segments

The `raw/` folder stores **immutable, sanitized JSONL segments** for managed work-streams. These files serve two purposes:

1. **Replayable observations** — reconstruct any prior system state
2. **Legacy fallback searches** — operate when the SQLite index is rebuilding

```rust
// Writing work-stream segments happens in ai-memory-hooks
// See crates/ai-memory-hooks/src/workstream.rs for implementation

```

The workstream implementation in [`crates/ai-memory-hooks/src/workstream.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-hooks/src/workstream.rs) appends to these files atomically—once written, segments are never modified or deleted, only rotated.

## The `logs/` Directory: Operational Visibility

Daily-rolling tracing logs from both server and CLI components:

```rust
use tracing::info;
use std::env;

let log_dir = std::env::var("AI_MEMORY_DATA_DIR")
    .unwrap_or_else(|_| "./data".into());
info!(target: "ai_memory", "server started – logs in {}/logs", log_dir);

```

Log path handling is centralized in [`crates/ai-memory-mcp/src/admin.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/admin.rs) alongside other administrative operations.

## The `models/` Folder: Future Local Embeddings

Currently reserved for bundled embedding models. The [`docs/design-decisions.md`](https://github.com/akitaonrails/ai-memory/blob/main/docs/design-decisions.md) document outlines plans for local model support, eliminating external LLM dependencies for certain operations.

## The [`client-projects.json`](https://github.com/akitaonrails/ai-memory/blob/main/client-projects.json) File: Client Metadata

This private JSON file maintains checkout links between the central ai-memory instance and client-local working directories. It is **not part of the public API** and should not be manually edited.

```rust
use std::fs;
use std::path::Path;

let data_dir = Path::new("/my/custom/data_dir");
let json_path = data_dir.join("client-projects.json");
let contents = fs::read_to_string(json_path)?;
println!("Client projects: {}", contents);

```

Managed by [`crates/ai-memory-mcp/src/admin.rs`](https://github.com/akitaonrails/ai-memory/blob/main/crates/ai-memory-mcp/src/admin.rs) alongside authentication and configuration files.

## Architecture Invariants and Design Principles

The ai-memory data directory structure enforces these core invariants:

- **Single source of truth** — Only `wiki/` holds authoritative data; all other stores are derived
- **Immutable work-streams** — `raw/` segments are append-only, enabling perfect reproducibility
- **Single-writer database** — SQLite operations are serialized through the `WriterHandle` actor
- **Git-native versioning** — Markdown files in `wiki/` use libgit2 for automatic version tracking

These constraints enable reliable backups (just `wiki/` and [`config.toml`](https://github.com/akitaonrails/ai-memory/blob/main/config.toml)), clean disaster recovery, and deterministic testing.

## Customizing the ai-memory Data Directory Location

All components accept a `data_dir: PathBuf` parameter. The system respects the `AI_MEMORY_DATA_DIR` environment variable as a default fallback when no explicit path is provided.

| Component | Initialization Function | Data Directory Usage |
|-----------|------------------------|----------------------|
| Wiki | `Wiki::new(data_dir, writer)` | Creates `<data_dir>/wiki` |
| Store | `Store::open(data_dir)` | Opens `<data_dir>/db/memory.sqlite` |
| MCP Admin | Internal path resolution | Manages [`config.toml`](https://github.com/akitaonrails/ai-memory/blob/main/config.toml), [`auth.json`](https://github.com/akitaonrails/ai-memory/blob/main/auth.json), [`client-projects.json`](https://github.com/akitaonrails/ai-memory/blob/main/client-projects.json) |

## Summary

- The ai-memory `<data_dir>` contains **six top-level elements**: `wiki/`, `db/memory.sqlite`, `raw/`, `logs/`, `models/`, and [`client-projects.json`](https://github.com/akitaonrails/ai-memory/blob/main/client-projects.json)
- **`wiki/` is the sole source of truth** — all other data can be reconstructed from markdown files
- **`db/memory.sqlite`** provides fast search via single-writer SQLite with WAL mode
- **`raw/` stores immutable JSONL work-stream segments** for replay and fallback
- **Configuration and auth files** ([`config.toml`](https://github.com/akitaonrails/ai-memory/blob/main/config.toml), [`auth.json`](https://github.com/akitaonrails/ai-memory/blob/main/auth.json), `auth-token`, `auth-header`) live at the root
- All paths resolve through a `data_dir: PathBuf` passed to `Wiki::new()`, `Store::open()`, and administrative functions

## Frequently Asked Questions

### Where is the actual content stored in ai-memory?

All user content lives in the **`wiki/` folder** as markdown files. The SQLite database at `db/memory.sqlite` only holds derived indexes and embeddings for fast searching. If you back up only one folder, make it `wiki/`.

### Can I move the ai-memory data directory to a different location?

Yes. Pass any `PathBuf` to `Wiki::new()` or `Store::open()`, or set the `AI_MEMORY_DATA_DIR` environment variable. All subdirectories are created relative to your specified path. The `models/` folder and configuration files will initialize automatically on first run.

### What happens if `db/memory.sqlite` becomes corrupted?

Delete it and restart ai-memory. The system will rebuild the entire database from the `wiki/` markdown source. The `raw/` work-stream segments are similarly rebuildable. This design makes corruption recovery straightforward—your actual data in `wiki/` remains safe.

### Is [`client-projects.json`](https://github.com/akitaonrails/ai-memory/blob/main/client-projects.json) required for operation?

No. This file is optional and used only when connecting client-local checkouts to a central ai-memory instance. If absent, the system operates in standalone mode. The file is managed automatically by the MCP admin component and should not be manually edited.