How Tolaria Handles Complex Codebases: A Three-Layer Architecture Deep Dive

Tolaria handles complex codebases by treating the filesystem as the single source of truth, implementing a three-layer caching strategy that synchronizes JSON disk cache, React state, and Git-based incremental updates to maintain performance with 10,000+ files.

Tolaria (refactoringhq/tolaria) is designed for developers managing extensive markdown vaults where traditional note-taking applications struggle with sync issues and performance degradation. The application never stores duplicate state that could diverge from disk, instead relying on a strict ownership model between the filesystem, a JSON cache, and the UI. This architecture ensures that even massive vaults remain responsive, consistent, and instantly searchable.

The Three-Layer Architecture

Tolaria's approach to handling complex codebases rests on three distinct representations that maintain strict invariants to prevent data drift.

The filesystem remains the authoritative source, containing the actual .md files on disk. Tauri Rust commands such as save_note_content and update_frontmatter own all writes to this layer.

The cache layer stores a JSON index at ~/.laputa/cache/<hash>.json, managed by scan_vault_cached() in src-tauri/src/vault/cache.rs. This cache can be rebuilt completely from the filesystem at any time, ensuring it remains disposable rather than authoritative.

The React state layer holds an in-memory VaultEntry[] array consumed by the UI. The useVaultLoader and useEntryActions hooks populate this state from the cache on initial load, while external changes trigger reloads that bypass stale cache entries.

Layer	Owner	Writes to	Reads from
Filesystem	Tauri Rust commands (save_note_content, update_frontmatter)	Disk	—
Cache	scan_vault_cached() (src-tauri/src/vault/cache.rs)	~/.laputa/cache/	Filesystem + git diff
React state	useVaultLoader / useEntryActions	In-memory entries	Cache (on load), filesystem (on reload)

Incremental Vault Scanning and Git-Based Caching

When opening a vault, Tolaria's scan_vault_cached() function in src-tauri/src/vault/cache.rs evaluates three distinct strategies to minimize startup time.

A full rescan walks the entire directory tree using walkdir logic defined in src-tauri/src/vault/scan.rs, occurring only when the cache is missing or corrupt. A cache hit reuses the existing JSON index when the cached Git HEAD commit matches the current repository state. An incremental update runs git diff between the cached commit and current HEAD, re-parsing only changed files rather than the entire vault.

This Git-aware strategy means that opening a 10,000-file vault after minor edits requires parsing only a handful of changed files rather than the entire corpus.

Real-Time External Change Detection

Complex development workflows require handling external edits—when users modify files in VS Code, Vim, or other editors while Tolaria remains open.

The native file-watcher implemented in src-tauri/src/vault_watcher.rs monitors vault directories for changes, batches rapid-fire events, and filters noise from .git/ directories and temporary files. When external changes are detected, the watcher triggers the reload_vault Tauri command, which deletes the existing cache and performs a fresh scan. This guarantees the UI always reflects the latest on-disk state without requiring manual refresh or application restart.

Progressive UI Loading for Large Vaults

Tolaria employs progressive loading to ensure complex codebases feel immediately usable regardless of vault size.

The useVaultLoader hook defined in src/lib/hooks/useVaultLoader.tsx exposes an isLoading state that remains true while the vault index builds. However, the application shell—including the sidebar and folder tree—renders immediately, allowing users to navigate directory structures before the full note list is available. This architecture prevents UI blocking during the initial scan of massive vaults.

// src/lib/hooks/useVaultLoader.tsx
import { useEffect, useState } from 'react';
import { invoke } from '@tauri-apps/api/tauri';
import { VaultEntry } from '@/types';

export function useVaultLoader(vaultPath: string) {
  const [entries, setEntries] = useState<VaultEntry[]>([]);
  const [isLoading, setLoading] = useState(true);

  // Initial load – may use the cache if valid
  useEffect(() => {
    async function load() {
      setLoading(true);
      const result = await invoke<VaultEntry[]>('load_vault_entries', {
        path: vaultPath,
      });
      setEntries(result);
      setLoading(false);
    }
    load();
  }, [vaultPath]);

  // Manual reload triggered by external watcher or UI button
  const reload = async () => {
    setLoading(true);
    await invoke('reload_vault', { path: vaultPath });
    const fresh = await invoke<VaultEntry[]>('load_vault_entries', {
      path: vaultPath,
    });
    setEntries(fresh);
    setLoading(false);
  };

  return { entries, isLoading, reload };
}

Handling Large Individual Notes

Beyond vault scale, Tolaria optimizes for individual file complexity. When opening a note, the application checks the cache for matching modifiedAt and fileSize metadata. If these values align with the current filesystem state, the cached text content is reused immediately.

If the metadata differs, validate_note_content reads the file fresh, discards the stale cache entry, and loads the updated content. The editor then parses markdown into BlockNote blocks on a background thread, ensuring the UI thread never blocks on heavy parsing operations.

Code Examples

The following examples demonstrate the core APIs that enable Tolaria to handle complex codebases efficiently.

The Rust implementation in src-tauri/src/vault/cache.rs shows the decision logic for cache strategies:

// src-tauri/src/vault/cache.rs (simplified)
pub async fn scan_vault_cached(vault_path: PathBuf) -> Result<Vec<VaultEntry>> {
    // 1️⃣ Check if a cache file exists
    if let Some(cache) = load_cache(&vault_path).await? {
        // 2️⃣ Verify that the cached Git HEAD matches the current HEAD
        if cache.head == current_git_head(&vault_path).await? {
            // 3️⃣ Perform an incremental diff‑only update
            let changed = git_changed_files(&vault_path, &cache.head).await?;
            return update_cache_with_changes(cache, changed).await;
        }
    }
    // Fallback: full rescan
    full_rescan(vault_path).await
}

For command-line operations or debugging, you can force a full rescan using the Tauri CLI:

# Command‑line usage – force a full rescan (e.g. after a massive rename)

pnpm tauri invoke reload_vault --path /path/to/my/vault

Summary

• Filesystem as source of truth: Tolaria never stores authoritative state outside the actual .md files, preventing data divergence.
• Three-layer synchronization: The architecture separates concerns between disk files, JSON cache (~/.laputa/cache/), and React state, with clear ownership rules for each layer.
• Git-aware incremental updates: scan_vault_cached() uses Git history to perform differential updates, making large vaults load in seconds rather than minutes.
• Native file watching: src-tauri/src/vault_watcher.rs detects external changes and triggers reload_vault to maintain UI consistency.
• Progressive loading: The UI renders immediately while indexing continues in the background, with useVaultLoader managing the loading state transparently.

Frequently Asked Questions

How does Tolaria ensure data consistency across its three layers?

Tolaria enforces strict ownership rules where the filesystem remains the only persistent authority. The JSON cache in ~/.laputa/cache/<hash>.json is treated as ephemeral and can be regenerated at any time. React state only reads from the cache during initial load or after explicit reload commands. When external changes are detected via the file watcher, the reload_vault command invalidates the cache and rebuilds it from disk, ensuring all layers converge on the filesystem state.

What triggers a full rescan versus an incremental update?

The scan_vault_cached() function in src-tauri/src/vault/cache.rs checks for three conditions. A full rescan occurs when no cache exists or when the cache file is corrupt. An incremental update runs when the cache exists and the Git HEAD commit differs from the cached commit, allowing Tolaria to parse only files changed between those commits via git diff. A cache hit occurs when the Git commits match, loading the existing JSON index immediately without filesystem traversal.

How does Tolaria handle external edits from other editors?

External changes are captured by the native Rust file watcher in src-tauri/src/vault_watcher.rs, which monitors the vault directory and batches filesystem events. The watcher filters out .git/ noise and temporary files, then triggers the reload_vault Tauri command. This command deletes the stale cache and performs a fresh scan, ensuring the React state updates to reflect the current disk contents regardless of which application modified the files.

Can Tolaria handle vaults with more than 10,000 markdown files?

Yes. Tolaria is specifically architected for complex codebases and large vaults through progressive loading and incremental updates. The useVaultLoader hook allows the UI to render immediately while indexing continues in the background. Git-based differential scanning means only modified files require re-parsing, keeping startup times consistent regardless of total vault size. The cache location (~/.laputa/cache/) ensures that repeated openings of the same vault are nearly instantaneous.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}