# Understanding the File-System Architecture of LLM Wiki: A Deep Dive into the Knowledge Base Structure

> Explore the LLM Wiki file-system architecture. Discover its three-layer data persistence model (Raw Wiki Schema) and monorepo structure, separating React frontend from Rust backend.

- Repository: [nash_su/llm_wiki](https://github.com/nashsu/llm_wiki)
- Tags: deep-dive
- Published: 2026-09-12

---

**The file-system architecture of LLM Wiki implements a three-layer data persistence model (Raw → Wiki → Schema) within each project directory, supported by a monorepo codebase that separates the React TypeScript frontend from the Rust Tauri backend.**

LLM Wiki, an open-source knowledge management system from the `nashsu/llm_wiki` repository, organizes data through a deliberate file-system architecture inspired by Andrej Karpathy’s original LLM Wiki pattern. This design separates immutable source evidence from generated knowledge and structural rules, enabling the application to maintain a queryable graph while preserving raw document integrity.

## Project-Level Knowledge Base Structure

When a user creates a new LLM Wiki project, the application generates a folder containing a strict hierarchical layout that separates raw inputs from processed outputs.

### The Three-Layer Data Model

The architecture centers on three distinct directories that reflect a transformation pipeline:

- **`raw/sources/`** – Stores immutable source documents including PDFs, DOCX files, Markdown, images, and web clips. The application watches this folder for external changes via file system monitors. Implementation logic resides in [`src/lib/url-source-import.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/url-source-import.ts) and [`src/lib/wiki-page-resolver.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-page-resolver.ts).

- **`wiki/`** – Contains generated Markdown files written by the LLM. Each file includes YAML front-matter recording its type, title, and originating sources via the `sources: []` array. The resolution logic is handled in [`src/lib/wiki-page-resolver.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-page-resolver.ts) and [`src/lib/wiki-filename.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-filename.ts).

- **[`schema.md`](https://github.com/nashsu/llm_wiki/blob/main/schema.md)** – An optional configuration file defining structural rules, page-type definitions, and routing logic for the knowledge base. The parser is implemented in [`src/lib/wiki-schema.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-schema.ts).

### Configuration and Navigation Files

Beyond the core data layers, each project root contains specific metadata files:

- **[`index.md`](https://github.com/nashsu/llm_wiki/blob/main/index.md)** – Serves as the entry point and navigation anchor for the LLM during query operations. The ingest pipeline creates and updates this file automatically.

- **[`log.md`](https://github.com/nashsu/llm_wiki/blob/main/log.md)** – Maintains a chronological record of operations including ingest, delete, and lint actions. This enables "time-travel" reasoning where the LLM can parse historical state changes.

- **[`purpose.md`](https://github.com/nashsu/llm_wiki/blob/main/purpose.md)** – An optional human-written statement defining project goals and research scope. The ingest pipeline reads this via `loadProjectStore` on every operation to align LLM behavior with user intent.

### Hidden Application State

The **`.llmwiki/`** directory stores persistent application configuration using Tauri’s plugin-store mechanism. Located at [`src/lib/project-store.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/project-store.ts), this layer manages settings, recent projects, and model overrides in a hidden subdirectory to separate application metadata from user content.

## Repository Source Code Layout

The `nashsu/llm_wiki` repository follows a conventional monorepo structure that isolates frontend, backend, and auxiliary services.

### Frontend TypeScript Core

The `src/` directory contains the React-based user interface and core business logic:

- **[`src/App.tsx`](https://github.com/nashsu/llm_wiki/blob/main/src/App.tsx)** – Top-level component responsible for booting the UI and restoring persisted state.
- **[`src/main.tsx`](https://github.com/nashsu/llm_wiki/blob/main/src/main.tsx)** – React entry point that renders the application root.
- **`src/lib/`** – Core TypeScript logic including:
  - [`wiki-graph.ts`](https://github.com/nashsu/llm_wiki/blob/main/wiki-graph.ts) – Builds the knowledge graph, computes relevance scores, and runs Louvain community detection via `detectCommunities`.
  - [`wiki-page-resolver.ts`](https://github.com/nashsu/llm_wiki/blob/main/wiki-page-resolver.ts) – Resolves `[[wikilinks]]` to actual wiki pages and maintains source references.
  - [`wiki-schema.ts`](https://github.com/nashsu/llm_wiki/blob/main/wiki-schema.ts) – Parses and applies routing rules from [`schema.md`](https://github.com/nashsu/llm_wiki/blob/main/schema.md).
  - [`web-search.ts`](https://github.com/nashsu/llm_wiki/blob/main/web-search.ts) – Provides a unified façade for Firecrawl, SearXNG, and other search providers.
  - [`vision-caption.ts`](https://github.com/nashsu/llm_wiki/blob/main/vision-caption.ts) – Generates image captions using vision LLMs for multimodal ingest.
- **`src/stores/`** – Zustand state management including [`wiki-store.ts`](https://github.com/nashsu/llm_wiki/blob/main/wiki-store.ts), [`chat-store.ts`](https://github.com/nashsu/llm_wiki/blob/main/chat-store.ts), and [`update-store.ts`](https://github.com/nashsu/llm_wiki/blob/main/update-store.ts).

### Native Rust Backend

The `src-tauri/` directory houses the Tauri-based native layer:

- **[`src-tauri/src/lib.rs`](https://github.com/nashsu/llm_wiki/blob/main/src-tauri/src/lib.rs)** – Exposes core commands for file I/O and HTTP server functionality.
- **`src-tauri/src/agent/`** – Contains the agent runtime responsible for tool-using chat and file generation.

### Auxiliary Services

Additional directories provide supporting functionality:

- **`mcp-server/`** – A lightweight local MCP (Model Context Protocol) server implemented in TypeScript. Contains [`src/api-client.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/api-client.ts) for search operations and [`src/project-binding.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/project-binding.ts) for project integration.
- **`extension/`** – Chrome extension code for one-click web clipping, including [`background.js`](https://github.com/nashsu/llm_wiki/blob/main/background.js) and [`popup.html`](https://github.com/nashsu/llm_wiki/blob/main/popup.html).
- **`assets/`** – Static illustrations for UI and documentation.

## Runtime Caching Strategy

LLM Wiki optimizes performance through strategic disk caching within the `.llmwiki/` directory:

- **`.llmwiki/cache/`** – Stores incremental SHA-256 hashes for source files. The ingest pipeline compares these hashes to skip unchanged files, significantly reducing redundant processing.

- **`.llmwiki/graph/`** – Persists memoized graph data structures for fast UI rendering. The system respects the `MAX_CACHED_PROJECT_GRAPHS` limit defined in [`src/lib/wiki-graph.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-graph.ts) to prevent unbounded disk usage.

These caches refresh automatically when the ingest queue processes new sources or detects file modifications.

## Interacting with the File System Programmatically

The codebase exposes several utilities for direct file-system manipulation from TypeScript.

### Building the Knowledge Graph

The `buildWikiGraph` function constructs the complete knowledge representation by scanning the `wiki/` directory:

```typescript
import { buildWikiGraph } from "@/lib/wiki-graph";

// projectPath is the absolute path to the current LLM Wiki project
const result = await buildWikiGraph(projectPath);
console.log(result.nodes.length, "nodes", result.edges.length, "edges");

```

As implemented in [`src/lib/wiki-graph.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-graph.ts), this function extracts wikilinks from every Markdown file, calculates relevance scores, and executes community detection algorithms.

### Importing External Sources

The `importUrlAsSource` utility downloads remote content into the immutable raw layer:

```typescript
import { importUrlAsSource } from "@/lib/url-source-import";

await importUrlAsSource({
  project,
  url: "https://arxiv.org/pdf/2302.00001.pdf",
  title: "Attention Is All You Need",
});

```

This writes the downloaded file into `raw/sources/` and registers it with the auto-watcher, as defined in [`src/lib/url-source-import.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/url-source-import.ts).

### Executing Web Searches

The unified search interface supports multiple providers through a single function:

```typescript
import { searchWeb } from "@/lib/web-search";

const results = await searchWeb({
  query: "transformer architecture survey",
  provider: "firecrawl",   // or "searxng", "none"
});
console.log(results.slice(0, 3));

```

The implementation in [`src/lib/web-search.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/web-search.ts) resolves API keys from application state and normalizes results across different search backends.

### Managing Project Persistence

Application metadata access occurs through the project store API:

```typescript
import { getRecentProjects, saveLastProject } from "@/lib/project-store";

const recent = await getRecentProjects();
await saveLastProject({ 
  id: "proj-1", 
  path: "/home/user/my-wiki", 
  name: "My Wiki" 
});

```

[`src/lib/project-store.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/project-store.ts) wraps Tauri’s `plugin-store` to persist JSON state to [`.llmwiki/app-state.json`](https://github.com/nashsu/llm_wiki/blob/main/.llmwiki/app-state.json).

## Summary

- **LLM Wiki** employs a three-layer file-system architecture (Raw → Wiki → Schema) that treats source documents as immutable evidence while allowing LLM-generated content to evolve in the `wiki/` directory.
- The **monorepo structure** cleanly separates the React frontend (`src/`), Rust Tauri backend (`src-tauri/`), MCP server (`mcp-server/`), and browser extension (`extension/`).
- **Runtime caches** in `.llmwiki/cache/` and `.llmwiki/graph/` optimize ingest performance and graph rendering while respecting configurable limits.
- Key implementation files including [`src/lib/wiki-graph.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-graph.ts), [`src/lib/wiki-page-resolver.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-page-resolver.ts), and [`src/lib/project-store.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/project-store.ts) provide deterministic access to the file-system layers.

## Frequently Asked Questions

### What is the purpose of the `raw/sources/` directory in LLM Wiki?

The `raw/sources/` directory stores immutable source documents such as PDFs, images, and web clips. According to the source code in [`src/lib/url-source-import.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/url-source-import.ts), this folder is watched for external changes, and its contents serve as the ground-truth evidence that the LLM references when generating wiki pages.

### How does LLM Wiki handle application state persistence?

Application state persists through Tauri’s plugin-store API, wrapped by [`src/lib/project-store.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/project-store.ts). This stores configuration, recent projects, and model overrides in a hidden `.llmwiki/` directory, specifically within [`.llmwiki/app-state.json`](https://github.com/nashsu/llm_wiki/blob/main/.llmwiki/app-state.json), separate from user-generated knowledge content.

### What role does [`schema.md`](https://github.com/nashsu/llm_wiki/blob/main/schema.md) play in the file-system architecture?

The optional [`schema.md`](https://github.com/nashsu/llm_wiki/blob/main/schema.md) file defines structural rules, page-type definitions, and routing logic for the knowledge base. Parsed by [`src/lib/wiki-schema.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-schema.ts), it enforces consistency across the `wiki/` directory by constraining how the LLM generates and links pages.

### How does the Chrome extension integrate with the desktop application’s file system?

The extension in [`extension/background.js`](https://github.com/nashsu/llm_wiki/blob/main/extension/background.js) captures web pages and forwards them to the desktop app, which then writes the content into `raw/sources/` using the same pipeline as `importUrlAsSource`. This creates a seamless bridge between browser content and the local knowledge base.