# How LLM Wiki Handles Immutable Source Documents in the raw/sources Directory

> Discover how LLM Wiki safeguards immutable source documents in raw/sources. Learn about its three-layer architecture that copies, extracts, and references files without altering originals.

- Repository: [nash_su/llm_wiki](https://github.com/nashsu/llm_wiki)
- Tags: how-to-guide
- Published: 2026-09-12

---

**LLM Wiki enforces strict read-only access to files in `raw/sources` through a three-layer architecture that copies, extracts, and references source materials without ever mutating the originals.**

The `nashsu/llm_wiki` repository implements a robust content pipeline designed to treat files in the `raw/sources` directory as immutable source documents. This architectural pattern ensures that original research papers, PDFs, and data files remain pristine while the system generates editable wiki summaries. Understanding this immutability guarantee is essential for maintaining data integrity when building LLM-powered knowledge bases.

## The Three-Layer Immutability Architecture

According to the source code in `nashsu/llm_wiki`, immutability is enforced through three complementary layers that operate during ingestion, processing, and resolution.

### Layer 1: Source Watch Configuration

The first layer of protection resides in [`src/lib/source-watch-config.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/source-watch-config.ts), where the `isPathAllowedBySourceWatch` function filters incoming files. This utility exclusively validates read eligibility by checking file extensions and path patterns while explicitly excluding binaries, temporary files, and hidden items. Crucially, the configuration never returns a write-permission flag, establishing a read-only contract at the entry point.

### Layer 2: Ingestion and Lifecycle

Once admitted, files enter the lifecycle management system in [`src/lib/source-lifecycle.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/source-lifecycle.ts). Functions such as `importSources`, `preprocessFile`, and `copyFile` operate exclusively on a destination copy inside `raw/sources`. The pipeline extracts text content and generates a separate markdown summary under `wiki/sources`, ensuring the original file timestamp and bytes remain unaltered after the initial copy operation.

### Layer 3: Reference Resolution

The final safeguard occurs during page rendering in [`src/lib/wiki-page-resolver.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-page-resolver.ts). When the `resolveSourceName` function encounters a front-matter entry like `sources: [paper.pdf]`, it searches `wiki/sources` first for summary pages, then falls back to `raw/sources` for the original asset. The returned path is treated as read-only; any write operations triggered by the LLM target the generated wiki page, never the source file itself.

## How Source Files Are Ingested Without Mutation

When you add a document to the project, LLM Wiki immediately creates a protected copy while generating editable derivatives elsewhere.

```markdown

# File system layout

/raw/sources/2024/paper.pdf     ←  User-added original (immutable)
/wiki/sources/paper.md          ←  LLM-generated summary (editable)

```

The ingestion pipeline in [`src/lib/source-lifecycle.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/source-lifecycle.ts) handles this automatically:

```typescript
// src/lib/source-lifecycle.ts
await copyFile(sourcePath, `${projectRoot}/raw/sources/${relativePath}`);
await preprocessFile(`${projectRoot}/raw/sources/${relativePath}`);
// Generates editable summary: wiki/sources/paper.md
// Original /raw/sources/2024/paper.pdf remains untouched

```

## Resolving Front-Matter References to Immutable Sources

Wiki pages reference immutable sources through YAML front matter, which the resolver maps to read-only filesystem locations.

```yaml

# wiki/notes.md

sources:
  - paper.pdf          # Resolves to raw/sources/2024/paper.pdf

```

During rendering, the resolution logic enforces the read-only constraint:

```typescript
// src/lib/wiki-page-resolver.ts
const path = resolveSourceName(index, "paper.pdf", sourcesRoot);
// Returns: "/project/raw/sources/2024/paper.pdf"
// Writes are redirected to wiki/sources/paper.md instead

```

## Guarding Against Disallowed File Types

The system prevents ingestion of potentially problematic files that could violate immutability assumptions. In [`src/lib/source-watch-config.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/source-watch-config.ts), the validation layer rejects unsupported formats before they enter the pipeline:

```typescript
// src/lib/source-watch-config.ts
if (!isPathAllowedBySourceWatch("/project/raw/sources/video.mp4", config)) {
  // video.mp4 is ignored – never ingested or stored in raw/sources
}

```

This filtering ensures only appropriate document types participate in the immutable storage system, while [`src/lib/source-identity.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/source-identity.ts) normalizes all valid source references to maintain canonical paths throughout the application.

## Summary

- **Read-only guarantee**: Files in `raw/sources` are never modified by the LLM or ingestion pipeline after initial copy.
- **Derivative workflow**: All editing occurs in `wiki/sources/*.md` files generated from the immutable originals.
- **Path enforcement**: `resolveSourceName` in [`src/lib/wiki-page-resolver.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-page-resolver.ts) treats raw source paths as read-only while routing mutations to wiki summaries.
- **Entry validation**: `isPathAllowedBySourceWatch` blocks binaries and hidden files from entering the immutable storage layer.

## Frequently Asked Questions

### Can I edit files directly in the raw/sources directory?

No. The architecture explicitly forbids modifications to files in `raw/sources`. If you need to update content, delete the original file and re-import it, or edit the generated markdown summary in `wiki/sources` that corresponds to your document.

### What happens if I add a binary file to raw/sources?

The `isPathAllowedBySourceWatch` function in [`src/lib/source-watch-config.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/source-watch-config.ts) filters out binaries, videos, and executables during the source-watching phase. These files are ignored by the ingestion pipeline and never copied into the immutable storage area, preventing corruption of the knowledge base with non-textual assets.

### How does the system prevent accidental overwrites of source documents?

Three mechanisms prevent overwrites: the source watcher only grants read permissions, the lifecycle functions in [`src/lib/source-lifecycle.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/source-lifecycle.ts) copy rather than move files, and the wiki resolver in [`src/lib/wiki-page-resolver.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/wiki-page-resolver.ts) routes all write operations to separate wiki pages. This creates a physical separation between storage (`raw/sources`) and working memory (`wiki/sources`).

### Where are the editable versions of my sources stored?

Editable summaries reside in `wiki/sources/` as markdown files with `.md` extensions. When you reference `paper.pdf` in your front matter, the system displays the content from [`wiki/sources/paper.md`](https://github.com/nashsu/llm_wiki/blob/main/wiki/sources/paper.md) while preserving the original PDF in `raw/sources/` as an immutable backup.