How LLM Wiki Handles Cascade Cleanup of Deleted Sources

LLM Wiki performs cascade cleanup of deleted sources by removing the markdown file, purging vector embeddings from LanceDB, deleting associated media directories, and sweeping all remaining wiki pages to strip references, clean index listings, and update front-matter arrays.

When a source file or its generated summary page is removed from the knowledge base, LLM Wiki must ensure no dangling references or orphaned data remain. The nashsu/llm_wiki project implements a robust cascade-delete workflow that spans the filesystem, vector database, and cross-page wikilinks to maintain consistency across the entire wiki structure.

The Core Deletion Workflow

The cascade cleanup operates through five coordinated stages, implemented primarily in src/lib/wiki-page-delete.ts and src/lib/wiki-cleanup.ts.

Step 1: Remove the Primary Page and Embeddings

The cascadeDeleteWikiPage function initiates the deletion sequence by targeting the source markdown file and its derived artifacts. For source-summary pages, this includes the dedicated media folder at wiki/media/<slug>/ that stores extracted images alongside the document.

The function drops all vector chunks associated with the page's slug from LanceDB, ensuring the semantic search index remains synchronized with the filesystem state. This prevents "ghost" embeddings from surfacing deleted content in query results.

Step 2: Capture Deletion Metadata and Keys

Before any file is erased, the system extracts the title from front-matter using extractFrontmatterTitle and captures the page slug. The buildDeletedKeys helper in src/lib/wiki-cleanup.ts normalizes this data into a comprehensive set of lookup keys representing all possible reference formats—slug form, title form, and any defined aliases.

This normalization is critical because wikilinks throughout the project may reference the deleted page using any of these variants.

Step 3: Sweep Wiki Files for References

After removing the target pages, cascadeDeleteWikiPagesWithRefs walks the entire wiki/ tree to eliminate residual references. This sweep performs three distinct cleanup operations:

• Index Cleanup: The cleanIndexListing function removes entries from wiki/index.md where the primary wikilink points to a deleted page.
• Body Cleanup: stripDeletedWikilinks locates all [[deleted]] or [[deleted|alias]] syntax in page bodies and replaces them with plain text.
• Front-matter Cleanup: The related: array in YAML front-matter is pruned of any references to the deleted keys.

All writes follow a best-effort pattern; a single unreadable file does not abort the entire operation, ensuring the cleanup remains resilient against filesystem edge cases.

Step 4: Clean Up Media Directories

Source-summary pages exclusively own image directories under wiki/media/. The cascade logic checks isSourcePage(pagePath) and, if true, deletes the corresponding media folder after the page file itself is removed. This prevents orphaned image folders while protecting non-source pages from accidental media removal.

Step 5: Error Handling and Fault Tolerance

The system implements tiered error handling to balance data integrity with operational continuity:

• File-system errors from initial page deletion or embedding removal propagate to callers, allowing the UI to display toasts or halt further processing.
• Media-directory deletion errors are caught and ignored—missing directories are treated as no-ops.
• All warnings are logged via console.warn to aid diagnostics without breaking user flow.

Entry Points and Integration

The cascade delete is invoked from multiple application surfaces depending on user action:

• Source View: The "delete" button in the source view triggers the cascade via source-delete logic.
• Lint View: Orphan-page removal operations call the cleanup routine to eliminate unlinked content.
• Ingest Queue: When a batch ingest is cancelled, src/lib/ingest-queue.ts calls the cascade to remove partially processed files and embeddings.
• Batch Operations: src/lib/source-lifecycle.ts exposes await cascadeDeleteWikiPagesWithRefs(projectPath, pagesToDelete) for bulk source deletion workflows.

Key Implementation Files

• src/lib/wiki-page-delete.ts: Core cascade-delete implementation including cascadeDeleteWikiPage, cascadeDeleteWikiPagesWithRefs, and media cleanup logic.
• src/lib/wiki-cleanup.ts: Pure-string utilities including extractFrontmatterTitle, buildDeletedKeys, cleanIndexListing, and stripDeletedWikilinks.
• src/lib/source-lifecycle.ts: Invokes cascade delete when source files are removed from the raw input folder.
• src/lib/ingest-queue.ts: Ensures cleanup occurs when ingest jobs are aborted mid-process.

Code Examples

Deleting a Single Source-Summary Page

import { cascadeDeleteWikiPage } from '@/lib/wiki-page-delete';

await cascadeDeleteWikiPage('/my/project', '/my/project/wiki/sources/rope-paper.md');
// → file removed
// → embedding chunks for slug "rope-paper" removed
// → /my/project/wiki/media/rope-paper/ directory deleted

Bulk Deletion with Reference Cleanup

import { cascadeDeleteWikiPagesWithRefs } from '@/lib/wiki-page-delete';

const pages = [
  '/my/project/wiki/sources/rope-paper.md',
  '/my/project/wiki/concepts/rope.md',
];
const result = await cascadeDeleteWikiPagesWithRefs('/my/project', pages);

console.log('Deleted:', result.deletedPaths);
console.log('Files rewritten:', result.rewrittenFiles);

Cleaning Index Entries

import { cleanIndexListing } from '@/lib/wiki-cleanup';

const indexMd = await readFile('/my/project/wiki/index.md');
const deletedKeys = new Set(['rope-paper', 'Rope Paper']); // slug & title normalized
const cleaned = cleanIndexListing(indexMd, deletedKeys);
await writeFile('/my/project/wiki/index.md', cleaned);

Stripping Wikilinks from Content

import { stripDeletedWikilinks } from '@/lib/wiki-cleanup';

const overview = await readFile('/my/project/wiki/overview.md');
const cleaned = stripDeletedWikilinks(overview, deletedKeys);
await writeFile('/my/project/wiki/overview.md', cleaned);

Summary

• cascadeDeleteWikiPage handles single-page deletion including LanceDB embedding removal and media folder cleanup for source pages.
• cascadeDeleteWikiPagesWithRefs orchestrates bulk deletion while sweeping the wiki tree to strip references and update indices.
• Normalized deletion keys derived from front-matter titles and slugs ensure all wikilink variants are caught during cleanup.
• Best-effort error handling allows the sweep to continue even if individual files are corrupted or locked.
• Integrated triggers across the source view, lint view, and ingest queue ensure consistency regardless of how content is removed.

Frequently Asked Questions

What happens to vector embeddings when a source is deleted?

When cascadeDeleteWikiPage executes, it removes all embedding chunks associated with the page's slug from LanceDB immediately after the markdown file is deleted. This ensures the semantic search pipeline does not return results pointing to non-existent content.

How does LLM Wiki prevent broken wikilinks after deletion?

The stripDeletedWikilinks utility scans every remaining wiki page and converts any [[deleted-page]] or [[deleted-page|alias]] references to plain text. Additionally, cleanIndexListing removes entries from wiki/index.md that point to deleted pages, and the front-matter related: arrays are pruned of stale references.

Are media files deleted automatically with their source pages?

Yes, but only for source-summary pages. The cascade logic checks isSourcePage(pagePath) and deletes the corresponding wiki/media/<slug>/ directory only when the source page itself is removed. Non-source pages do not trigger media deletion, preventing accidental loss of shared assets.

What error handling exists if a file is corrupted during cleanup?

The system employs best-effort writes during the reference-sweep phase. If a single file cannot be read or written during the cleanup sweep, the operation logs a warning via console.warn and continues processing remaining files. However, errors during the initial page deletion or embedding removal propagate to the caller, allowing the UI to decide whether to halt the operation or notify the user.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}