# How LLM Wiki Preserves Directory Structures During Folder Imports

> Discover how LLM Wiki maintains directory structures during folder imports by recreating your exact file hierarchy within the project for seamless organization.

- Repository: [nash_su/llm_wiki](https://github.com/nashsu/llm_wiki)
- Tags: internals
- Published: 2026-09-12

---

**LLM Wiki preserves directory structures by calculating each file's path relative to the imported folder root, then recreating that exact hierarchy inside the project's `raw/sources` directory using atomic file operations.**

When importing external folders into a knowledge base, maintaining the original organizational structure is critical for context preservation. The `nashsu/llm_wiki` project implements a robust folder import system that mirrors external hierarchies exactly inside the project's `raw/sources` directory. Understanding how LLM Wiki preserves directory structures during folder imports reveals a pattern of relative path resolution combined with recursive directory reconstruction.

## The Import Pipeline in source-lifecycle.ts

The core logic resides in [`src/lib/source-lifecycle.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/source-lifecycle.ts), specifically within the `importSourceFolder` function. This implementation uses a four-stage pipeline to ensure faithful replication of complex folder trees.

### Step 1: Recursive File Discovery

First, the system collects all files including those nested in subdirectories and hidden folders. The `flattenFiles` utility combines with `listDirectory` to produce a flat array of absolute paths while traversing the entire tree:

```typescript
const sourceFiles = flattenFiles(await listDirectory(selectedFolder, true));

```

This operation, found at line 394, ensures that no content is missed regardless of nesting depth. The `true` parameter enables recursive descent into subdirectories.

### Step 2: Relative Path Calculation

For each discovered file, the system computes its position relative to the import root using `getRelativePath` from [`src/lib/path-utils.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/path-utils.ts):

```typescript
const relativeSourcePath = getRelativePath(file.path, sourceRoot);

```

Located at line 397, this calculation strips the external folder's base path while preserving the internal hierarchy. This relative path becomes the blueprint for destination placement.

### Step 3: Directory Recreation and File Copying

The destination path is constructed by appending the relative path to the project's `raw/sources` directory:

```typescript
const destPath = `${destDir}/${relativeSourcePath}`;

```

Before copying, the system ensures parent directories exist. At lines 420-423, `parentPath` extracts the directory component, `createDirectory` builds missing intermediate folders, and `copyFile` performs the atomic transfer:

```typescript
const parent = parentPath(destPath);
if (parent) await createDirectory(parent);
await copyFile(file.path, destPath);

```

This sequence guarantees that deep hierarchies like `notes/2024/january/` are recreated faithfully even if multiple parent levels are missing.

### Step 4: Deterministic Ordering

After successful imports, the file list is sorted using `naturalCompare` at lines 430-433. This alphabetic ordering ensures consistent processing sequences across different operating systems and filesystems.

## Safety Mechanisms and Edge Cases

Beyond basic copying, the implementation includes safeguards for production environments.

### Handling Hidden Files and Dot-Folders

The system explicitly honors hidden directories such as `.claude` or `.git` because `flattenFiles` includes them in its traversal (lines 90-94). However, it filters out tool-configuration files that might contain secrets, balancing completeness with security.

### Preventing Self-Referential Imports

To avoid infinite recursion or circular copies, the code rejects attempts to import the project folder itself or any of its subdirectories. Lines 81-83 validate the source path against the project root before processing begins.

## Practical Implementation Examples

Developers can trigger imports programmatically using the exposed API.

Standard programmatic usage:

```typescript
import { importSourceFolder } from "@/lib/source-lifecycle";

await importSourceFolder(
  { id: "proj1", name: "MyWiki", path: "/home/user/mywiki" },
  "/home/user/external-notes",
  llmConfig,
  sourceWatchConfig
);

```

This execution transforms an external structure like:

```

/home/user/external-notes/
├─ notes/
│  └─ meeting.md
└─ .claude/
   └─ research.md

```

Into:

```

/home/user/mywiki/raw/sources/imported/
├─ notes/
│  └─ meeting.md
└─ .claude/
   └─ research.md

```

Filtered import with configuration:

```typescript
await importSourceFolder(
  { id: "p1", name: "Project", path: "/project" },
  "/external/imported",
  fakeLlmConfig,
  {
    enabled: true,
    includeExtensions: ["md"],
    excludeExtensions: ["json"],
    excludeDirs: ["drafts"],
    maxFileSizeMb: 100,
  },
);

```

This configuration selectively imports only Markdown files while excluding JSON configurations and draft directories.

## Summary

- **Relative path resolution** via `getRelativePath` ensures the original folder hierarchy is mathematically preserved during transit from source to destination.
- **Atomic directory creation** using `parentPath` and `createDirectory` guarantees that deep nesting levels are reconstructed exactly, even with missing intermediate folders.
- **Recursive traversal** through `flattenFiles` and `listDirectory` captures all content including hidden dot-folders while excluding sensitive configuration files.
- **Self-reference protection** prevents catastrophic circular copying by validating the import target against the project root.
- **Deterministic ordering** with `naturalCompare` maintains consistent file sequences across different platforms and filesystem implementations.

## Frequently Asked Questions

### Does LLM Wiki maintain the exact folder hierarchy when importing nested directories?

Yes. The system calculates each file's path relative to the imported folder root using `getRelativePath`, then reconstructs that identical path inside `raw/sources`. This approach preserves unlimited nesting depths, ensuring that a file located at [`external/docs/teams/engineering/specs.md`](https://github.com/nashsu/llm_wiki/blob/main/external/docs/teams/engineering/specs.md) appears at [`project/raw/sources/imported/docs/teams/engineering/specs.md`](https://github.com/nashsu/llm_wiki/blob/main/project/raw/sources/imported/docs/teams/engineering/specs.md).

### How does the import system handle hidden folders like `.git` or `.claude`?

The `flattenFiles` utility explicitly includes dot-folders in its traversal (lines 90-94), meaning hidden directories are preserved in the imported structure. However, the system filters out specific tool-configuration files that might contain API keys or secrets, striking a balance between completeness and security.

### What prevents importing the project folder into itself?

Lines 81-83 in [`src/lib/source-lifecycle.ts`](https://github.com/nashsu/llm_wiki/blob/main/src/lib/source-lifecycle.ts) implement a validation check that compares the import source path against the project root. If the system detects a self-referential import attempt—such as importing a parent directory containing the wiki project—it rejects the operation to prevent infinite recursion and circular file copying.

### Can I filter which files get imported while still preserving directory structure?

Yes. The `importSourceFolder` function accepts a configuration object allowing `includeExtensions`, `excludeExtensions`, and `excludeDirs` parameters. These filters apply during the file discovery phase, meaning only matched files are copied, but those files maintain their original relative paths within the preserved hierarchy.