# How Hydrus Manages Physical File Storage Locations: A Deep Dive into Hash-Based Directory Trees

> Learn how Hydrus manages physical file storage with hash-based directory trees. Explore deterministic folder generation and file distribution across storage locations.

- Repository: [Hydrus Network Developer/hydrus](https://github.com/hydrusnetwork/hydrus)
- Tags: deep-dive
- Published: 2026-03-03

---

**Hydrus stores every imported file in a hierarchical, prefix-based directory tree derived from SHA-256 hashes, distributing files across weighted base locations using deterministic folder generation.**

The hydrusnetwork/hydrus client manages massive media collections through a sophisticated physical file storage system that translates cryptographic hashes into predictable folder paths. This architecture balances storage across multiple disks while enabling seamless reorganization and retrieval. Understanding how Hydrus manages file locations reveals a design optimized for scalability, deterministic access, and dynamic capacity management.

## Base Locations and Weighted Storage Distribution

Hydrus organizes storage into **base locations**—root directories configured with specific capacity limits and relative weights. The system selects where to write new files based on which location is furthest from its ideal storage weight.

### FilesStorageBaseLocation Architecture

The `FilesStorageBaseLocation` class in [`hydrus/client/files/ClientFilesPhysical.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/client/files/ClientFilesPhysical.py) encapsulates each storage root. According to the source code (lines 77‑86), each location maintains:

- A **real path** for absolute file system access
- A **portable path** for cross-platform compatibility  
- An **ideal weight** defining the target percentage of total storage
- An optional **max-bytes limit** to prevent disk overflow

Capacity checks (lines 41‑57) determine whether a location can accept additional sub-folders or requires clearing. When the client evaluates write targets, it calculates current weight distributions and selects the location with the greatest deficit relative to its ideal allocation.

### Weight Distribution Algorithm

The static method `STATICGetIdealWeights` (referenced in lines 27‑110) implements the distribution logic. When multiple base locations are defined in the client database under service `HYDRUS_LOCAL_FILE_STORAGE`, Hydrus computes the optimal prefix allocation across locations. The algorithm prioritizes locations with available capacity and the largest negative deviation from their target weight, ensuring balanced utilization across heterogeneous storage configurations.

## Prefix Generation from SHA-256 Hashes

All file placement derives from the content’s SHA-256 hash, transformed into a hexadecimal string and segmented into folder names.

### GetPrefix Implementation and Granularity

The core prefix generation resides in [`hydrus/core/files/HydrusFilesPhysicalStorage.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/core/files/HydrusFilesPhysicalStorage.py). The function `GetPrefix(hash, prefix_type, prefix_length)` (lines 31‑34) extracts segments from the hexadecimal hash representation:

```python
from hydrus.core.files import HydrusFilesPhysicalStorage

hash_bytes = b'\x3a\x5f\xf9...'  # 32-byte SHA-256 digest

prefix = HydrusFilesPhysicalStorage.GetPrefix(
    hash_bytes, 
    'f',  # 'f' for raw files, 't' for thumbnails

    HydrusFilesPhysicalStorage.DEFAULT_PREFIX_LENGTH  # defaults to 2

)

# Returns: "f3a5"

```

The **default granularity** is defined as `DEFAULT_PREFIX_LENGTH = 2`, creating 256 top-level folders per location (16² combinations). Administrators may increase this to 3 or higher to reduce per-folder file counts, generating up to 4,096 folders (16³).

### Prefix Types and Folder Hierarchy

The prefix system distinguishes content types through the first character:

- **'f'** prefixes denote raw imported files
- **'t'** prefixes denote generated thumbnails

The remaining characters form the directory hierarchy. For a prefix length of 2, the structure creates folders like `f3/a5/` beneath the base location root.

## Subfolder Resolution and Path Construction

Once Hydrus determines the appropriate base location and calculates the hash prefix, it constructs the nested directory structure through the `FilesStorageSubfolder` class.

### FilesStorageSubfolder Mechanics

Located in [`hydrus/client/files/ClientFilesPhysical.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/client/files/ClientFilesPhysical.py) (lines 48‑56), this class splits the hexadecimal portion of the prefix into two-character chunks. A prefix of `f3a5` becomes the array `['f3', 'a5']`, which the system joins onto the base location’s root path to form the absolute folder path.

The class exposes `GetFilePath(filename)` to append the actual filename—typically the full hexadecimal hash with extension—to the constructed directory path. This ensures every file rests at a predictable location: `<base>/<chunk1>/<chunk2>/<hash>.ext`.

### Building Absolute File Paths

When the client retrieves a file, it follows this resolution chain:

```python
from hydrus.core.files import HydrusFilesPhysicalStorage
from hydrus.client.files.ClientFilesPhysical import FilesStorageBaseLocation, FilesStorageSubfolder

# Base location normally loaded from database configuration

base = FilesStorageBaseLocation('/home/user/hydrus/storage', ideal_weight=1)

hash_bytes = b'\x3a\x5f\xf9...'
prefix = HydrusFilesPhysicalStorage.GetPrefix(
    hash_bytes, 'f', 
    HydrusFilesPhysicalStorage.DEFAULT_PREFIX_LENGTH
)

subfolder = FilesStorageSubfolder(prefix, base)
filename = f'{hash_bytes.hex()}.ext'
full_path = subfolder.GetFilePath(filename)

# Result: /home/user/hydrus/storage/f3/a5/3a5ff9....ext

```

This mirrors the lookup performed in [`hydrus/client/files/ClientFilesManager.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/client/files/ClientFilesManager.py) (line 508), where the client calls `HydrusFilesPhysicalStorage.GetPrefix` to resolve file requests.

## Rebalancing Storage with RegranulariseFileStorage

Hydrus supports dynamic reorganization through the `RegranulariseFileStorage` function in [`hydrus/client/files/ClientFilesPhysical.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/client/files/ClientFilesPhysical.py), enabling administrators to change folder granularity or redistribute files across base locations without breaking references.

### Changing Granularity Levels

The re-granulation process (lines 91‑124) accepts a starting prefix length (e.g., 2) and a target length (e.g., 3). It enumerates all existing source subfolders using `IteratePrefixes` (lines 36‑46 in the core storage module), then computes destination prefixes for the new granularity level.

### Moving Files Between Locations

The rebalancing workflow (lines 150‑210) scans each source subfolder and matches files to their destination prefixes based on hash analysis. It builds rename jobs mapping `source_path → destination_path`, executes them in batches (lines 222‑242), and cleans up empty source directories (lines 245‑251).

```python
from hydrus.client.files.ClientFilesPhysical import RegranulariseFileStorage
from hydrus.client import ClientThreading

job_status = ClientThreading.JobStatus('Regranularising', total_work=1)
base_paths = ['/home/user/hydrus/storage']

new_map, moved, weird_dirs, weird_files = RegranulariseFileStorage(
    base_location_paths=base_paths,
    prefix_types=['f', 't'],
    starting_prefix_length=2,
    ending_prefix_length=3,
    job_status=job_status
)

```

The function returns a mapping of new prefixes to canonical base-location paths, along with statistics on moved files and any non-conforming directories requiring manual intervention.

## Summary

- **Base locations** (`FilesStorageBaseLocation`) define weighted root directories with capacity limits and portable path handling
- **Prefix generation** (`HydrusFilesPhysicalStorage.GetPrefix`) creates deterministic folder names from SHA-256 hashes, using 2-character granularity by default
- **Subfolder construction** (`FilesStorageSubfolder`) splits prefixes into two-character chunks to build nested directory hierarchies
- **File resolution** combines base locations, prefixes, and filenames to produce absolute paths without database lookups during retrieval
- **Rebalancing** (`RegranulariseFileStorage`) enables live migration between granularity levels and across storage locations while maintaining hash-based organization

## Frequently Asked Questions

### How does Hydrus determine which hard drive to store a file on?

Hydrus evaluates all configured base locations and selects the one whose current storage weight is furthest below its ideal weight while still having available capacity. The `FilesStorageBaseLocation` class tracks ideal weights and max-bytes limits, and the selection algorithm prioritizes locations with the largest negative deviation from their target allocation to maintain balanced distribution across multiple disks.

### What is the default folder structure depth in Hydrus storage?

The default configuration uses a prefix length of 2 (`DEFAULT_PREFIX_LENGTH = 2`), creating 256 top-level folders per base location. Each folder contains subfolders named from subsequent two-character pairs of the hexadecimal hash, typically resulting in paths like `f3/a5/` beneath the storage root. This provides 65,536 possible leaf directories (256 × 256) for file distribution.

### Can I change the folder granularity after importing files?

Yes. The `RegranulariseFileStorage` function in [`hydrus/client/files/ClientFilesPhysical.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/client/files/ClientFilesPhysical.py) supports live re-granulation from any source prefix length to a different target length. The system enumerates existing prefixes, computes new destinations based on hash analysis, and performs batched file moves while updating internal mappings, allowing migration from 2-character to 3-character granularity without data loss.

### How does Hydrus handle thumbnail storage differently from raw files?

Thumbnails use the prefix type **'t'** rather than **'f'**, causing them to reside in parallel directory trees (e.g., `t3/a5/` instead of `f3/a5/`). The `GetPrefix` function accepts a `prefix_type` parameter that directs files into separate hierarchies within the same base locations, enabling independent granularity management and storage weighting for thumbnails versus raw media files.