# Hydrus Service System Architecture: Local Repository Server Design

> Explore the Hydrus service system architecture. Learn how the local repository server design uses Twisted HTTP/HTTPS, a ServerController, SQLite, and networking for administration and data management.

- Repository: [Hydrus Network Developer/hydrus](https://github.com/hydrusnetwork/hydrus)
- Tags: architecture
- Published: 2026-03-03

---

**The Hydrus service system architecture implements a Twisted-based HTTP/HTTPS server that manages distinct services (administration, file repository, and tag repository) through a centralized ServerController, SQLite persistence layer, and specialized networking resources.**

The hydrusnetwork/hydrus repository uses this modular architecture of the service system to power its local repository server, allowing users to self-host file and metadata repositories with granular permission controls. This design separates bootstrapping, database operations, and HTTP request handling into discrete components that communicate through well-defined interfaces. The architecture supports multiple concurrent service types running on separate ports while maintaining a unified persistence layer under `hydrus/server/`.

## Core Components of the Service System

### Bootstrapping and Reactor Initialization

The entry point [`hydrus/hydrus_server_boot.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/hydrus_server_boot.py) parses command-line options, decides whether to start, stop, or restart the daemon, and launches the Twisted reactor in a dedicated thread. This bootstrapping component initializes the global state and prepares the environment before handing control to the main orchestrator.

### The ServerController Orchestrator

At the heart of the system sits [`hydrus/server/ServerController.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/server/ServerController.py). The `Controller` class initializes the `ServerDB` instance, manages the session manager, and maintains the list of active services. During startup, it calls `InitModel` followed by `InitView` to prepare the data layer and networking stack. The controller exposes `SetRunningTwistedServices(services)`, which iterates over each service definition—generated via `HydrusNetwork.GenerateService`—and binds them to SSL listeners.

### Persistence and Storage Layers

Data durability relies on two primary abstractions:

- **[`hydrus/server/ServerDB.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/server/ServerDB.py)** – Implements the SQLite-backed persistence layer for accounts, sessions, files, tags, and petitions. It provides the `Read` and `WriteSynchronous` methods that resources invoke to query or mutate state.
- **[`hydrus/server/ServerFiles.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/server/ServerFiles.py)** – Maps cryptographic file hashes to deterministic on-disk paths under `server_files/`, handling physical storage independently of the database schema.
- **[`hydrus/server/ServerGlobals.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/server/ServerGlobals.py)** – Exposes a global reference (`server_controller`) used by networking resources to reach the controller without circular imports.

### Networking and Resource Handling

The HTTP layer resides in `hydrus/server/networking/`:

- **[`ServerServer.py`](https://github.com/hydrusnetwork/hydrus/blob/main/ServerServer.py)** – Constructs Twisted `Resource` trees for each service type and registers URL endpoints. It calls `listenSSL` for each service port (default admin port **45871**).
- **[`ServerServerResources.py`](https://github.com/hydrusnetwork/hydrus/blob/main/ServerServerResources.py)** – Contains concrete request handlers such as `HydrusResourceAccessKey` and `HydrusResourceRestrictedRepositoryFile`. These classes parse GET/POST arguments via `HydrusNetworkVariableHandling`, check `HG.server_busy` status, validate account permissions, and delegate to `SG.server_controller.Read` or `WriteSynchronous`.

## Service Types and Registration Flow

The architecture supports three primary service types defined in the `HC` constants module:

- **SERVER_ADMIN** (`HC.SERVER_ADMIN`) – Runs on port 45871 by default; provides administration UI and maintenance actions (backup, lock, shutdown).
- **FILE_REPOSITORY** (`HC.FILE_REPOSITORY`) – Stores raw files and thumbnails; exposes endpoints like `/file` and `/thumbnail`.
- **TAG_REPOSITORY** (`HC.TAG_REPOSITORY`) – Manages tag-to-hash mappings; supports `/update`, `/metadata`, and `/tag_filter`.

During initialization, `Controller.SetRunningTwistedServices` creates a `ServerServer.HydrusService*` subclass for each configured service, builds a URL root (e.g., `https://127.0.0.1:45871/`), and registers it with the Twisted reactor.

## Request Handling Lifecycle

A complete HTTP request follows this path through the architecture:

1. **Startup** – [`hydrus_server_boot.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus_server_boot.py) spawns the reactor thread and instantiates the `Controller`.
2. **Controller Init** – The controller creates `ServerDB`, reads the admin service configuration, and invokes `InitModel` → `InitView`.
3. **Service Registration** – `SetRunningTwistedServices` iterates over service objects, binds each to a port via `listenSSL`, and attaches the appropriate resource tree.
4. **Request Dispatch** – Twisted receives an HTTPS request and routes it to the corresponding `HydrusResource*` class in [`ServerServerResources.py`](https://github.com/hydrusnetwork/hydrus/blob/main/ServerServerResources.py). The resource checks permissions and calls `Read` or `WriteSynchronous` on the controller.
5. **Database Interaction** – `ServerDB` executes SQL queries against tables such as `files_info`, `hashes`, `tags`, and `accounts`, while `ServerFiles` mirrors uploaded content to disk.
6. **Shutdown** – On SIGINT or a `/shutdown` request, the controller stops Twisted listeners, flushes pending writes, and releases global locks.

## Starting the Server and Uploading Files

The following example demonstrates the bootstrap-to-upload pipeline:

```python

# Start the local server (equivalent to running `hydrus_server.py start`)

import subprocess, os, time

# Assume the repository lives in ~/hydrus_repo

repo_dir = os.path.expanduser('~/hydrus_repo')
os.makedirs(repo_dir, exist_ok=True)

# Launch the server in a subprocess (the boot script will spawn Twisted)

proc = subprocess.Popen(
    ['python3', '-m', 'hydrus.hydrus_server_boot', 'start',
     '--db_dir', repo_dir],
    stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT,
    text=True,
)

# Wait a few seconds for the server to bind its admin port

time.sleep(5)
print('Server started, admin port should be listening on 45871')

```

Once running, clients interact with the service system via the HTTP API:

```bash

# Upload a file via the HTTP API (requires registration key → access key)

# 1️⃣ Get a registration key (admin account can request one)

curl -s -X GET "https://127.0.0.1:45871/auto_create_registration_key?account_type_key=admin" \
    --insecure | python -c "import sys, json; print(json.load(sys.stdin)['registration_key'])"

# → prints a 64‑byte registration key (hex)

# 2️⃣ Exchange the registration key for an access key

REG_KEY=<registration_key_from_above>
curl -s -X GET "https://127.0.0.1:45871/access_key?registration_key=$REG_KEY" \
    --insecure | python -c "import sys, json; print(json.load(sys.stdin)['access_key'])"

# → prints an access key

# 3️⃣ Obtain a session cookie (required for uploading)

ACCESS_KEY=<access_key_from_above>
curl -i -X GET "https://127.0.0.1:45871/session_key?access_key=$ACCESS_KEY" \
    --insecure | grep Set-Cookie

# → Set-Cookie: session_key=...

# 4️⃣ Upload a file to the file repository service (assume its port is 45872)

FILE_REPO_PORT=45872
curl -X POST "https://127.0.0.1:$FILE_REPO_PORT/file" \
    -H "Cookie: session_key=$(echo $COOKIE | cut -d'=' -f2)" \
    -F "hash=$(sha256sum myimage.jpg | cut -d' ' -f1)" \
    -F "size=$(stat -c%s myimage.jpg)" \
    -F "mime=image/jpeg" \
    -F "file=@myimage.jpg" \
    --insecure

```

## Summary

- The **Hydrus service system architecture** relies on a Twisted-based HTTP/HTTPS server that hosts multiple concurrent services on separate ports.
- **[`ServerController.py`](https://github.com/hydrusnetwork/hydrus/blob/main/ServerController.py)** acts as the central orchestrator, managing the SQLite database ([`ServerDB.py`](https://github.com/hydrusnetwork/hydrus/blob/main/ServerDB.py)), session state, and service lifecycle.
- **Three service types**—SERVER_ADMIN, FILE_REPOSITORY, and TAG_REPOSITORY—expose distinct REST endpoints registered via [`ServerServer.py`](https://github.com/hydrusnetwork/hydrus/blob/main/ServerServer.py).
- **Physical storage** is abstracted by [`ServerFiles.py`](https://github.com/hydrusnetwork/hydrus/blob/main/ServerFiles.py), which maps hashes to paths under `server_files/`, while [`ServerGlobals.py`](https://github.com/hydrusnetwork/hydrus/blob/main/ServerGlobals.py) provides global access to the running controller.
- Requests flow from Twisted resources in [`ServerServerResources.py`](https://github.com/hydrusnetwork/hydrus/blob/main/ServerServerResources.py) through permission checks to synchronous database reads/writes.

## Frequently Asked Questions

### What is the role of the ServerController in Hydrus?

The `ServerController` class in [`hydrus/server/ServerController.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/server/ServerController.py) serves as the central orchestrator for the entire service system. It initializes the SQLite database via `ServerDB`, manages the session manager, and drives the main event loop. The controller also handles graceful shutdown by stopping Twisted listeners and flushing pending database writes.

### How does Hydrus handle file storage in the service system?

File storage uses a dual-layer approach: [`hydrus/server/ServerDB.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/server/ServerDB.py) tracks file metadata, hashes, and ownership in SQLite, while [`hydrus/server/ServerFiles.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/server/ServerFiles.py) maps each file hash to a deterministic on-disk path under `server_files/`. This separation allows the database to manage logical relationships while the storage layer handles physical byte persistence.

### What types of services can run in a Hydrus repository?

The architecture supports three core service types defined in the `HC` constants module: **SERVER_ADMIN** for maintenance and user management (default port 45871), **FILE_REPOSITORY** for raw file and thumbnail storage, and **TAG_REPOSITORY** for tag-to-hash mappings. Each service runs as an isolated Twisted SSL listener with its own resource tree.

### How does the boot process initialize the Twisted reactor?

The [`hydrus/hydrus_server_boot.py`](https://github.com/hydrusnetwork/hydrus/blob/main/hydrus/hydrus_server_boot.py) script parses command-line arguments to determine the run mode (start, stop, or restart), then creates a dedicated thread for the Twisted reactor. This bootstrapping component instantiates the `ServerController`, which subsequently calls `SetRunningTwistedServices` to register each service's SSL endpoint with the reactor before entering the main event loop.