How Hydrus Persists and Loads Configuration Using Its Serializable Data System

Hydrus stores runtime configuration as versioned, JSON-serializable objects that convert to compact tuples for SQLite storage and reconstruct exactly using integer type IDs mapped to concrete classes.

The hydrusnetwork/hydrus media organization client relies on a robust serializable data system to persist user preferences, download rules, and UI state between sessions. Unlike simple JSON dumps, this architecture guarantees type safety, supports backward compatibility, and handles schema migrations automatically when the application updates.

The Core Serializable Hierarchy

All persistent configuration objects inherit from bases defined in hydrus/core/HydrusSerialisable.py. The hierarchy provides uniform methods for serialization, deserialization, and duplication.

SerialisableBase

The abstract foundation for every storable object, SerialisableBase defines the contract for converting instances to transportable formats. Its DumpToString method orchestrates the conversion by calling GetSerialisableTuple, then passing the result through json.dumps to produce a compact string representation 【line 77‑82】. Concretely, subclasses implement:

• _GetSerialisableInfo() – Returns the actual data payload
• _InitialiseFromSerialisableInfo() – Populates the object from loaded data
• _UpdateSerialisableInfo() – Handles version migrations

Specialized Containers

For complex configuration, Hydrus provides container classes that recursively serialize their contents:

• SerialisableBaseNamed – Extends the base with a human-readable name, storing tuples as (type, name, version, info) 【line 30‑33】
• SerialisableDictionary & SerialisableList – Wrap standard Python collections, converting nested objects using ConvertObjectToMetaSerialisableTuple and ConvertMetaSerialisableTupleToObject 【line 55‑74】
• SerialisableBytesDictionary – Optimized for raw bytes keys and values, encoding binary data as hexadecimal strings within the JSON structure 【line 86‑106】

Type Registration and Object Mapping

Hydrus identifies concrete classes using integer constants rather than class names to ensure stability across refactors. The system maintains a global dictionary SERIALISABLE_TYPES_TO_OBJECT_TYPES that maps these IDs to class constructors.

When defining a new configuration object, developers register it at the module level:

# From hydrus/client/ClientOptions.py (simplified)

SERIALISABLE_TYPES_TO_OBJECT_TYPES[ SERIALISABLE_TYPE_CLIENT_OPTIONS ] = ClientOptions

This registration occurs near the end of hydrus/client/ClientOptions.py around line 2173. During deserialization, CreateFromSerialisableTuple performs the lookup and instantiates the correct class before calling InitialiseFromSerialisableInfo to populate fields 【line 94‑100】.

Persisting Configuration to SQLite

Client configuration—including UI preferences, bandwidth rules, and tag filters—ultimately serializes to the SQLite database managed by ClientDB. The persistence flow follows a strict protocol:

1. The application calls GetJSONDump with the appropriate type ID (e.g., HydrusSerialisable.SERIALISABLE_TYPE_CLIENT_OPTIONS)
2. The method retrieves the object and calls obj.DumpToString() from SerialisableBase
3. The resulting JSON string stores as TEXT in the client_options table

In hydrus/client/db/ClientDB.py, this pattern appears throughout, such as at line 6267:

new_options = self.modules_serialisable.GetJSONDump(
    HydrusSerialisable.SERIALISABLE_TYPE_CLIENT_OPTIONS
)

The DumpToString method guarantees a consistent tuple format containing the type identifier, version number, and serialized payload, ensuring the database stores self-describing data.

Loading and Reconstructing Objects

During client startup, ClientController initiates configuration restoration by requesting the serializable object from the database layer. At line 1250 of hydrus/client/ClientController.py:

self.new_options = self.Read(
    'serialisable',
    HydrusSerialisable.SERIALISABLE_TYPE_CLIENT_OPTIONS
)

The Read operation fetches the stored JSON text and invokes HydrusSerialisable.CreateFromString(json_string). This static method parses the JSON, extracts the type tuple, looks up the concrete class via the global mapping, and reconstructs the object through InitialiseFromSerialisableInfo.

# Example: Loading configuration from raw database text

import hydrus.core.HydrusSerialisable as HydrusSerialisable

json_dump = db.execute('SELECT options FROM client_options').fetchone()[0]
client_options = HydrusSerialisable.CreateFromString(json_dump)

Version Migration and Schema Updates

The serializable data system handles software updates through explicit version tracking. Each class declares a SERIALISABLE_VERSION integer. When loading older data, InitialiseFromSerialisableInfo detects the version mismatch and iteratively applies _UpdateSerialisableInfo until the data structure matches the current schema 【line 36‑44】.

If the stored version exceeds the current code's understanding—meaning the database was written by a newer client—the system either raises an error or logs a warning based on the raise_error_on_future_version parameter 【line 15‑33】. This design allows cautious forward-compatibility while preventing data corruption.

Summary

• Type-safe serialization – Hydrus uses SerialisableBase subclasses with integer type IDs to ensure objects serialize to and deserialize from JSON tuples reliably.
• Centralized persistence – The ClientDB module stores configuration as TEXT fields in SQLite, using DumpToString for writes and CreateFromString for reads.
• Automatic migrations – Version numbers in serializable objects enable _UpdateSerialisableInfo to upgrade old configurations automatically when the schema evolves.
• Container support – Nested structures use SerialisableDictionary and SerialisableList to recursively process child objects while maintaining type information.

Frequently Asked Questions

How does Hydrus handle configuration schema changes over time?

Hydrus embeds a version integer in every serialized object's JSON tuple. When loading, InitialiseFromSerialisableInfo compares the stored version against the class's current SERIALISABLE_VERSION. If the stored version is older, the system calls _UpdateSerialisableInfo repeatedly to transform the data structure through each intermediate schema version until it matches the current code.

What is the difference between SerialisableBase and SerialisableBaseNamed?

SerialisableBase provides the core serialization protocol with methods like DumpToString and _GetSerialisableInfo. SerialisableBaseNamed extends this to include a human-readable name field in the serialized tuple, storing data as (type, name, version, info) rather than just (type, version, info). Named variants are used for user-visible configuration objects that require identification labels.

Where is configuration actually stored in the database?

Configuration objects store as JSON text in the SQLite database managed by hydrus/client/db/ClientDB.py. Specific tables like client_options hold the output of DumpToString() for each configuration type. The database uses the integer type ID (e.g., SERIALISABLE_TYPE_CLIENT_OPTIONS = 22) to distinguish which class should reconstruct each row.

Can the serializable system handle binary data efficiently?

Yes, through SerialisableBytesDictionary, which specializes in raw bytes keys and values. Rather than embedding binary data directly in JSON, this class encodes bytes as hexadecimal strings during serialization and decodes them back to bytes during loading 【line 86‑106】. This approach ensures binary data remains valid within JSON text fields while preserving exact byte values.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}