How the Hydrus Metadata Migration Exporter/Importer System Works

The Hydrus metadata migration exporter/importer system uses a modular node-based architecture where ImporterExporterNode classes handle reading from or writing to side-car files and the database, enabling bidirectional metadata transfer between media files and JSON/TXT side-cars.

The hydrusnetwork/hydrus metadata migration exporter/importer system provides a flexible framework for moving metadata between media files, side-car files, and the Hydrus database. This system allows users to export tags, notes, timestamps, and URLs to external JSON or TXT files, then re-import that data later or transfer it between different Hydrus clients. All components are implemented in the hydrus/client/metadata/ directory and follow a consistent inheritance pattern based on serialisable nodes.

Core Architecture Concepts

The system is built around three foundational abstractions defined in hydrus/client/metadata/ClientMetadataMigrationCore.py.

ImporterExporterNode and SidecarNode

The ImporterExporterNode class serves as the abstract base for both importers and exporters. It provides a common interface with Export and Import methods, plus a human-readable ToString method for UI display.

The SidecarNode mixin adds side-car handling capabilities to nodes that work with external files. It manages filename generation, removal of original extensions, suffix handling, and optional filename conversion. Any exporter or importer that reads from or writes to files inherits from this mixin.

Serialisation Support

Every exporter and importer inherits from HydrusSerialisable.SerialisableBase. This allows the UI to store and restore exact configurations—including service keys, string processors, and JSON parsing rules—between sessions. The serialisation ensures that complex metadata migration pipelines persist correctly across client restarts.

The Exporter Hierarchy

All exporters inherit from SingleFileMetadataExporter, which extends ImporterExporterNode. The concrete implementations in hydrus/client/metadata/ClientMetadataMigrationExporters.py fall into two categories: media-specific exporters that write directly to the database, and side-car exporters that create external files.

Media-Specific Exporters

These classes update the Hydrus database directly:

• SingleFileMetadataExporterMediaNotes – Writes media notes (name:text pairs) directly to the media's notes manager using CG.client_controller.WriteSynchronous('content_updates', …).
• SingleFileMetadataExporterMediaTags – Updates tag mappings via CONTENT_UPDATE_ADD or CONTENT_UPDATE_PEND operations.
• SingleFileMetadataExporterMediaTimestamps – Updates the timestamp manager for a specific stub (e.g., archived time).
• SingleFileMetadataExporterMediaURLs – Adds URLs to the media's location manager.

Side-Car File Exporters

These classes write to external files alongside the media:

• SingleFileMetadataExporterJSON – Writes arbitrary JSON objects to .json files with optional nested key structures.
• SingleFileMetadataExporterTXT – Writes plain-text lines to .txt files using a configurable separator.

All side-car exporters inherit from SingleFileMetadataExporterSidecar, which provides GetExportPath. This method calls ClientMetadataMigrationCore.GetSidecarPath to construct the output filename based on the original file path, a suffix string, whether to drop the original extension, and an optional filename converter.

Exporting Tags to JSON

from hydrus.client.metadata import ClientMetadataMigrationExporters as exporters

# Create a JSON exporter that stores tags under the "tags" key

json_exporter = exporters.SingleFileMetadataExporterJSON(
    remove_actual_filename_ext=False,
    suffix='tags',
    filename_string_converter=None,   # default converter

    nested_object_names=['tags']
)

# Export tags for a given file hash

hash = b'\x12\x34\x56...'   # 64-bit hash bytes

tag_rows = ['blue eyes', 'character:jane_smith']
json_exporter.Export(hash, tag_rows)

This creates a file named originalfilename.tags.json containing:

{ "tags": ["blue eyes", "character:jane_smith"] }

The Importer Hierarchy

Importers mirror the exporter structure in hydrus/client/metadata/ClientMetadataMigrationImporters.py, inheriting from SingleFileMetadataImporter. They read metadata either from the database or from side-car files and return standardized list formats.

Media-Specific Importers

• SingleFileMetadataImporterMediaNotes – Reads notes directly from a media result, returning a list of name:note strings.
• SingleFileMetadataImporterMediaTags – Reads tags from a media result with configurable service and display type filters.
• SingleFileMetadataImporterMediaTimestamps – Extracts a timestamp according to a specific stub (e.g., archived), returning a list with a single timestamp string.
• SingleFileMetadataImporterMediaURLs – Retrieves URLs from the media's location manager.

Side-Car File Importers

• SingleFileMetadataImporterJSON – Parses .json side-car files using a ParseFormulaJSON to extract specific items.
• SingleFileMetadataImporterTXT – Reads .txt files and splits content on a configurable separator.

String Processing Pipeline

Importers utilize a ClientStrings.StringProcessor that applies transformations—such as strip, replace, or regex operations—to raw strings before returning them to the calling code. This allows normalization of tag formats or cleanup of imported data during the migration process.

Importing Tags from JSON

from hydrus.client.metadata import ClientMetadataMigrationImporters as importers

json_importer = importers.SingleFileMetadataImporterJSON(
    string_processor=None,
    remove_actual_filename_ext=False,
    suffix='tags',
    filename_string_converter=None,
    json_parsing_formula=None   # defaults to "all items"

)

file_path = '/path/to/image.jpg'          # original media file

tag_rows = json_importer.Import(file_path)   # → ['blue eyes', 'character:jane_smith']

Wiring the UI and Router Together

The UI layer in hydrus/client/gui/metadata/ClientGUIMetadataMigrationExporters.py and ClientGUIMetadataMigrationImporters.py presents available classes in dropdown menus. When a user selects an exporter or importer, the system instantiates the class, serialises the configuration, and stores it in the client options.

The router concept, defined in hydrus/client/gui/metadata/ClientGUIMetadataMigration.py, holds an ordered list of exporter or importer nodes. When executing an export or import operation, Hydrus iterates over selected media files and calls the appropriate Export or Import method on each node in sequence. This design creates a plug-in-style pipeline where each node handles a specific metadata representation without affecting the surrounding logic.

Practical Implementation Examples

Export Tags to a TXT Side-Car

from hydrus.client.metadata import ClientMetadataMigrationExporters as exp

txt_exporter = exp.SingleFileMetadataExporterTXT(
    remove_actual_filename_ext=True,
    suffix='tags',
    filename_string_converter=None,
    separator='\n'
)

hash = b'\xab\xcd\xef...'          # file hash

tags = ['blue eyes', 'blonde hair']
txt_exporter.Export(hash, tags)    # creates "filename.tags.txt"

Import Tags from the TXT Side-Car

from hydrus.client.metadata import ClientMetadataMigrationImporters as imp

txt_importer = imp.SingleFileMetadataImporterTXT()
imported_tags = txt_importer.Import('/path/to/filename.jpg')
print(imported_tags)   # ['blue eyes', 'blonde hair']

Summary

• The metadata migration system relies on ImporterExporterNode and SidecarNode abstractions to provide a consistent interface for file and database operations.
• Exporters and importers inherit from SingleFileMetadataExporter and SingleFileMetadataImporter, respectively, with concrete implementations handling specific formats like JSON, TXT, or direct database updates.
• All nodes are HydrusSerialisable.SerialisableBase instances, enabling persistent storage of complex configurations including string processors and parsing formulas.
• The UI router in ClientGUIMetadataMigration.py orchestrates execution order, allowing users to chain multiple exporters or importers into sequential pipelines.
• Side-car path generation is handled centrally via ClientMetadataMigrationCore.GetSidecarPath, supporting customizable suffixes and filename transformations.

Frequently Asked Questions

What file formats does the metadata migration system support for side-car files?

The system natively supports JSON and TXT formats through SingleFileMetadataExporterJSON, SingleFileMetadataImporterJSON, SingleFileMetadataExporterTXT, and SingleFileMetadataImporterTXT. JSON exporters support nested object structures, while TXT exporters use configurable separators. The modular architecture in ClientMetadataMigrationCore.py allows for straightforward extension to additional formats by implementing new subclasses of the base exporter and importer nodes.

Can metadata be written directly to the Hydrus database instead of side-car files?

Yes. Classes like SingleFileMetadataExporterMediaTags, SingleFileMetadataExporterMediaNotes, and SingleFileMetadataExporterMediaTimestamps write directly to the database using CG.client_controller.WriteSynchronous calls. These bypass side-car generation entirely and update tag mappings, notes managers, and timestamp managers immediately. This is useful for importing metadata from external sources directly into the Hydrus media library.

How does the system handle filename generation for side-car files?

Side-car filename generation is handled by ClientMetadataMigrationCore.GetSidecarPath, called through SingleFileMetadataExporterSidecar.GetExportPath. The routine accepts parameters including a suffix string (e.g., 'tags'), a boolean flag remove_actual_filename_ext to strip the original extension, and an optional filename_string_converter. This allows generation of paths like image.tags.json or image.tags depending on configuration.

What is the purpose of the StringProcessor in importers?

The ClientStrings.StringProcessor provides a transformation pipeline for raw imported strings before they are returned to the application. Importers like SingleFileMetadataImporterJSON and SingleFileMetadataImporterTXT accept a string_processor parameter that can apply operations such as whitespace stripping, substring replacement, or regex matching. This ensures imported metadata conforms to Hydrus tagging standards or user-specific formatting requirements without requiring manual post-processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}