What Is the watcher.py Script in RomM? Automated Filesystem Monitoring Explained

The watcher.py script in RomM monitors your ROM library filesystem for changes and automatically triggers platform rescans to keep your database synchronized without manual intervention.

The watcher.py script serves as the automated bridge between your filesystem and RomM's scanning pipeline. Located in the backend/ directory of the RomM repository, this Python script detects when ROMs are added or removed from your library and initiates appropriate rescans with configurable delays. Understanding how watcher.py functions helps administrators optimize library synchronization and prevent unnecessary background processing.

Core Functionality of the RomM watcher.py Script

The watcher.py script operates as a backend component that translates filesystem events into scheduled scan jobs. It runs as a separate process within the RomM container, waiting for notifications from the file watcher before executing its logic.

Configuration and Initialization

At startup, watcher.py imports critical settings from the configuration manager in config/config_manager.py. These include ENABLE_RESCAN_ON_FILESYSTEM_CHANGE, which toggles automatic monitoring, and RESCAN_ON_FILESYSTEM_CHANGE_DELAY, which defines the cooldown period before triggering a scan. The script also initializes Sentry for error tracking and sets up an OpenTelemetry tracer for observability.

The script relies on the WATCHFILES_CHANGES environment variable to receive filesystem event data. This variable is typically populated by the watchfiles wrapper that monitors the library directory.

Duplicate Prevention Mechanism

Before scheduling any new scan, watcher.py calls get_pending_scan_jobs() to query the RQ (Redis Queue) job queue and active workers. This function scans for any existing scan_platforms jobs that are already queued, scheduled, or currently running. By checking for pending jobs first, the script prevents duplicate rescans that could overwhelm the system when multiple files change simultaneously.

Processing Filesystem Changes

The process_changes(changes) function receives a list of (event_type, path) tuples parsed from the WATCHFILES_CHANGES environment variable. This function implements three critical filtering layers:

1. Event Filtering: Only added and deleted events are processed; modified files are ignored to reduce noise.
2. Path Exclusion: The _is_excluded(path) helper function filters out hidden directories like .romm_tmp_* and applies user-defined exclusion patterns from the configuration.
3. Platform Detection: The script extracts affected platform slugs from the paths and determines which metadata sources (IGDB, MOBY, etc.) are enabled for processing.

def _is_excluded(path: str) -> bool:
    parts = path.strip("/").split("/")
    for part in parts:
        if part.startswith(".romm_tmp_"):
            return True
        if any(part == pat or fnmatch.fnmatch(part, pat) for pat in excluded_patterns):
            return True
    return False

How watcher.py Schedules Full and Quick Rescans

The script intelligently determines the scope of rescanning required based on where the filesystem change occurred within the library hierarchy.

Full vs. Quick Scan Detection

watcher.py distinguishes between two types of changes:

• Full Rescan: Triggered when the root of a platform directory is added or deleted, requiring a complete re-indexing of that platform.
• Quick Rescan: Triggered when changes occur inside an existing platform folder, scanning only the affected platform with minimal overhead.

This distinction allows RomM to minimize resource usage when only a single ROM file changes versus when an entire platform is restructured.

Integration with the Task Queue

Once the scan type is determined, watcher.py schedules the job using tasks_scheduler.enqueue_in() from tasks/tasks.py. The function accepts a timedelta calculated from RESCAN_ON_FILESYSTEM_CHANGE_DELAY, ensuring scans don't trigger immediately during active file transfers:

tasks_scheduler.enqueue_in(
    timedelta(minutes=RESCAN_ON_FILESYSTEM_CHANGE_DELAY),
    scan_platforms,
    platform_ids=[db_platform.id],
    metadata_sources=metadata_sources,
    scan_type=ScanType.QUICK,
    timeout=SCAN_TIMEOUT,
    job_result_ttl=TASK_RESULT_TTL,
    meta={"task_name": "Quick Scan", "task_type": TaskType.SCAN},
)

Entry Point and Execution Flow

The watcher.py script is executed by entrypoint.sh when the RomM container boots. The entry point script starts the watcher as a background process alongside other services.

When invoked, the script executes its main block:

if __name__ == "__main__":
    changes = cast(list[Change], json.loads(os.getenv("WATCHFILES_CHANGES", "[]")))
    if changes:
        process_changes(changes)

If no changes are present in the environment variable, the script exits silently without consuming additional resources. This design ensures the watcher only runs when actual filesystem events have occurred, making it efficient for long-running container deployments.

Summary

• watcher.py acts as the automated filesystem monitor for RomM, detecting ROM additions and deletions in real-time.
• The script prevents duplicate scans by checking the RQ job queue via get_pending_scan_jobs() before scheduling new tasks.
• Path exclusions are handled by _is_excluded(), which filters temporary files and user-defined patterns.
• The script intelligently chooses between full and quick rescans based on whether platform root directories or individual files changed.
• Execution is triggered via entrypoint.sh, reading changes from the WATCHFILES_CHANGES environment variable.

Frequently Asked Questions

What triggers the watcher.py script to run?

The watcher.py script is triggered by the WATCHFILES_CHANGES environment variable, which is set by the watchfiles wrapper monitoring your library directory. When the filesystem watcher detects relevant changes, it populates this variable with JSON-encoded event data and invokes the script, typically via entrypoint.sh during container startup.

How does watcher.py prevent duplicate scan jobs?

Before scheduling any scan, watcher.py calls get_pending_scan_jobs() to query the Redis Queue for existing scan_platforms jobs that are already queued, scheduled, or running. This check ensures that multiple rapid filesystem changes don't spawn redundant scan processes that would waste CPU and I/O resources.

What is the difference between a full scan and a quick scan in watcher.py?

A full scan is scheduled when watcher.py detects that an entire platform directory was added or removed, requiring a complete re-indexing of all ROMs in that platform. A quick scan is triggered when changes occur inside an existing platform folder, scanning only the affected platform with minimal overhead. The script determines which to use based on the path depth and event type of the filesystem change.

Can I configure which filesystem events watcher.py ignores?

Yes, watcher.py respects exclusion patterns defined in your RomM configuration. The _is_excluded(path) function in backend/watcher.py automatically ignores hidden directories starting with .romm_tmp_ and applies any user-specified exclusion patterns from the configuration manager. Additionally, the script only processes added and deleted events, ignoring file modifications to prevent excessive scanning during file updates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}