# How Immich's LibraryService Manages External Library Synchronization and Scanning

> Discover how Immich's LibraryService synchronizes and scans external libraries with database locks, file watchers, and multi-phase batch scanning for real-time updates and full library integrity.

- Repository: [Immich/immich](https://github.com/immich-app/immich)
- Tags: internals
- Published: 2026-02-27

---

**The `LibraryService` orchestrates external library synchronization by acquiring database locks, setting up file system watchers, and managing batch scanning jobs through a multi-phase workflow that handles both real-time changes and periodic full-library scans.**

Immich uses external libraries to monitor media folders outside its internal storage structure. The `LibraryService`, located at [`server/src/services/library.service.ts`](https://github.com/immich-app/immich/blob/main/server/src/services/library.service.ts), coordinates these operations through a sophisticated architecture that ensures data consistency while efficiently handling file system events.

## Phase 1: Initialization and Configuration

During application startup, the service initializes its coordination mechanisms to ensure only one instance manages library operations across a potential cluster of micro-services.

### Database Locking and Cron Setup

The `onConfigInit` method acquires a `DatabaseLock.Library` lock to guarantee singleton execution. Upon obtaining the lock, it registers a cron job using `CronJob.LibraryScan` that periodically triggers full-library scans. If real-time watching is enabled in the configuration, the method immediately invokes `watchAll()` to begin monitoring all configured libraries.

```typescript
// Simplified initialization flow from onConfigInit
async onConfigInit() {
  // Acquire distributed lock
  await this.databaseRepository.withLock(DatabaseLock.Library, async () => {
    // Register periodic scanning
    this.cronRepository.create({
      name: CronJob.LibraryScan,
      expression: '0 0 * * *', // Daily at midnight
      handler: () => this.handleQueueScanAll()
    });
    
    // Start real-time watchers if enabled
    if (this.config.library.watch) {
      await this.watchAll();
    }
  });
}

```

## Phase 2: Real-time File System Watching

For immediate synchronization, the service leverages file system watchers that react to changes as they occur rather than waiting for scheduled scans.

### Chokidar Integration and Event Handling

The `watch` method creates a **chokidar** watcher via `storageRepository.watch` for each library's `importPaths`. This watcher monitors for `add`, `change`, and `unlink` events while respecting the library's `exclusionPatterns`.

```typescript
// Watcher setup in the watch method
async watch(libraryId: string) {
  const library = await this.libraryRepository.get(libraryId);
  
  // Create matcher for supported mime types
  const matcher = new MimeTypeMatcher(
    Object.values(MimeType).filter(m => m.startsWith('image/') || m.startsWith('video/'))
  );
  
  // Initialize chokidar watcher
  const watcher = this.storageRepository.watch(
    library.importPaths,
    {
      ignored: library.exclusionPatterns,
      persistent: true,
      ignoreInitial: true
    }
  );
  
  // Handle file additions and modifications
  watcher.on('add', async (path) => {
    if (matcher.match(path)) {
      await this.jobRepository.queue({
        name: JobName.LibrarySyncFiles,
        data: { libraryId, paths: [path] }
      });
    }
  });
  
  // Handle deletions
  watcher.on('unlink', async (path) => {
    await this.jobRepository.queue({
      name: JobName.LibraryRemoveAsset,
      data: { libraryId, path }
    });
  });
}

```

### Filtering and Job Queuing

The watcher uses a **mime-type matcher** to process only supported image and video extensions. When events fire, the service queues specific jobs: `JobName.LibrarySyncFiles` for additions and changes, and `JobName.LibraryRemoveAsset` for deletions.

## Phase 3: Batch Scanning and Asset Synchronization

For comprehensive consistency and initial imports, the service performs batch operations that crawl the entire library structure.

### Full Library Scans

The `queueScan` and `queueScanAll` methods initiate comprehensive scans. These queue two critical jobs per library: `LibrarySyncFilesQueueAll` to crawl the disk for new files, and `LibrarySyncAssetsQueueAll` to verify existing assets against current disk state.

```typescript
// Triggering a full scan
async queueScan(libraryId: string) {
  await this.jobRepository.queue({
    name: JobName.LibrarySyncFilesQueueAll,
    data: { libraryId }
  });
  await this.jobRepository.queue({
    name: JobName.LibrarySyncAssetsQueueAll,
    data: { libraryId }
  });
}

```

### Disk Crawling and File Discovery

The `handleQueueSyncFiles` method walks each valid import path using `storageRepository.walk`. It filters out already-imported files by checking against the database, then queues `LibrarySyncFiles` jobs containing batches of new file paths for efficient processing.

### Asset Import and Database Creation

The `handleSyncFiles` method processes queued path batches. It creates `AssetTable` rows through `processEntity`, which builds database records including checksums, timestamps, and mime types. After bulk insertion, it calls `queuePostSyncJobs` to schedule sidecar detection and metadata extraction via `JobName.SidecarCheck`.

### Asset Validation and Offline Detection

The `handleSyncAssets` method reconciles existing database assets with the current file system state. It marks missing files as offline and newly-found files as online, scheduling metadata updates through `queuePostSyncJobs` where necessary.

## Key Helper Methods and Utilities

Several utility methods support the synchronization workflow:

- **`validateImportPath`**: Ensures import paths are absolute, exist, are directories, and are readable before processing.
- **`processEntity`**: Constructs `AssetTable` objects with computed checksums, mime types, and timestamps for database insertion.
- **`queuePostSyncJobs`**: Schedules follow-up processing including sidecar file detection (`JobName.SidecarCheck`) and metadata extraction after assets are imported.
- **`checkExistingAsset`**: Determines whether existing assets should be marked offline, updated, or ignored based on current file system statistics.

## Triggering Scans via the API

Consumers such as API controllers can trigger synchronization through public methods:

```typescript
// Manually trigger a full scan of a single library
await libraryService.queueScan(libraryId);
// Enqueues LibrarySyncFilesQueueAll & LibrarySyncAssetsQueueAll

// Manually trigger a scan of all external libraries
await libraryService.queueScanAll();
// Queues LibraryScanQueueAll which cascades to every library

```

Real-time watching requires no explicit API calls once enabled; the `watch` handlers automatically enqueue `LibrarySyncFiles` and `LibraryRemoveAsset` jobs when file system events occur.

## Summary

- **Singleton Coordination**: `LibraryService` uses `DatabaseLock.Library` to ensure only one instance manages synchronization across clustered deployments.
- **Dual Sync Strategy**: Combines real-time file watching via chokidar with periodic batch scans via cron jobs for comprehensive coverage.
- **Job-Based Architecture**: All operations queue discrete jobs (`LibrarySyncFiles`, `LibrarySyncAssetsQueueAll`, etc.) enabling asynchronous processing and failure recovery.
- **Path Validation**: Strict validation via `validateImportPath` ensures only accessible, absolute directory paths are processed.
- **Asset Lifecycle Management**: Handles creation, updates, and offline marking through `handleSyncFiles` and `handleSyncAssets`, with automatic metadata extraction scheduling.

## Frequently Asked Questions

### How does Immich prevent duplicate scans when running multiple server instances?

The `LibraryService` acquires a distributed `DatabaseLock.Library` during the `onConfigInit` lifecycle event. Only the instance that successfully obtains this lock registers the cron job and starts file system watchers. This ensures that even in a horizontally scaled deployment, only one instance manages the synchronization workload.

### What is the difference between real-time watching and batch scanning in Immich?

**Real-time watching** uses chokidar to monitor import paths continuously, immediately queuing sync jobs when files are added, changed, or deleted. **Batch scanning** runs periodically via cron or manual triggers, crawling the entire library structure to discover new files and validate existing assets against the current disk state. Watching provides low-latency updates for active changes, while scanning ensures comprehensive consistency and handles missed events.

### How does the service handle file deletions in external libraries?

When the chokidar watcher detects an `unlink` event (file deletion), the `watch` method queues a `JobName.LibraryRemoveAsset` job containing the library ID and file path. This job processes the deletion by locating the corresponding asset in the database and handling the removal according to Immich's asset lifecycle policies, ensuring the database remains consistent with the file system state.

### What happens when a new file is discovered during a full library scan?

The `handleQueueSyncFiles` method walks the import paths using `storageRepository.walk`, filtering out files already present in the database. For new files, it batches paths and queues `LibrarySyncFiles` jobs. The `handleSyncFiles` processor then creates `AssetTable` records via `processEntity`, computing checksums and mime types, bulk-inserts them into the database, and finally calls `queuePostSyncJobs` to schedule metadata extraction and sidecar file detection.