How the Immich Job Queue System Manages Background Processing: Architecture and Components

The Immich job queue system utilizes a modular, BullMQ-backed architecture split into three layers—QueueService for orchestration, JobRepository for worker management, and a TypeScript SDK for administrative UI control—to execute background tasks like thumbnail generation and facial recognition asynchronously.

The immich-app/immich repository processes resource-intensive operations outside the HTTP request cycle through a sophisticated job queue system. By leveraging BullMQ workers and a decorator-based handler registration system, Immich ensures reliable parallel processing while maintaining strict separation between API serving and background computation.

Three-Layer Architecture Overview

The system organizes background processing into distinct responsibilities:

• Orchestration Layer: The QueueService in server/src/services/queue.service.ts handles application bootstrap, worker initialization, concurrency configuration, and nightly job scheduling.
• Queue Engine Layer: The JobRepository in server/src/repositories/job.repository.ts manages BullMQ Queue and Worker instances, registers job handlers, and exposes CRUD operations for queue management.
• Interface Layer: TypeScript enums in server/src/enum.ts define QueueName and JobName constants, while the web SDK in web/src/lib/services/queue.service.ts provides the Svelte admin UI with type-safe API access.

Bootstrapping Workers on Application Startup

When the NestJS application emits the AppBootstrap event, the QueueService initializes the job infrastructure with priority BootstrapEventPriority.JobService. During this phase, the system discovers all job handlers and conditionally starts worker processes.

@OnEvent({ name: 'AppBootstrap', priority: BootstrapEventPriority.JobService })
onBootstrap() {
  this.jobRepository.setup(this.services);
  if (this.worker === ImmichWorker.Microservices) {
    this.jobRepository.startWorkers();
  }
}

Source: QueueService.onBootstrap in queue.service.ts.

The setup method scans injected services for methods decorated with @OnJob({ name, queue }), building a handler map that associates each JobName with its corresponding QueueName and bound function. If any job lacks a registered handler, the application throws an ImmichStartupError during initialization.

Starting BullMQ Workers and Processing Jobs

The JobRepository.startWorkers method instantiates a BullMQ Worker for every defined QueueName. Each worker initializes with process-level concurrency of 1, which is subsequently adjusted per-queue via configuration.

this.workers[queueName] = new Worker(
  queueName,
  (job) => this.eventRepository.emit('JobRun', queueName, job as JobItem),
  { ...bull.config, concurrency: 1 },
);

Source: JobRepository.startWorkers in job.repository.ts.

When a worker receives a job, it emits a JobRun event that the repository catches and routes to the appropriate handler. The handler executes the business logic (e.g., generating thumbnails), updates progress, and returns a JobStatus to mark completion.

Dynamic Concurrency Configuration

Administrators control parallel processing limits through the system configuration without restarting the server. The QueueService.updateConcurrency method reads config.job[queueName].concurrency and propagates the value to the underlying worker:

this.jobRepository.setConcurrency(queueName, concurrency);

The JobRepository.setConcurrency implementation directly mutates the BullMQ worker property:

worker.concurrency = concurrency;

Source: QueueService.updateConcurrency in queue.service.ts and JobRepository.setConcurrency in job.repository.ts.

Scheduling Periodic Nightly Jobs

The system automates maintenance tasks through the QueueService.handleNightlyJobs method. Based on SystemConfig.nightlyTasks flags (database cleanup, memory generation), it constructs a list of JobItem objects and delegates bulk queuing to jobRepository.queueAll.

if (config.nightlyTasks.databaseCleanup) {
  jobs.push({ name: JobName.AssetDeleteCheck });
}
await this.jobRepository.queueAll(jobs);

The nightly schedule initializes during onConfigInit by converting config.nightlyTasks.startTime into a cron expression via asNightlyTasksCron. The queueAll method groups jobs by queue and efficiently submits them using BullMQ's addBulk when possible.

Source: QueueService.handleNightlyJobs and asNightlyTasksCron in queue.service.ts.

Administrative Controls and SDK Integration

The server exposes REST endpoints through queue.controller.ts that map to QueueService methods, consuming DTOs like QueueCommandDto and QueueUpdateDto. The web client accesses these through the generated @immich/sdk:

• runQueueCommandLegacy({ name, queueCommandDto }) executes start, pause, resume, or empty commands.
• emptyQueue({ name, queueDeleteDto }) clears jobs, optionally filtering for failed states only.
• updateQueue({ name, queueUpdateDto }) pauses or resumes specific queues.

The Svelte-based admin UI in web/src/lib/managers/queue-manager.svelte.ts displays real-time queue statistics and provides buttons for Pause, Resume, Clear, and Create Job actions. After each operation, the UI emits QueueUpdate events to refresh status indicators automatically.

Example Execution Flow: Thumbnail Generation

A typical background job flows through the system as follows:

1. User initiates job: The admin clicks "Create Job → Thumbnail Generation" in the Svelte UI.
2. SDK call: JobCreateModal invokes runQueueCommandLegacy with QueueName.ThumbnailGeneration and QueueCommand.Start.
3. HTTP request: The SDK sends POST /queues/:name/command to the NestJS controller.
4. Command resolution: QueueService.runCommandLegacy resolves the command to the concrete JobName.AssetGenerateThumbnailsQueueAll.
5. Job queuing: QueueService.start calls jobRepository.queue({ name, data }), which routes to queueAll and executes BullMQQueue.add.
6. Worker pickup: The dedicated worker for QueueName.ThumbnailGeneration receives the job, emits JobRun, and JobRepository.run invokes the bound handler.
7. Processing: The handler processes assets, reports progress, and completes with JobStatus.Success.

Key files: queue.service.ts (command handling), job.repository.ts (queueing and execution), enum.ts (job definitions), and web/src/lib/services/queue.service.ts (SDK).

Programmatic Job Queuing

Server-side scripts or custom services can enqueue jobs directly through the SDK:

import { queue, QueueName, JobName } from '@immich/sdk';

// Queue a memory generation job
await queue({
  name: JobName.MemoryGenerate,
  data: {}, // job-specific payload
});

This call chains through JobRepository.queueAll to BullMQ's Queue.addBulk, ensuring the job enters the appropriate worker stream immediately.

Summary

• BullMQ Foundation: The Immich job queue system builds on BullMQ, with one worker per QueueName initialized at bootstrap.
• Decorator-Based Registration: The @OnJob decorator auto-discovers handlers, binding JobName to specific service methods in JobRepository.setup.
• Dynamic Concurrency: Per-queue parallelism adjusts at runtime via QueueService.updateConcurrency, modifying the underlying worker.concurrency property without restarts.
• Nightly Automation: QueueService.handleNightlyJobs schedules bulk maintenance tasks based on system configuration flags.
• Full-Stack Control: TypeScript enums provide type safety, the SDK exposes CRUD operations, and the Svelte admin UI enables real-time queue management.

Frequently Asked Questions

How does Immich ensure every job has a valid handler?

During application bootstrap, JobRepository.setup iterates through all @OnJob decorators to build a handler map. If any entry in the JobName enum lacks a corresponding handler implementation, the repository throws an ImmichStartupError, preventing the server from starting with undefined job routes.

Can I change job concurrency without restarting the Immich server?

Yes. The QueueService.updateConcurrency method reads the current system configuration and calls JobRepository.setConcurrency, which directly mutates the worker.concurrency property on the running BullMQ worker instance. This change takes effect immediately for subsequent job pickups.

What is the difference between QueueName and JobName in Immich?

QueueName (defined in server/src/enum.ts) represents the BullMQ queue categories (e.g., ThumbnailGeneration, FaceRecognition), while JobName represents specific task types (e.g., AssetGenerateThumbnailsQueueAll, PersonCleanup). The JobRepository maps each JobName to its parent QueueName and bound handler function during initialization.

How are failed jobs handled in the Immich job queue?

The JobRepository exposes methods to query and remove failed jobs through the admin API. The emptyQueue SDK function accepts a queueDeleteDto parameter that can filter for failed states only, allowing administrators to clear stuck jobs via the Svelte admin UI or direct API calls without affecting pending or completed work.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}