How Immich Processes and Extracts Metadata (EXIF and OCR) Using metadata.service.ts and ocr.service.ts

Immich extracts EXIF metadata and OCR text through two dedicated NestJS services—MetadataService and OcrService—which process background jobs from the QueueName.MetadataExtraction and QueueName.Ocr queues, persisting structured data to PostgreSQL for search and organization.

Immich processes and extracts metadata from uploaded assets using a dual-pipeline architecture that handles EXIF camera data and searchable OCR text. The system relies on MetadataService in server/src/services/metadata.service.ts for EXIF extraction, motion-photo handling, and face detection, while OcrService in server/src/services/ocr.service.ts manages optical character recognition via machine learning models. Both services operate as background job workers that transform raw files into structured, queryable database records according to the immich-app/immich source code.

EXIF Metadata Extraction Pipeline

The EXIF extraction pipeline runs through MetadataService.handleMetadataExtraction, which processes individual assets queued in QueueName.MetadataExtraction.

Job Orchestration and Batching

The pipeline initiates via handleQueueMetadataExtraction, which streams assets requiring processing and creates batch jobs using JOBS_ASSET_PAGINATION_SIZE (defined in server/src/constants.ts). This queue-all job (JobName.AssetExtractMetadataQueueAll) feeds individual JobName.AssetExtractMetadata tasks to workers.

// MetadataService queues metadata extraction jobs in batches
await jobRepository.queue({
  name: JobName.AssetExtractMetadata,
  data: { id: assetId },
});

Core Extraction Flow in handleMetadataExtraction

When a worker picks up a metadata job, handleMetadataExtraction executes a twelve-step validation and persistence flow:

1. Configuration check – Verifies metadata extraction is enabled via isMetadataExtracted.

2. Asset loading – Fetches the asset via assetJobRepository.getForMetadataExtraction.

3. Tag reading – Reads EXIF from the original file, side-car XMP via metadataRepository.readTags, and video streams via mediaRepository.probe.

4. Tag merging – Side-car dates overwrite media dates; video duration is stripped for non-animated images to prevent data pollution.

5. Date normalization – firstDateTime scans ordered EXIF date tags (DateTimeOriginal, CreateDate, ModifyDate) and returns the first valid ExifDateTime. The getDates helper normalizes time-zones (fallback to UTC) and falls back to file-system timestamps when EXIF dates are missing.

6. Dimension extraction – getImageDimensions prefers ImageSize for RAW formats, otherwise uses ImageWidth/ImageHeight tags.

7. Data validation – Helpers like validate, validateRange, getLensModel, and getBitsPerSample ensure numeric values fit PostgreSQL integer limits before populating AssetExifTable.

8. Database persistence – Upserts the EXIF record via assetRepository.upsertExif and updates the asset record with duration, local time, and hidden dimensions.

9. Side-car writeback – If tags changed, metadataRepository.writeTags updates the XMP side-car file.

10. Event emission – Fires AssetMetadataExtracted and marks the extraction timestamp via assetRepository.upsertJobStatus.

Motion Photos, Live Photos, and Face Tags

The service executes three critical side-effects in a Tasks batch after primary EXIF extraction:

• Motion-photo extraction – applyMotionPhotos extracts embedded video bytes from motion-photo containers, creates hidden video assets, links them via motionPhotoVideoId, and queues encode jobs.

• Live-photo linking – linkLivePhotos pairs matching photo/video assets using ContentIdentifier tags to establish livePhotoVideoId relationships.

• Face tag import – When metadata.faceImport.enabled is true, hasTaggedFaces discovers RegionInfo tags in XMP data, orientRegionInfo adjusts coordinates based on EXIF orientation, and creates/updates Person and AssetFace rows.

OCR Text Extraction Pipeline

The OCR pipeline mirrors the metadata architecture but targets searchable text extraction using machine learning models.

Queue Management and Job Flow

OcrService.handleQueueOcr streams assets with preview images via assetJobRepository.streamForOcrJob and queues JobName.Ocr jobs. When force is true, existing OCR data is purged first via ocrRepository.deleteAll.

// OcrService queues OCR jobs for assets with preview files
await jobRepository.queue({
  name: JobName.Ocr,
  data: { id: assetId },
});

OCR Processing and Search Tokenization

When executing handleOcr, the service performs:

1. Enablement check – Validates OCR via isOcrEnabled in the machine-learning configuration.

2. Asset validation – Loads the asset via assetJobRepository.getForOcr, ensuring previewFile exists (the thumbnail used for OCR).

3. ML inference – Calls machineLearningRepository.ocr to send the preview image to the configured OCR provider (e.g., Tesseract).

4. Result parsing – parseOcrResults extracts bounding box arrays (box property) and confidence scores into ocrDataList entries.

5. Search preparation – tokenizeForSearch (from server/src/utils/database.ts) splits detected text on whitespace and punctuation, lower-casing tokens for PostgreSQL full-text search compatibility.

6. Persistence – ocrRepository.upsert stores per-box rows with bounding coordinates and the concatenated searchText token string.

The searchText column enables full-text search via PostgreSQL tsvector indexing, allowing users to locate assets by text content captured in images.

Database Integration and Search Architecture

Both services write to distinct PostgreSQL tables optimized for query performance:

• EXIF data persists to asset_exif with columns for GPS coordinates, camera make/model, lens information, and orientation flags.

• OCR data writes to the ocr table with geometric bounding boxes and the tokenized searchText field used by search.service.ts for tsvector queries.

The repositories abstract database access: metadataRepository wraps exiftool-vendored for EXIF I/O, while ocrRepository handles geometric data and search token persistence.

Practical Code Examples

Manually Triggering Metadata Extraction

import { JobName } from 'src/interfaces/job.interface';

// Queue a single asset for EXIF extraction
await metadataService.handleQueueMetadataExtraction();
// Or directly queue a job:
await jobRepository.queue({
  name: JobName.AssetExtractMetadata,
  data: { id: 'c1a2b3d4-...' },
});

Direct OCR Invocation

// Process OCR immediately for a specific asset
await ocrService.handleOcr({ id: 'c1a2b3d4-...' });

Querying Extracted Metadata

// Read EXIF data using Kysely query builder
const exif = await db
  .selectFrom('asset_exif')
  .select(['make', 'model', 'city', 'country', 'dateTimeOriginal'])
  .where('assetId', '=', assetId)
  .executeTakeFirst();

console.log(`Shot with ${exif?.make} ${exif?.model} in ${exif?.city}`);

Searching by OCR Content

// Search service queries the tsvector index on ocr.search_text
const results = await searchService.search({
  query: 'invoice 2025',
  ocr: true, // restricts to OCR text matches
});

Summary

• MetadataService (server/src/services/metadata.service.ts) handles EXIF extraction, side-car XMP management, motion-photo video extraction, live-photo linking, and face-tag import via handleMetadataExtraction.

• OcrService (server/src/services/ocr.service.ts) processes preview images through handleOcr, calling machineLearningRepository.ocr and tokenizing results via tokenizeForSearch for full-text search.

• Both services use NestJS job queues (QueueName.MetadataExtraction, QueueName.Ocr) with pagination constants from JOBS_ASSET_PAGINATION_SIZE to process assets in manageable batches.

• EXIF data validates numeric ranges before persistence to PostgreSQL, while OCR data includes geometric bounding boxes and search-token strings for tsvector indexing.

Frequently Asked Questions

How does Immich handle time-zones when EXIF data is missing GPS offset?

When GPS-based time-zone data is unavailable, MetadataService uses the getDates helper to normalize timestamps to UTC as a fallback. The system first attempts to extract OffsetTime, OffsetTimeOriginal, or OffsetTimeDigitized from EXIF; if these are missing, it falls back to the file-system modification time while preserving the original localDateTime in the asset record.

Can Immich extract text from scanned documents or screenshots?

Yes. The OcrService processes any asset with a generated previewFile, including scanned PDFs, screenshots, and photographs of documents. The machineLearningRepository.ocr method sends the preview image to the configured OCR model (Tesseract or external provider), and parseOcrResults extracts bounding boxes and confidence scores for searchable storage in the ocr table.

What happens to motion-photo video data during metadata extraction?

MetadataService.applyMotionPhotos extracts embedded video bytes from motion-photo formats (Samsung, Google Pixel), creates a hidden video asset linked via the motionPhotoVideoId foreign key, and queues an encode job for the extracted video. This occurs within the Tasks batch after primary EXIF extraction completes, ensuring the video is searchable and thumbnail-ready.

Does modifying EXIF tags in Immich update the side-car XMP files?

Yes. When MetadataService detects tag modifications during processing, it invokes metadataRepository.writeTags to persist changes back to side-car XMP files if they exist. This ensures that external tools reading the side-car files see the same metadata that Immich displays in its interface.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}