Assets are permanently left without metadata, thumbnails, OCR and smart search when queued jobs are lost, because Redis has no volume and nothing re-queues them #32726

I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.

• Yes

The bug

Immich queues each asset's processing once, at import: metadata extraction, then thumbnails, then smart search, face detection and OCR. The queue lives in Redis, and the default docker-compose.yml gives Redis no volume. Any docker compose down / up while an import is still being processed therefore discards the pending jobs.

That used to be harmless. In #1233 the reason given for not persisting Redis was that the queue is "regularly refilled by a cronjob". In v3.3 that is no longer true. Nothing re-queues the lost work, and the affected assets stay unprocessed until an admin happens to notice and runs the "Missing" jobs by hand, in the right order:

• Nightly tasks re-queue only thumbnails (AssetGenerateThumbnailsQueueAll) and face clustering. Metadata extraction, smart search, face detection and OCR are never re-queued.
• The nightly missing-thumbnails pass skips exactly these assets. streamForThumbnailJob inner-joins asset_job_status ("If there aren't any entries, metadata extraction hasn't run yet"). An asset whose metadata job was lost has no row there, so it is skipped every night.
• A thumbnail regenerated by that pass does not chain onward anyway. onDone for AssetGenerateThumbnails queues SmartSearch, AssetDetectFaces and Ocr only when source === 'upload' or notify is set.
• The "Missing" OCR job has the same blind spot. streamForOcrJob also inner-joins asset_job_status, and smart search and face detection select from assetsWithPreviews(). So the only way back is a manual sequence: Extract Metadata → Missing, wait for it to finish, Generate Thumbnails → Missing, wait, then Smart Search, Face Detection and OCR → Missing. Running them all at once silently skips the stranded assets.
• The nightly external library scan does not catch them either. The files are unchanged, so they are not re-queued.

The asset shows up in the timeline as a grey tile with no preview, and in every search it is simply absent. Nothing reports an error, and the Job Queues page shows empty queues.

Real-world impact. An external-library import of about 18,700 images overloaded a small 2-core host. The stack was stopped and started a few hours into processing. Three months later:

That is about 15% of the library, invisible to smart search, OCR search and people. It came to light only because a search for receipts turned up nothing from those months.

Suggested fix. Either change, or both:

1. Persist the queue by default. Give the redis service a named volume and AOF:

   redis:
     command: valkey-server --appendonly yes
     volumes:
       - redis-data:/data

   added redis data volume #16505 proposed this and was closed with "this data is ephemeral". As shown above, it isn't ephemeral in practice, because nothing regenerates it.

2. Make recovery automatic. Add nightly "missing" passes for metadata extraction, smart search, face detection and OCR, as already exists for thumbnails. Make the thumbnail pass include assets with no asset_job_status row (or queue metadata extraction for them first). This also recovers from crashes, OOM kills and transient ML failures, not only restarts, and it makes (1) a nice-to-have rather than a requirement.

The OS that Immich Server is running on

Ubuntu 26.04.1 LTS (Docker, docker-compose-v2)

Version of Immich Server

v3.3.0 (code paths cited above checked against main as of 2026-10-25)

Version of Immich Mobile App

n/a

Platform with the issue

• Server
• Web

Your docker-compose.yml content

Stock release docker-compose.yml. The only changes are two extra read-only
bind mounts on immich-server for the external library.

Your .env content

Stock example.env (UPLOAD_LOCATION, DB_DATA_LOCATION, IMMICH_VERSION=v3.3.0, DB_* credentials).

Reproduction steps

1. Start from the stock compose file. Add an external library containing a few thousand images, or upload a large batch, and start a scan.

2. While Extract Metadata or Generate Thumbnails still shows waiting jobs, run docker compose down, then docker compose up -d.

3. Queues are now empty. Let the nightly tasks and the nightly library scan run for a few nights.

4. Some of the images still show grey tiles, and none of them are found by smart search or OCR search. Confirm in the database:

   -- assets metadata extraction never reached
   select count(*) from asset a
   left join asset_job_status j on j."assetId" = a.id
   where a."deletedAt" is null and j."assetId" is null;
   
   -- assets with no CLIP embedding
   select count(*) from asset a
   left join smart_search s on s."assetId" = a.id
   where a."deletedAt" is null and a.type = 'IMAGE' and s."assetId" is null;

   Both counts stay above zero indefinitely.

Relevant log output

No errors are logged for the stranded assets. They are never attempted again, which is the problem.

Additional information

Relevant code on main:

• server/src/services/queue.service.ts: the nightly job list (thumbnails and face clustering are the only "missing" refills)
• server/src/repositories/asset-job.repository.ts: streamForThumbnailJob and streamForOcrJob (innerJoin('asset_job_status', …)), streamForMetadataExtraction (leftJoin, so the manual Extract Metadata → Missing does find them)
• server/src/services/job.service.ts: onDone, case JobName.AssetGenerateThumbnails (chains only for source === 'upload' / notify)

Previous discussion: #1233 (2022, "regularly refilled by a cronjob"), #16505 (2025, closed as ephemeral), discussion #25580 (a maintainer notes that Redis persistence "lets them survive restarts or crashes").