Skip to content

Support async asset error recovery and preserve render failures - #12290

Open
alec-watts wants to merge 10 commits into
remotion-dev:mainfrom
alec-watts:codex/consistent-asset-error-recovery
Open

alec-watts wants to merge 10 commits into
remotion-dev:mainfrom
alec-watts:codex/consistent-asset-error-recovery

Conversation

@alec-watts

@alec-watts alec-watts commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Asset recovery callbacks can now await URL refreshes or application replacements consistently across Img, CanvasImage, AnimatedImage, Gif, Video, and Audio. Returning handled keeps rendering paused until the replacement or removal commits; throwing/rejecting cancels with the application's Error, including cause and custom fields in browser-local rendering. Results from obsolete sources are ignored.

The review fixes preserve existing callbacks returning incidental strings, numbers, Errors, or Responses: only recognized action strings select a policy; everything else uses the component's legacy default. The exported MediaErrorAction remains fallback | fail, so existing exhaustive switches compile. The new handled decision is accepted by MediaOnError without widening that old alias.

Explicit Video/Audio fail retains browser-rendering context and the asset error as cause. Handler replacement errors retain identity. Preview fallback logging is restored. GIF original failures retain source/CORS context, while handler replacement errors remain unchanged.

The guide documents the intended behavior changes: returned telemetry promises are awaited; Video/Audio handlers run during browser rendering; throwing image handlers cancel the render. Existing native Img void callbacks still wait for load or replacement.

Validation

  • Previous head: Upstream CI passed on 903e8697df; this verification predates the main merge below.
  • Merged current upstream 6f214a19aa.
  • Core: 724 tests passed; builds/typechecks passed for all 14 affected package/dependency tasks.
  • Recovery browser suite: 64 passed, 2 skipped across Chrome, Firefox, and WebKit. Only the two added GIF preview cases are skipped in WebKit: its worker does not deliver the parse error in this harness. Those cases pass in Chrome and Firefox.
  • Media preview/error-policy suite: 7 passed.
  • Verified old incidental-return callbacks and exhaustive MediaErrorAction switches compile. Removing the compatible callback arm causes the type fixtures to fail.
  • Removing the runtime fixes makes all four incidental-return render cases fail, loses GIF source context, and fails the explicit browser-error context regression; restoring the fixes passes.
  • Scoped ESLint and formatting checks passed (existing TODO warnings remain). Diff cleanup found no formatting bloat or unused mechanisms.

Why each file changes

File Reason
bun.lock Record the existing test dependency on @remotion/media.
packages/core/src/Img.tsx Await application recovery while preserving native Img waiting and callback behavior.
packages/core/src/animated-image/AnimatedImage.tsx Apply async recovery with source cancellation and commit-owned render holds.
packages/core/src/animated-image/props.ts Expose the compatible asynchronous callback contract.
packages/core/src/cancel-render.ts Preserve the scoped original Error for browser-local failures.
packages/core/src/canvas-image/CanvasImage.tsx Apply async recovery and release default-handler holds after a React commit.
packages/core/src/canvas-image/props.ts Expose the compatible asynchronous callback contract.
packages/core/src/handle-image-error.ts Share image failure policy and ignore obsolete source results.
packages/core/src/index.ts Export the public callback type.
packages/core/src/internals.ts Expose shared recovery helpers to media and GIF packages.
packages/core/src/media-error-handler.ts Recognize only known actions, preserve legacy returns, and coordinate failure/commit holds.
packages/docs/docs/animatedimage.mdx Describe the async callback and link the shared recovery guide.
packages/docs/docs/asset-error-handling.mdx Document recovery, compatibility defaults, awaited telemetry, and preview/browser side effects.
packages/docs/docs/canvasimage.mdx Describe the async callback and link the shared recovery guide.
packages/docs/docs/gif/gif.mdx Describe the async callback and link the shared recovery guide.
packages/docs/docs/img.mdx Describe the async callback and link the shared recovery guide.
packages/docs/docs/media/audio.mdx Describe the async callback and link the shared recovery guide.
packages/docs/docs/media/video.mdx Describe the async callback and link the shared recovery guide.
packages/docs/sidebars.ts Make the recovery guide discoverable.
packages/docs/src/data/articles.ts Register the recovery guide.
packages/gif/src/Gif.tsx Await recovery during intrinsic duration discovery.
packages/gif/src/GifForDevelopment.tsx Preserve source/CORS context for asset failures and replacement Error identity.
packages/gif/src/GifForRendering.tsx Apply asynchronous recovery with source cancellation and render holds.
packages/gif/src/props.ts Expose the compatible asynchronous callback contract.
packages/media/src/audio/audio-for-preview.tsx Await one current-source decision and retain fallback diagnostics.
packages/media/src/audio/audio-for-rendering.tsx Invoke recovery during browser rendering and preserve failure identity.
packages/media/src/on-error.ts Preserve MediaErrorAction, known-action defaults, and contextual browser failures.
packages/media/src/test/video-onerror-stable.test.tsx Pin stable handlers, replacement Error propagation, and explicit failure context.
packages/media/src/video/video-for-preview.tsx Await one current-source decision and retain fallback diagnostics.
packages/media/src/video/video-for-rendering.tsx Apply async recovery to normal and ProRes browser-render failures.
packages/web-renderer/package.json Use real media components in recovery regression coverage.
packages/web-renderer/src/test/asset-error-recovery.test.tsx Exercise real rendering, suspended replacements, stale failures, aborts, legacy returns, and GIF context.
packages/web-renderer/src/wait-for-ready.ts Return the scoped original Error instead of losing its identity and fields.

Current main integration

Merged upstream bbe11edd85 in 5c0339ef89. 64 recovery browser tests passed, 2 documented WebKit GIF-preview skips. Affected-package builds/typechecks passed; Upstream CI passed on 5c0339ef89; GitHub reports no merge conflicts.

Main refresh — 2026-10-23

Merged upstream 2bd172f29a into this branch as f2ca672618. The merge required no manual conflict resolutions. Diff cleanup and whitespace checks passed; retained file changes are scoped to this proposal. Fresh upstream CI passed on this head, including Required CI. Earlier local validation above applies to the commits identified there.

Local verification on this refreshed head: core TypeScript declaration build passed; all 724 core tests passed.

@vercel

vercel Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
bugs Ready Ready Preview Oct 29, 2026 11:29am UTC
remotion Error Error Oct 29, 2026 11:29am UTC

Request Review

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No blocking issues — the async recovery contract reads coherently and is well covered by the new real-browser tests. Two inline notes on the shared plumbing, plus two docs-pipeline follow-ups.

Reviewed changes

Read the full diff (32 files) end to end, and compared the base origin/main implementations of on-error.ts, use-delay-render.tsx, and cancel-render.ts.

  • Shared async recovery contract — new MediaErrorHandler / resolveMediaError / handleImageError / useErrorRecoveryCommit in packages/core, wired into <Img>, <CanvasImage>, <AnimatedImage>, <Gif>, and @remotion/media <Video>/<Audio>.
  • Original-error preservation — cancel-render.ts now stores remotion_cancelledErrorObject, and wait-for-ready.ts rejects with the original Error (cause/custom fields intact), prioritising cancellation over readiness.
  • Scoped cancellation fix — audio/video rendering now cancel through the active useDelayRender() scope instead of importing the global window-bound cancelRender.
  • Preview initialization failures — routed through the same async path; a fail now surfaces via terminalError rather than a rejected promise, and onError is invoked once.
  • Docs + tests — new asset-error-handling page plus per-component notes, real-browser recovery tests, and a media preview rejection test.

ℹ️ New docs page is missing from the generated articles catalog

packages/docs/src/data/articles.ts is a committed, generated index of every docs page (written by render-cards.ts, which walks docs/**/*.mdx) and drives the per-page social cards. The new asset-error-handling.mdx page has no entry, so the social-card catalog desyncs until it is regenerated.

Technical details
# Missing generated entry for the new docs page

## Affected sites
- `packages/docs/docs/asset-error-handling.mdx` — new page, not present in the catalog
- `packages/docs/src/data/articles.ts` — no `compId: 'articles-docs-asset-error-handling'` / `slug: 'asset-error-handling'` entry

## Required outcome
The new page appears in `articles.ts` with the same field set/ordering the generator produces (`id`, `title`, `relativePath`, `compId`, `crumb`, `noAi`, `slug`).

## Suggested approach
Run `bun render-cards` from `packages/docs` (it rewrites `articles.ts` and renders only missing cards), or add the entry by hand matching the generator's field ordering.

ℹ️ New async behavior carries no <AvailableFrom> marker

This PR extends public callbacks (onImageError, onError on <CanvasImage>/<AnimatedImage>/<Gif>/<Video>/<Audio>) and exports a new public type MediaErrorHandler, but the docs don't say which version supports it. The repo convention is to annotate new public API with <AvailableFrom v="..." /> (the update-version skill exists for exactly this).

Technical details
# Missing version annotation for new public API

## Affected sites
- `packages/docs/docs/asset-error-handling.mdx` — new page, no version
- `packages/docs/docs/img.mdx:113`, `canvasimage.mdx:110`, `animatedimage.mdx:31`, `gif/gif.mdx:38`, `media/video.mdx:203`, `media/audio.mdx:155` — added "Supports asynchronous asset recovery" lines, no version
- `packages/core/src/index.ts:263` — new `MediaErrorHandler` export

## Required outcome
Mark the new behavior / type with the next patch version (main + 0.0.1) so users can tell which release added async recovery.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash | 𝕏

Comment thread packages/media/src/on-error.ts Resolved
…t-error-recovery

# Conflicts:
#	packages/core/src/animated-image/AnimatedImage.tsx
#	packages/gif/src/props.ts

@JonnyBurger JonnyBurger left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you think about these points if they are tenable or could lead to a serious breakage? There are some behavior changes to review:

Breaking changes

1. TypeScript: some existing callbacks stop compiling. The old type (error: Error) => void let a callback return anything. The new 'handled' | 'fail' | void | Promise<…> union doesn't get that exception, so these used to compile and now fail (verified with tsc):

<CanvasImage onError={(e) => Sentry.captureException(e)} />  // returns a string
<Img onImageError={(e) => errors.push(e)} />                 // returns a number
<AnimatedImage onError={(e) => (ref.current = e)} />          // returns Error

This affects Img.onImageError, CanvasImage, AnimatedImage and Gif.

2. MediaErrorAction gained 'handled'. It's exported from @remotion/media, so exhaustive switch statements with a never check stop compiling. Minor.

Bug: callbacks returning any other value now hang the render

This is the runtime side of point 1, for plain JS users, casts, or anyone who silences the type error. In resolveMediaError, any value other than undefined counts as an explicit decision:

action: action ?? defaultAction,
isDefaultAction: action === undefined,

handleImageError only acts on 'fail', or on 'handled' when it was the default. Any other value, like Sentry's event ID string or a fetch() Response, does nothing, so the render pause is never released.

  • Before: onError={(e) => Sentry.captureException(e)} on CanvasImage or AnimatedImage logged the error and the render continued.
  • After: the render waits until the delayRender timeout. The same goes for Gif duration discovery, and for onError={(e) => fetch('/log', …)}.

Suggested fix: treat any value that isn't a known action as the default (isDefaultAction: true). The public type could then also accept any return value, which avoids the compile break.

Smaller problems

  • Logging callbacks now pause the render. Async CanvasImage/AnimatedImage/Gif callbacks that send telemetry used to run in the background. Now the render waits for them, and a slow endpoint can cause a delayRender timeout. This is intended, but it's a behavior change worth documenting.
  • Worse error message in browser rendering. In callOnErrorAndResolve, an explicit 'fail' returns before the browser-rendering branch. A Video/Audio handler that returns 'fail' gets the bare Unknown container format X. instead of Cannot render video "X": Unknown container format. See supported formats: ….
  • Video/Audio onError now runs during browser rendering, where it was skipped for these failures before. Handlers with side effects (toasts, analytics) will start firing there.
  • Fewer preview log lines. video-for-preview.tsx no longer logs Cannot decode X, falling back to <OffthreadVideo>, No video or audio tracks found…, falling back, or [VideoForPreview] Failed to initialize MediaPlayer, so fallbacks now happen silently.
  • A throwing onImageError now cancels the render. Before, it was an uncaught error from an event handler. Probably better, but still a behavior change.
  • <Gif onError> in preview. If the handler fails, the raw error now goes to the error boundary, skipping the CORS-specific error message.
@alec-watts

Copy link
Copy Markdown
Contributor Author

Addressed these in 903e869, after merging current main:

  • Restored TypeScript compatibility for logging callbacks returning strings, numbers, Errors, or promises of Responses. Only recognized actions override legacy defaults, so incidental returns no longer leave rendering pending.
  • Kept exported MediaErrorAction as fallback | fail; MediaOnError accepts the new handled result separately. Existing exhaustive switches still compile.
  • Explicit Video/Audio fail now keeps the contextual browser-render error with the original cause. A thrown/rejected replacement Error still keeps its identity and fields.
  • Restored Video/Audio preview fallback diagnostics. GIF asset failures retain source/CORS context; replacement handler errors remain untouched.
  • Documented awaited telemetry promises, browser-render handler side effects, and throwing image-handler cancellation in the recovery guide.

Validation: 724 core tests; 64 browser recovery cases (two additional GIF preview cases skipped only in WebKit because the worker does not deliver the parse error in this harness); 7 media regression cases; all affected builds/typechecks. Removing the fixes reproduces the type break, all four incidental-return rendering failures, lost GIF source context, and lost explicit-failure context. The PR description has the full validation and per-file rationale.

# Conflicts:
#	packages/core/src/Img.tsx
#	packages/media/src/audio/audio-for-preview.tsx
#	packages/media/src/audio/audio-for-rendering.tsx
#	packages/media/src/video/video-for-preview.tsx
#	packages/media/src/video/video-for-rendering.tsx
# Conflicts:
#	packages/docs/sidebars.ts

This branch had an error being deployed

1 failed and 1 active deployments
Preview – remotion — 7e1f260a Deployed Oct 29, 2026 by vercel[bot]
Preview – bugs — 7e1f260a Deployed Oct 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants