Repository navigation
Conversation
Contributor
![pullfrog[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
- Retain audio position through Player buffering (
packages/core/src/use-media-playback.ts) — For<Html5Audio>in the Player withpauseWhenBuffering, when another media element starts buffering while the audio has advanced slightly ahead of the frozen timeline, the audio is held (not seeked back) and resumed without seeking once the timeline catches up. Explicit seeks, source/timebase/rate/fps changes, the audio's own buffering, and drift beyond the existing tolerance still reset to normal synchronization. - Seek-revision plumbing — Subscribes to
SetTimelineContext.seekviauseSyncExternalStoreand recordssrc,seekRevision,playbackRate,fps,trimBefore,absoluteFrame, andplayerBufferinginbufferingRecovery; playback advances viasetFrameWithoutSeek(revision unchanged), so only explicit navigations reset recovery. - Docs (
packages/docs/docs/html5-audio.mdx) — Documents the retained-position behavior and the added pause underpauseWhenBuffering. - Tests (
packages/player/src/test/html5-audio-buffering.test.tsx) — 16 cases covering retention, catch-up resume, explicit/backwards seeks, pause, drift beyond tolerance, source/rate/fps/trim changes, audio self-buffering, successive waits, and thepauseWhenBuffering: falsepath.
I verified locally after rebuilding packages/core: 724 core tests and 79 player tests (including all 16 new cases) pass, and ESLint plus oxfmt --check are clean on the changed files.
deepseek-v4.1-flash ｜ 𝕏
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When HLS video buffers in Player, a separate HTML5 audio track can advance slightly beyond the frozen timeline. Seeking it back repeats audio that has already been heard. A subsequent wait can also seek a slightly lagging track forward, skipping unheard audio.
Add an explicit
retainAudioDuringBufferingopt-in to<Html5Audio>:Reproduction and public-domain video with synthesized separate audio: #12440.
The option defaults to
falseand requirespauseWhenBufferingin Player. Retaining audio trades repeated/skipped content for additional waiting.Validation summary
@remotion/mediaAudio: repeated source intervals in 3/9 trials at 1× (29–43 ms during startup), and 9/9 at 3× with either default or pitch-matched settings. Its scheduler is separate from this HTML5 change.Implementation, compatibility, tradeoff and full validation results
Behavior and compatibility
false: keep the existing synchronization behavior. The new option defaults tofalseindependently of the Remotion 5.0pauseWhenBufferingdefault.true: retain small audio offsets during another media element's buffering interval in Player, providedpauseWhenBufferingis alsotrue. Keep audio paused until the timeline catches up, then resume without seeking.play()can clearpausedbeforecurrentTimeprogresses, so that flag alone is insufficient. A later wait captures small leads and lags again.pauseWhenBuffering, independently of this opt-in. Studio, rendering, HTML5 video and the Mediabunny-based audio path retain their existing behavior.The prop is forwarded through the trimmed/looped audio paths and consumed before native DOM props in preview and rendering. There are no new timers, polling loops, media requests or dependencies, and the existing drift tolerance is unchanged.
Tradeoff
Retaining audio exchanges repeated/skipped content for additional waiting. In the current public-API comparison, cumulative native audio pauses outside Player waiting were 1.389–2.714 seconds with the option enabled versus 0.011–0.089 seconds with it omitted or false. These are native pause-to-playing intervals after playback starts and before audio EOF, with Player waiting removed; they are not measurements of acoustic silence or a listening-preference study. The fixture uses synthesized test audio, so these results do not establish a preference for speech or music. The independent opt-in leaves that choice to the caller.
Current public-API verification
Compared the actual public prop omitted, explicitly false and explicitly true, with three sequential trials per option in each engine (27 low-bandwidth trials total):
All 27 trials completed both video and separate audio with no recorded browser/media errors. All nine enabled trials had zero backward and forward assignments. HLS requests and the canvas video provider were observed in every trial.
With the public opt-in enabled, normal-network 1×/3× playback passed in all three engines (six trials, zero backward/forward assignments). All 33 operation checks passed (11 per engine): pause/resume, backward/forward/same-frame seek, audio-source replacement, 1×/3× changes, buffering pause and same-frame seek while buffering. Each operation was checked against its recorded target and native audio state; intentional seeks are excluded from the no-seek comparison. Both tracks completed in all three operation runs.
Core: 724 tests passed. Player: 89 tests passed, including 26 recovery/API cases. Coverage includes omitted/false/true, disabling recovery while active, independent own-audio buffering, native startup across successive waits, small lag at the next wait, return to ordinary synchronization, explicit seeks and source/rate/fps changes. Core/Player TypeScript builds, changed-file ESLint and upstream formatting passed.
Low-bandwidth 1× frequency
Kept the same 33-second 720p HLS video, separate MP3, shared 1.6384 Mbps, 200 ms per media request, no-store cache and no user seeks. Tested each audio implementation three times per engine at 1× (27 trials):
At 1×, backward assignments occurred in 1/9 legacy HTML5 trials and 0/9 opt-in trials. Media Audio had repeated source boundaries in 3/9 trials, with a maximum interval of 0.043 source seconds. These counts describe this controlled fixture and network condition.
The 1× Media Audio overlaps returned to the start of the source after approximately 29–43 ms of source playback during startup. No skipped source boundaries were observed in those nine 1× Media Audio trials.
For example, in the webkit 1× legacy trial, Player entered waiting at frame 23 (0.767 s). 1 ms later, the paused audio was assigned from 0.988 s to 0.767 s: a 0.221-second backward jump.
Separate
@remotion/mediaAudioReplaced only the separate audio track with
<Audio>from@remotion/media, leaving the HLS video, sequences, material and throttling unchanged. SetdisallowFallbackToHtml5Audioand observed actual Web Audio source-node starts/stops/disconnects, with positive source-node playback and media-duration observations in every trial. The HTML5-only opt-in is not applied to this path.At 3×, tested both the Media Audio defaults and
toneFrequency={1 / 3}to match the original HTML5 audio’s preserved pitch. Each cell below lists the three consecutive trial counts:At 3×, repeated source boundaries occurred in 9/9 Media-default trials and 9/9 pitch-matched trials. This is separate evidence for the Media Audio path; the HTML5 recovery change in this PR does not alter its scheduler.
Observation method: HTML5 columns count native
currentTimeassignments after Player playback starts. Media columns reconstruct played source intervals from actual Web Audio calls and decoded timestamps; they do not infer success from the absence of HTML5 seeks. Cancelled-before-start nodes and pre-play intervals are excluded. Natural EOF uses the full scheduled buffer duration, while explicit stop/disconnect clips it. Differences at or below one decoded-source sample are excluded from boundary counts. The methods measure different API boundaries; event counts are not interchangeable and neither is an acoustic listening test. Source seconds are reported, rather than dividing by playback speed. Nine deterministic observer fixtures passed.Across these 45 additional trials, 45/45 completed both video and the observed separate-audio path; 0 recorded media/error/timeout events. Primary Media Audio covered the decoded 32.544-second source through its end.
Holding-release comparison
Before wiring the public opt-in, compared five recovery variants using the same upstream base and fixture. Each row contains nine sequential low-bandwidth trials: three each in Chromium 156, Firefox 157 and Playwright WebKit 27.2.
The immediate-release variant could seek forward when another wait arrived after
play()clearedpausedbut before the native clock moved. Releasing on clock progress fixed that boundary, but capturing only leads still skipped audio when the next wait began with a small lag. Both cases have deterministic regression tests. The proposed lifecycle also passes the test that a later unrelated native pause returns to ordinary synchronization; the earlier indefinite-recovery version fails that test.All 45 trials completed both tracks without recorded media errors. A separate internal opt-out control reproduced backward assignments in all three Chromium trials (15 / 15 / 16); it is distinct from the public-API checks above. Earlier normal-network 1×/3× controls and 33 browser-operation checks also passed. These earlier controls are supporting lifecycle evidence, not substitutes for testing the new prop.
Conditions and scope
Base:
434aff6af0a5214a677ab9fa895bf6102fdf58d8. Bundled core, Player and media from source on macOS with React 19.2.3 and Mediabunny 1.62.0. The 33-second fixture uses 720p HLS video and separate MP3, 3× playback, a shared 1.6384 Mbps pool, 200 ms artificial delay per media request, no-store cache and no user seeks during the playback comparison. Video uses the canvas-backed@remotion/mediapath. NativecurrentTimeassignments are recorded separately from intentional operation-test seeks. The final public-API runs use the actual prop wiring without replacing or instrumenting the synchronization hook.Playwright WebKit is not branded Safari. Branded Safari results from the earlier PR implementation are not presented as validation of this revision. The full monorepo suite has not been run locally.
AI assistance: implementation and verification were prepared with OpenAI Codex.