AV1 MP4 sync samples can be written without a Sequence Header OBU

The original affected file was produced with an older Mediabunny version, but the relevant muxing behavior is still present in mediabunny@1.49.0.

Summary

When muxing AV1 WebCodecs output into MP4, Mediabunny can mark AV1 key packets as MP4 sync samples even when the packet does not contain a Sequence Header OBU.

The AV1 ISOBMFF binding requires an AV1 sync sample to be a random access point, with a Sequence Header OBU before the first Frame Header OBU.

In our tests, some Chrome AV1 WebCodecs encoder configurations include a Sequence Header OBU in every key packet. However, we also observed an output file produced on Chrome / Windows 11 / NVIDIA RTX 4060 where only the first key packet contained the Sequence Header OBU. Later key packets contained only:

Temporal Delimiter -> Frame

The resulting MP4 plays correctly from the start, but fast seek / decode starting from a later key sample can fail.

Observed packet layout

Working output:

key sample: TD -> Sequence Header -> Frame

Affected output:

Sample 1:   TD -> Sequence Header -> Frame
Sample 37:  TD -> Frame
Sample 97:  TD -> Frame
Sample 157: TD -> Frame
Sample 217: TD -> Frame

This looks like a broken random access point rather than missing frame data, because decode from the beginning works.

Reproduction sample

I no longer have the original affected .mp4 file available, so I cannot attach it here.

The OBU layout above was captured from that file before it was lost. I realize this makes the issue harder to reproduce directly. Still, the muxing behavior below may be worth checking: AV1 key packets are currently written as MP4 sync samples based on packet.type === 'key', without validating that the sample contains a Sequence Header OBU.

Current Mediabunny behavior

In src/isobmff/isobmff-muxer.ts, addEncodedVideoPacket() starts with:

let packetData = packet.data

Only AVC/HEVC packets are normalized through the Annex B transformation path. AV1 packets keep the original packet data.

In src/isobmff/isobmff-boxes.ts, stss is built from samples where sample.type === 'key':

const keySamples = [...trackData.samples.entries()].filter(([, sample]) => sample.type === 'key')

So an AV1 EncodedPacket with packet.type === 'key' becomes an MP4 sync sample even if its OBUs are only:

TD -> Frame

av1C is currently generated from the codec string:

return box('av1C', generateAv1CodecConfigurationFromCodecString(trackData.info.decoderConfig.codec))

This writes the fixed AV1 configuration bytes, but does not add configOBUs from packet data.

Expected behavior

Every AV1 sample written as an MP4 sync sample should be decodable as an AV1 random access point.

Actual behavior

Later AV1 key packets can be written as MP4 sync samples without a Sequence Header OBU before the frame.

The file still decodes from the start, but fast seek / decode starting at one of those later sync samples can fail.

Possible implementation direction

One possible fix would be to handle AV1 packets with a small normalization step during muxing:

1. Inspect OBUs in AV1 packets.
2. Cache the latest complete Sequence Header OBU seen for the track.
3. If a key packet has no Sequence Header OBU, insert the cached Sequence Header before the first Frame Header OBU or Frame OBU.
4. Avoid duplicating it when the packet already contains a Sequence Header OBU.
5. If the packet starts with a Temporal Delimiter OBU, keep it first and insert the Sequence Header after it.

src/codec-data.ts already has useful parsing helpers:

• iterateAv1PacketObus() identifies OBU types.
• extractAv1CodecInfoFromPacket() already searches for OBU_SEQUENCE_HEADER.

One implementation detail: iterateAv1PacketObus() yields OBU payload data, not the full OBU bytes. The muxer would need to preserve or reconstruct the complete Sequence Header OBU, including the OBU header and LEB128 size field.

Writing the Sequence Header into av1C.configOBUs may also be useful, but the important requirement is that every AV1 MP4 sync sample remains a valid random access point.

References

• AV1 ISOBMFF sync sample requirements: https://aomediacodec.github.io/av1-isobmff/
• AV1 frame header semantics: https://github.com/AOMediaCodec/av1-spec/blob/master/07.bitstream.semantics.md