1027875 - Add MP4 support to MSE

Status: RESOLVED FIXED

(Core :: Audio/Video, defect)

Description

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Media Source Extensions needs to also support MP4 containers.

Comment 1

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Add MP4 support to MSE

Comment 2

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Fix CTS offset processing in fMP4

Comment 3

[Matthew Gregan [:kinetik]]

Comment on attachment 8443098 [details] [diff] [review]
Add MP4 support to MSE

Review of attachment 8443098 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/media/mediasource/SourceBufferResource.h
@@ +233,5 @@
>    virtual void Unpin() MOZ_OVERRIDE {}
>    virtual double GetDownloadRate(bool* aIsReliable) MOZ_OVERRIDE { return 0; }
>    virtual int64_t GetLength() MOZ_OVERRIDE { return mInputBuffer.GetLength(); }
>    virtual int64_t GetNextCachedData(int64_t aOffset) MOZ_OVERRIDE { return aOffset; }
> +  virtual int64_t GetCachedDataEnd(int64_t aOffset) MOZ_OVERRIDE { return GetLength() == aOffset ? -1 : GetLength(); }

So, this is actually wrong, sorry.  The old code was correct and this should not return a negative value.  GetNextCachedData, on the other hand, does need to return -1 when aOffset == GetLength().  That should fix the assert you're currently hitting, too.

Comment 4

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Add MP4 support to MSE

Still work in progress

Comment 5

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Remove Annex B assertion

Comment 6

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Add fMP4 stuff to CanHandleMediaType

Comment 7

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Add MP4 support to MSE

Comment 8

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Make GetCachedRanges() handle the empty case

Comment 9

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Attachment: Make MP4Reader::GetBuffered() accurate - WORK IN PROGRESS

Comment 10

[Matthew Gregan [:kinetik]]

Comment on attachment 8449969 [details] [diff] [review]
Add MP4 support to MSE

Review of attachment 8449969 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with comments

::: content/media/mediasource/SourceBuffer.cpp
@@ +165,5 @@
> +    // DocType == "webm"
> +    // ...
> +    // 0x18538067 // Segment (must be "unknown" size)
> +    // 0x1549a966 // -> Segment Info
> +    // 0x1654ae6b // -> One or more Tracks

Make the comment appropriate for MP4.

@@ +175,5 @@
> +    uint32_t chunk_size = BigEndian::readUint32(aData);
> +    uint32_t chunk_type = BigEndian::readUint32(aData + 4);
> +    return chunk_size > 8 && chunk_type == 'ftyp';
> +
> +    return true;

Remove the last two lines.

Comment 11

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

Comment on attachment 8449973 [details] [diff] [review]
Make MP4Reader::GetBuffered() accurate - WORK IN PROGRESS

Create a separate bug for GetBuffered()

Comment 12

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

https://tbpl.mozilla.org/?tree=Try&rev=3075b37c20d3

Comment 13

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

https://tbpl.mozilla.org/?tree=Try&rev=9ca40cd021c1

Comment 14

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/8be7f28c5ddd
https://hg.mozilla.org/integration/mozilla-inbound/rev/5c1a090ccc63
https://hg.mozilla.org/integration/mozilla-inbound/rev/00b90bfadaa2
https://hg.mozilla.org/integration/mozilla-inbound/rev/bd855ca58b41
https://hg.mozilla.org/integration/mozilla-inbound/rev/00a68fd085d5

Comment 15

[Tim Taubert [:ttaubert] (inactive)]

https://hg.mozilla.org/mozilla-central/rev/8be7f28c5ddd
https://hg.mozilla.org/mozilla-central/rev/5c1a090ccc63
https://hg.mozilla.org/mozilla-central/rev/00b90bfadaa2
https://hg.mozilla.org/mozilla-central/rev/bd855ca58b41
https://hg.mozilla.org/mozilla-central/rev/00a68fd085d5

Comment 16

[Kacper Michajłow [:kasper93]]

Do you have plans to enable it by default or only with media.mediasource.ignore_codecs pref? (like it's now).

Comment 17

[Matthew Gregan [:kinetik]]

It'll be enabled when it's ready.

Comment 18

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

The MSE part of MP4 is apparently working but there are still issues with the MP4 demuxer.

Comment 19

[Filip Filipović]

This doesn't work under Windows, right? I see there is no "media.mediasource.ignore_codecs" option in my Windows 7 x86.
What needs to be done to make this work in Windows?

Comment 20

[Kacper Michajłow [:kasper93]]

You need to have nightly and set both "media.mediasource.ignore_codecs" and "media.mediasource.enabled" prefs to true. (through about:config). But I wouldn't recommend that atm since it hang itself on youtube. Not sure about mp4 though, but webm works "sometimes".

Comment 21

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

This bug means the support in MSE is there and it plays more than nothing. However both MP4 and WebM hang up on bug 1024858.