Systems and methods for encoding source media in matroska container files for adaptive bitrate streaming using hypertext transfer protocol

• US 9,883,204 B2
• Filed: 01/25/2016
• Issued: 01/30/2018
• Est. Priority Date: 01/05/2011
• Status: Active Grant

• Current Assignee: Sonic IP Incorporated (Endeavor Group Holdings Inc.)
• Original Assignee: Sonic IP Incorporated (Endeavor Group Holdings Inc.)
• Inventors: Braness, Jason, van der Schaar, Auke Sjoerd, Soroushian, Kourosh
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): Fereja, Samuel D
• Application Number:
  US15/005,990
• Publication Number:
  US 20160219303A1
• Time in Patent Office: 736 Days
• Field of Search: None
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2387 Stream processing in response to a playback request from an end-user, e.g. for trick-play
• CPC Class Codes: H04N 21/26258 for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list retrieval of multimedia data based on playlists G06F16/40
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/42607 for processing the incoming bitstream
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8455 : involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 : by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/85406 : involving a specific file format, e.g. MP4 format
• CPC Class Codes: H04N 21/8543 : using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80

Abstract

Systems and methods for encoding source media in Matroska container files for adaptive bitrate streaming utilizing Hypertext Transfer Potocol (HTTP) in accordance with embodiments of the invention are disclosed. One embodiment of the invention includes a processor configured via a source encoding application to ingest at least one multimedia file containing a source video. In addition, the source encoding application further configures the processor to select a portion of the source video, transcode the selected portion of the source video into a plurality of alternative portions of encoded video, where each alternative portion is encoded using a different set of encoding parameters and commences with an intra frame starting a closed Group of Pictures (GOP), write each of the alternative portions of encoded video to an element of a different EBML container file, where each element is located within an EBML container file that also includes another element that indicates the encoding parameters used to encode the alternative portion of encoded video, and add an entry to at least one index that identifies the location of the element containing one of the alternative portions of encoded video within each of the EBML container files.

Claims

1. A playback device configured to decode video from a plurality of alternative video streams with different maximum bitrates and packed in container files, wherein the container files are extensible binary markup language (EBML) files, the playback device comprising:

a processor configured via a playback application to decode at least one multimedia file containing an encoded source video;

wherein the playback application further configures the processor to;

parse a top level index to obtain information that identifies a plurality of EBML container files, wherein each EBML container file of the plurality of EBML container files comprises a set of alternative portions of a source video, wherein the set of alternative portions for each EBML container file is encoded using a different maximum bitrate that is associated with a particular alternative video stream using a different set of encoding parameters, wherein each alternative portion of a set of alternative portions is stored in a Cluster element of the EBML container and commences with an intra frame starting a closed Group of Pictures (GOP), wherein corresponding alternative portions in the sets of alternative portions of different EBML container files share a same time code, wherein the top level index describes at least the maximum bitrate of the particular alternative video stream associated with each EBML container file and is separate from the EBML container files;

request a portion of at least one of the plurality of EBML container files based on the maximum bitrate of the particular alternative stream associated with the at least one EBML container file, wherein the requested portion comprises an element that specifies the set of encoding parameters used to encode the set of alternative portions of the source video contained within the EBML container file;

retrieve an index from a Cues element within the at least one EBML container file, wherein a first index entry of the index references an initial byte for a first Cluster element comprising a first alternative portion and a second index entry of the index references an initial byte for a next Cluster element comprising a second alternative portion;

infer a size of the given Cluster element from at least the first and second index entries of the Cues element within the at least one EBML container file;

request at least a portion of the first alternative portion from the Cluster element of the at least one EBML container file by retrieving a byte range of the at least one EBML container file based on the inferred size and the initial byte of the first Cluster;

receive and buffer the requested portion; and

decode the encoded video contained within the buffered Cluster elements utilizing the set of encoding parameters.

2. The playback device of claim 1, wherein the alternative portions in each of the Cluster elements have a same duration.

3. The playback device of claim 1, wherein the alternative portions in each of the Cluster elements have a two-second duration.

4. The playback device of claim 1, wherein each Cluster element contains a time code and each encoded frame of the alternative portion contained within the Cluster element is contained within a separate BlockGroup element.

5. The playback device of claim 4, wherein a first BlockGroup element in the Cluster element contains the intra frame.

6. The playback device of claim 5, wherein the first BlockGroup element contains a Block element which specifies a time code attribute of the intra frame relative to the time code of the Cluster element.

7. The playback device of claim 1, wherein at least two of the alternative streams of encoded video are encoded at a same display aspect ratio, but with different resolutions using different sample aspect ratios.

8. The playback device of claim 7, wherein the set of encoding parameters used to encode the set of alternative portions of the source video contained within the EBML container file specifies the frame height, frame width, and sample aspect ratio of the video stream contained within the EBML container file.

9. The playback device of claim 8, wherein the playback application further configures the processor to set the frame height, frame width, and sample aspect ratio prior to commencing the decoding of encoded video contained within buffered elements from one of the EBML container files.

10. The playback device of claim 1, wherein at least two of the alternative streams of encoded video are encoded at different frame rates.

11. The playback device of claim 10, where the set of encoding parameters used to encode the set of alternative portions of the source video contained within the EBML container file specifies the frame rate of the video stream contained within the EBML container file.

12. The playback device of claim 11, wherein the playback application further configures the processor to set the frame rate prior to commencing the decoding of encoded video contained within buffered Cluster elements from one of the EBML container files.

13. The playback device of claim 1, wherein the client application further configures the processor to retrieve the index that references an initial byte for each Cluster element within an EBML container file by requesting the portion of the EBML container file that includes the Cues element containing the index.

14. The playback device of claim 1, wherein the top level index comprises SMIL data and the information that identifies the plurality of EBML container files is a plurality of Universal Resource Indicators (URIs) that identify the location of each of the EBML container files.

15. The playback device of claim 1, wherein the Cues element comprises a plurality of Cue Point elements that each includes a time attribute and points to the location of each Cluster element containing an alternative portion within the EBML file.

16. The playback device of claim 15, wherein the playback application further configures the processor to:

search for the location of a Cluster element containing a specific portion of encoded video within the EBML file using the time attributes of the CuePoint elements within the index; and

request a portion of the EBML file including at least the Cluster element identified using the CuePoint elements.

17. The playback device of claim 1, wherein the set of encoding parameters utilized to decode the encoded video include at least one encoding parameter selected from the group consisting of frame rate, frame height, frame width, sample aspect ratio, maximum bitrate, and minimum buffer size.

18. The playback device of claim 1, wherein the playback application further configures the processor to:

measure current streaming conditions by measuring the time taken to receive requested Cluster elements from the time at which the Cluster elements were requested; and

retrieve content from a second EBML container file for a second alternative video stream encoded at a different, second maximum bitrate based on the measured current streaming conditions.

19. The playback device of claim 1, wherein:

the playback application further configures the processor to initially request a portion of a first EBML container file containing the element that specifies the set of encoding parameters of the alternative video stream contained within the first EBML container file; and

the playback application further configures the processor to request a portion of a second EBML container file containing the element that specifies the set of encoding parameters of the alternative video stream contained within the second EBML container file, wherein the playback application selects to retrieve Cluster elements containing portions of encoded video for decoding from the second EBML container file based upon measured streaming conditions and the description of the bitrate of the alternative stream contained within the top level index.

20. The playback device of claim 1, wherein the playback application further configures the processor to request portions of EBML container files from remote servers via Hypertext Transfer Protocol (HTTP) byte range requests.