Systems and methods for encoding media including subtitles for adaptive bitrate streaming

• US 9,025,659 B2
• Filed: 09/01/2011
• Issued: 05/05/2015
• Est. Priority Date: 01/05/2011
• Status: Active Grant

• Current Assignee: Sonic IP Incorporated (Endeavor Group Holdings Inc.)
• Original Assignee: Sonic IP Incorporated (Endeavor Group Holdings Inc.)
• Inventors: Soroushian, Kourosh, Douglas, Scott, Bramwell, Steve, Braness, Jason, Shivadas, Abhishek
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): ELAHI, SHAN E
• Application Number:
  US13/224,298
• Publication Number:
  US 20120170643A1
• Time in Patent Office: 1,342 Days
• Field of Search: 375/240.01
• US Class Current: 375/240.01
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/2387 Stream processing in response to a playback request from an end-user, e.g. for trick-play
• CPC Class Codes: H04N 21/26258 for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list retrieval of multimedia data based on playlists G06F16/40
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/42607 for processing the incoming bitstream
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44209 Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8455 : involving pointers to the content, e.g. pointers to the I-frames of the video stream
• CPC Class Codes: H04N 21/8456 : by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 21/85406 : involving a specific file format, e.g. MP4 format
• CPC Class Codes: H04N 21/8543 : using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80

Abstract

Systems and methods for adaptive bitrate streaming of media including subtitles utilizing HTTP in accordance with embodiments of the invention are disclosed. One embodiment of the invention includes selecting a portion of the source media using the source encoder, transcoding the selected portion of the source media into a plurality of alternative portions of encoded video, where each alternative portion is encoded using a different set of encoding parameters and commences with an intra frame at the start of a closed group of pictures (GOP), writing each of the alternative portions of encoded video to a separate container file using the source encoder, writing a subtitle stream segment from each subtitle track corresponding to the selected portion of the source media to a separate container file using the source encoder, and associating a font file with at least one of the container files containing subtitle stream segments.

Claims

1. A method of encoding source media including a subtitle track as a plurality of alternative video streams and a subtitle stream using a source encoder wherein the subtitle track is textually encoded, the method comprising:

selecting a portion of the source media using the source encoder;

transcoding the selected portion of the source media into a plurality of alternative portions of encoded video using the source encoder, where each alternative portion is encoded using a different set of encoding parameters and commences with an intra frame at the start of a closed group of pictures (GOP);

writing each of the plurality of alternative portions of the selected portion of encoded video to a separate one of a plurality of container files using the source encoder;

writing a subtitle stream segment from the subtitle track corresponding to the selected portion of the source media to a separate one of the plurality container files using the source encoder;

associating a font file with the one of the plurality of container files containing subtitle stream segments wherein the font file includes information for representing the encoded text of the stream segment;

attaching the font file to the associated one of the plurality of container files; and

including a separate element in the one of the plurality of container files that includes metadata describing the font file.

2. The method of claim 1, further comprising compressing the font file.

3. The method of claim 1, further comprising reducing the size of the font file by subsetting the font file, where the subsetted font file includes all of the characters within the subtitle stream segments within the one of the plurality of container files with which the font file is associated.

4. The method of claim 1 further comprising breaking the font file into a plurality of font files, where each of the plurality of the font files includes characters displayed during a particular subtitle stream segment.

5. The method of claim 4, wherein breaking the font file into a plurality of font files comprises generating font files corresponding to different subtitle stream segments.

6. The method of claim 5, wherein the plurality of font files is assembled as attachments to a single container file.

7. The method of claim 5, wherein starting and ending timecodes are used to indicate a time period over which each of the plurality of font files is used for rendering subtitles.

8. The method of claim 5, further comprising generating a union font file using the plurality of font files that includes characters that appear in at least two of the plurality of font files and subsetting the plurality of font files to remove the characters that are in the union font file.

9. The method of claim 5, further comprising subsetting each plurality of font files so that characters used in previous subtitle stream segments are removed from the font file.

10. The method of claim 1, wherein the selected portions of source media are selected based upon their duration.

11. The method of claim 10, wherein the selected portions of source media each have a duration of two seconds.

12. The method of claim 1, wherein the container files containing the alternative video streams and the subtitle streams are Extensible Binary Markup Language (EBML) container files.

13. The method of claim 12, wherein:

the element of the EBML container file to which each alternative portion of encoded video is written is a Cluster element; and

each Cluster element to which each of the alternative portions of encoded video is written is assigned the same time code.

14. The method of claim 12, wherein the element of the EBML container file to which the subtitle stream segment is written is a Cluster element containing the same time code as the Cluster elements to which the alternative portions of encoded video are written.

15. The method of claim 1, further comprising creating a top level index file that identifies each of the container files containing the alternative portions of encoded video and identifies the container files that include the at least one subtitle stream.

16. The method of claim 1, wherein the metadata includes the location of the font file within the container file and information describing the file.

17. A source encoder configured to encode source media including a subtitle track as a plurality of alternative video streams and a subtitle stream encoder wherein the subtitle track is textually encoded, the source encoder comprising:

a processor;

instructions for directing the processor to;

select a portion of the source media;

transcode the selected portion of the source media into a plurality of alternative portions of encoded video, where each alternative portion is encoded using a different set of encoding parameters and commences with an intra frame at the start of a closed group of pictures (GOP);

write each of the alternative portions of encoded video to a separate one of a plurality of container files using the source encoder;

write a subtitle stream segment from each subtitle track corresponding to the selected portion of the source media to a separate one of the plurality of container files using the source encoder;

associate a font file with the one of the plurality container files containing subtitle stream segments;

attach the font file to the one of the plurality of container files containing the subtitle segments; and

include a separate element in the one of the plurality of container files including the subtitle segments that includes metadata describing the attached font file; and

a memory for storing instructions.