Multimedia distribution system for multimedia files with interleaved media chunks of varying types

• US 9,369,687 B2
• Filed: 05/19/2014
• Issued: 06/14/2016
• Est. Priority Date: 12/08/2003
• Status: Active Grant

• Current Assignee: Sonic IP Incorporated (Endeavor Group Holdings Inc.)
• Original Assignee: Sonic IP Incorporated (Endeavor Group Holdings Inc.)
• Inventors: Ahsan, Abou Ul Aala, Braness, Jason, Durham, Christopher, Sayre, Timothy, Cote, Benjamin, Thompson, Darrius Nguyen, Rota, Jerome, Mayo, Alastair, Grab, Eric William, Donaldson, Jerald, Hitchcock, Heather, Chavarria, Damien, Floyd, Michael John, Fudge, Brian, Li, Adam H.
• Primary Examiner(s): HARVEY, DAVID E
• Application Number:
  US14/281,791
• Publication Number:
  US 20150104153A1
• Time in Patent Office: 757 Days
• Field of Search: 386337-340, 386/244
• US Class Current: 1/1
• CPC Class Codes: G11B 20/00086 Circuits for prevention of unauthorised reproduction or copying, e.g. piracy indicating unauthorised use of record carriers in general G11B23/28; scrambling for television signal recording H04N5/913; network architectures or network protocols for network security H04L63/00; cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
• CPC Class Codes: G11B 20/00731 involving a digital rights management system for enforcing a usage restriction
• CPC Class Codes: G11B 20/00739 wherein the usage restriction is associated with a specific geographical region
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 21/4856 for language selection, e.g. for the menu or subtitles
• CPC Class Codes: H04N 21/845 Structuring of content, e.g. decomposing content into time segments
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/278 Subtitling
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 5/92 Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
• CPC Class Codes: H04N 9/8047 using transform coding

Abstract

Methods, systems, and computer readable media for encoding, distributing, and decoding multimedia files with interleaved video, subtitle, and audio chunks are provided. Multimedia files in accordance with embodiments of the present invention can contain multiple video tracks, multiple audio tracks, multiple subtitle tracks, data that can be used to generate a menu interface to access the contents of the file and ‘meta data’ concerning the contents of the file. In one embodiment, the generated ‘audio,’ ‘video’ and ‘subtitle’ chunks are interleaved so that the ‘audio’ and ‘subtitle’ chunks are located within the file prior to the ‘video’ chunks containing information concerning the video frames to which they correspond. In other embodiments, the ‘audio’ and ‘subtitle’ chunks can be located after the ‘video’ chunks to which they correspond. One embodiment of a multimedia file in accordance with the present invention includes a series of encoded video frames and encoded menu information.

Claims

1. A system for decoding multimedia files, the system comprising:

a non-transitory memory storing a decoding application;

a network interface;

a processing unit, wherein, upon execution, the decoding application directs the processing unit to;

obtain a multimedia file using the network interface, wherein the multimedia file comprises;

a header for identifying the multimedia file and providing the residual length of the multimedia file after the header;

a plurality of chunks for carrying information, the chunks comprising a plurality of video chunks, a plurality of audio chunks, a plurality of subtitle chunks, and an index chunk, the video chunks being portions of at least one video track, the video track comprising a series of encoded video frames, the audio chunks being portions of at least one audio track, the subtitle chunks being portions of at least one subtitle track, the subtitle chunks comprising information concerning a start time and a stop time, and the index chunk carrying information supplying references to the location within the file of each of the video chunks;

wherein the audio chunks, subtitle chunks, and video chunks are interleaved so that the audio and subtitle chunks are located between video chunks and within the file prior to the video chunks containing information concerning the video frames to which they correspond; and

wherein the video chunk following a subtitle chunk includes the portion of the video track that occurs at the start time of the particular subtitle contained within the subtitle chunk and the audio chunk preceding a video chunk contains the portion of the audio track corresponding to the portion of the video track contained in the video chunk;

access the plurality of chunks for carrying information from the multimedia file;

inspect the multimedia file to identify and select at least one subtitle track;

decode a video chunk to produce a decoded video frame and identify a previous subtitle chunk that contains subtitle information for the decoded video chunk;

construct a frame of video for display by superimposing the subtitle from the identified subtitle chunk on the decoded video frame by;

determine whether the previous video frame to the decoded video frame included a previous subtitle;

superimpose the given subtitle on decoded video frame when the timing information for the previous subtitle indicates that the previous subtitle should be displayed with the decoded frame; and

when either (1) the previous frame does not include a subtitle or (2) the timing information for the previous subtitle on the previous frame indicates that the previous subtitle should not be displayed in conjunction with the decoded frame, seek a subtitle chunk for the selected subtitle track;

when a given subtitle chunk for the selected subtitle track is sought and is located, then superimpose the subtitle information from the given subtitle chunk on the decoded video frame.

2. The system for decoding multimedia files of claim 1, wherein:

each subtitle chunk includes information concerning the position within a frame in which the subtitle should be located; and

the decoding application further directs the processing unit to superimpose the subtitle in the position within each video frame indicated by the information within the subtitle chunk.

3. The system for decoding multimedia files of claim 1, wherein:

a subtitle chunk includes information concerning the color of the subtitle; and

the decoding application further directs the processing unit to superimpose the subtitle in the color indicated by the color information within the subtitle chunk.

4. The system of claim 1, wherein the video chunks 1 through n−

1 contain sufficient information for a decoder to decode the video frame n.

5. The system of claim 1, wherein the audio chunks are interleaved between the video chunks such that the video chunks are separated by an audio chunk from each of the audio tracks and the audio chunks separating the video chunks contain audio information for the portions of the audio tracks corresponding to the video information contained within the video chunk following the audio chunk.

6. The system of claim 1, wherein the encoded audio track is provided in chunks that do not contain audio information corresponding to the contents of a corresponding video chunk, and encoding the at least one encoded audio track encoded as audio chunks involve identifying the audio information in the encoded audio track that accompanies the video chunk and extracting the audio information from the existing audio chunks to create a new audio chunk.

7. The system of claim 1, wherein the audio and subtitle chunks are interleaved between the video chunks such that each particular audio/subtitle chunk is placed relative to a corresponding video chunk based on an amount of video capable of being buffered by a device capable of displaying the multimedia file.

8. A system for encoding multimedia files, the system comprising:

a non-transitory memory storing an encoding application;

a network interface;

a processing unit, wherein, upon execution, the encoding application directs the processing unit to;

obtain source media using the network interface, wherein the source media comprises video information, audio information, and subtitle information, wherein the obtained audio information comprises a particular encoded audio track stored in chunks that do not contain audio information corresponding to the contents of corresponding video chunks;

encode at least one video track as a plurality of video chunks, the video chunks being portions of at least one video track, the video track comprising a series of encoded video frames;

encode at least one audio track as audio chunks, the audio chunks being portions of at least one audio track;

encode the particular encoded audio track a set of new audio chunks by;

identifying audio information in the particular encoded audio track that corresponds to video information in a particular video track, wherein the particular video track has been encoded into a plurality of video chunks;

extracting the audio information from the existing audio chunks of the particular encoded audio track;

creating new audio chunks corresponding to the plurality of video chunks of the particular video track using the extracted audio information;

encode at least one subtitle track as a plurality of subtitle chunks, wherein each subtitle chunk includes information concerning a subtitle, each subtitle chunk includes information concerning a start time and a stop time for the subtitle;

encode at least one index chunk that includes information concerning the location of video chunks within the multimedia file;

interleave the audio chunks, subtitle chunks, and video chunks so that the audio and subtitle chunks are located between video chunks and within the file prior to the video chunks containing information concerning the video frames to which they correspond, and so that the video chunk following a subtitle chunk includes the portion of the video track that occurs at the start time of the particular subtitle contained within the subtitle chunk and the audio chunk preceding a video chunk contains the portion of the audio track corresponding to the portion of the video track contained in the video chunk;

write the interleaved chunks to a single file; and

transmit the single file using the network interface.

9. The system of claim 8, wherein the audio, video and subtitle chunks are interleaved such that the video chunk immediately following the audio and subtitle chunk contains the first video frame corresponding to the audio or subtitle.

10. The system of claim 8, wherein the audio chunks are interleaved between the video chunks such that the video chunks are separated by audio chunks from each of the audio tracks and the audio chunks separating the video chunks contain audio information for the portions of the audio tracks corresponding to the video information contained within the video chunk following the audio chunk.

11. The system of claim 8, wherein the audio and subtitle chunks are interleaved between the video chunks such that each particular audio/subtitle chunk is placed relative to a corresponding video chunk based on an amount of video capable of being buffered by a device capable of displaying the multimedia file.

12. The system of claim 8, wherein video chunks 1 through n−

1 contain sufficient information for a decoder to decode the video frame n.