Audio splitting with codec-enforced frame sizes

• US 9,338,523 B2
• Filed: 12/21/2009
• Issued: 05/10/2016
• Est. Priority Date: 12/21/2009
• Status: Active Grant

• Current Assignee: Dish Technologies LLC (Dish Network Corporation)
• Original Assignee: Echostar Technologies LLC (Echostar Corporation)
• Inventors: Owen, Calvin Ryan
• Primary Examiner(s): Atala, Jamie
• Assistant Examiner(s): Mung, On S
• Application Number:
  US12/643,700
• Publication Number:
  US 20110150099A1
• Time in Patent Office: 2,332 Days
• Field of Search: 375/240.26, 348/512
• US Class Current: 1/1
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 19/167 Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
• CPC Class Codes: G10L 21/055 for synchronising with other signals, e.g. video signals
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 7/52 Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

A method and apparatus for splitting the audio of media content into separate content files without introducing boundary artifacts is described.

Claims

1. A method comprising:

receiving, by a computing system, media content including audio and video;

encoding, by the computing system, the video according to a frame rate;

encoding, by the computing system, the audio according to a codec-enforced frame size;

generating, by the computing system, a plurality of content files, wherein each of the plurality of content files comprises an encoded portion of the video having a fixed-time duration and an encoded portion of the audio having a plurality of full audio frames having the codec-enforced frame size, wherein a duration of the encoded portion of the audio of one or more of the plurality of content files is greater than or less than the fixed-time duration.

2. The method of claim 1, wherein the last of the plurality of full audio frames is not padded with zeros.

3. The method of claim 1, further comprising splitting the media content into the audio and the video, wherein said encoding the video comprises encoding the video using a video codec according to the fixed-time duration, and wherein said encoding the audio comprises encoding the audio using an audio codec according to the codec-enforced frame size.

4. The method of claim 1, further comprising:

buffering encoded frames of the audio;

determining a number of encoded frames needed to fill the current one of the plurality of content files, wherein the number of frames is the smallest integer that is not less than a number of samples needed to fill the current one of the plurality of files divided by the codec-enforced frame size;

determining if there are enough of the encoded frames buffered to fill a current one of the plurality of content files;

if there is enough of the encoded frames buffered, filling the current one of the plurality of content files with the number of frames; and

if there are not enough of the encoded frames buffered, buffering an additional frame of the audio and filling the current one of the plurality of content files with the number of frames and the additional frame.

5. The method of claim 4, wherein said determining if there is enough encoded frames buffered comprises:

multiplying the number of buffered frames by the codec-enforced frame size;

adding a sample offset, if any, from a previous one of the plurality of content files to the product of the multiplication;

determining if the sum is greater than or equal to the number of samples needed to fill the first one of the plurality of content files.

6. The method of claim 4, further comprising determining a sample offset, if any, for a subsequent one of the plurality of content files.

7. The method of claim 6, wherein said determining the sample offset comprises multiplying the number of the encoded frames by the codec-enforced frame size minus the number of samples needed to fill the first one of the plurality of content files and plus the sample offset, if any, from a previous one of the plurality of content files.

8. The method of claim 1, further comprising buffering encoded frames of the audio, and wherein said generating the plurality of content files comprises:

calculating a number of samples needed to fill a current one of the plurality of content files;

calculating a number of frames needed for the current one of the plurality of content files;

adding a frame to the number of frames when the number of samples divided by the codec-enforced frame size is not equally divisible; and

filling the current one of the plurality of content files with the number of frames.

9. The method of claim 1, further comprising buffering encoded frames of the audio, and wherein said generating the plurality of content files comprises:

calculating a number of samples needed to fill a current content file of the one or more of the plurality of content files by multiplying a sampling rate by the fixed-time duration plus a sample offset, if any, from a previous one of the plurality of content files;

calculating a number of frames needed to fill the current content file by dividing the number of samples by the codec-enforced frame size; and

if the remainder of the division is zero, filling the current content file with the number of frames; and

if the remainder of the division is greater than zero, incrementing the number of frames by one and filling the current content file with the incremented number of frames.

10. The method of claim 9, wherein said generating the plurality of content files further comprises:

multiplying the number of frames by the codec-enforced frame size to convert back to the number of samples needed to fill the current content file;

calculating the duration of the encoded portion of the audio of the current content file by dividing the number of samples by the sampling rate;

determining a presentation offset for a subsequent one of the plurality of content files by subtracting the duration from the fixed-time duration; and

updating a sample offset for the subsequent one of the plurality of content files by multiplying the number of frames by the codec-enforced frame size minus the number of samples needed to fill the first one of the plurality of content files and plus the sample offset, if any, from the previous one of the plurality of content files.

11. The method of claim 1, wherein said receiving comprises receiving the media content as a plurality of raw streamlets, wherein each of the plurality of raw streamlets comprises a portion of the media content having the fixed-time duration.

12. The method of claim 11, wherein:

said receiving the media content comprises;

receiving a first of the plurality of raw streamlets and a second of the plurality of raw streamlets; and

splitting the audio and the video of the first raw streamlet and of the second raw streamlet;

said encoding the video comprises;

encoding the video of the first raw streamlet, wherein the video of the first raw streamlet is stored in a first of the plurality of content files; and

encoding the video of the second raw streamlet, wherein the video of the second raw streamlet is stored in a second of the plurality of content files;

said encoding the audio comprises;

encoding the audio of the first raw streamlet into a first plurality of audio frames;

buffering the first plurality of audio frames;

determining if there are enough buffered frames to fill the first content file;

when there are not enough buffered frames to fill the first content file, encoding the audio of the second raw streamlet into a second plurality of audio frames and buffering the second plurality of audio frames;

when there are enough buffered frames to fill the first content file, storing the buffered audio frames into the first content file.

13. The method of claim 1, wherein said fixed-time duration is approximately two seconds, wherein the audio is sampled at approximately 48,000 samples per second, wherein the codec-enforced frame size is 1024 samples per frame, wherein the audio portions of the first three of the plurality of content files each comprises ninety-four audio frames, and wherein the audio portion of a fourth of the plurality of content files comprises ninety-three audio frames, and wherein each of the video portions of the four content files comprises approximately sixty video frames.

14. The method of claim 1, wherein said fixed-time duration is approximately two seconds, wherein the audio is sampled at approximately 44,100 samples per second, wherein the codec-enforced frame size is 1024 samples per frame, wherein the audio portion of a first of the plurality of content files comprises eight-seven audio frames and a second of the plurality of content files comprises eighty-six audio frames.

15. The method of claim 1, wherein the codec-enforced frame size is 2048 samples per frame.

16. A computing system comprising:

means for receiving media content including video and audio;

means for encoding the video according to a frame rate;

means for encoding the audio according to a fixed-frame size;

means for segmenting the encoded video into a plurality of portions, wherein each portion of the encoded video is stored in a separate content file; and

means for splitting the encoded audio into the separate content files without introducing boundary artifacts, wherein the encoded audio of a first content file of the separate content files has a duration that is greater than or less than a duration of the portion of the encoded video stored in the first content file.

17. The apparatus of claim 16, further comprising:

means for tracking a sample offset, if any, for each of the content files; and

means for tracking a presentation offset, if any, for each of the content files.

18. A computing device comprising:

a splitter to receive media content including audio and video and to split the audio and the video;

a video encoder coupled to receive the video from the splitter and to encode the video according to a frame rate;

an audio encoder coupled to receive the audio from the splitter and to encode the audio according to a codec-enforced frame size; and

an audio-splitting multiplexer to generate a plurality of content files, wherein each of the plurality of content files comprises an encoded portion of the video having a fixed-time duration and an encoded portion of the audio having a plurality of full audio frames having the codec-enforced frame size, wherein a duration of the encoded portion of the audio of one or more of the plurality of content files is greater than or less than the fixed-time duration.

19. The apparatus of claim 18, wherein the last of the plurality of full audio frames is not padded with zeros.

20. The apparatus of claim 18, wherein the computing device further comprises an audio frame buffer to buffer encoded frames of the audio.

21. A non-transitory computer-readable storage medium storing instruction thereon when executed by a computing device cause the computing device to perform a method, comprising:

receiving media content including audio and video;

encoding the video according to a frame rate;

encoding the audio according to a codec-enforced frame size;

generating a plurality of content files, wherein each of the plurality of content files comprises an encoded portion of the video having a fixed-time duration and an encoded portion of the audio having a plurality of full audio frames having the codec-enforced frame size, wherein a duration of the encoded portion of the audio of one or more of the plurality of content files is greater than or less than the fixed-time duration.

22. The computer-readable storage medium of claim 21, wherein the method further comprises:

buffering encoded frames of the audio;

determining a number of encoded frames needed to fill the current one of the plurality of content files, wherein the number of frames is the smallest integer that is not less than a number of samples needed to fill the current one of the plurality of files divided by the codec-enforced frame size;

determining if there are enough of the encoded frames buffered to fill a current one of the plurality of content files;

if there is enough of the encoded frames buffered, filling the current one of the plurality of content files with the number of frames; and

if there are not enough of the encoded frames buffered, buffering an additional frame of the audio and filling the current one of the plurality of content files with the number of frames and the additional frame.