Encoding streaming media as a high bit rate layer, a low bit rate layer, and one or more intermediate bit rate layers

• US 8,325,800 B2
• Filed: 05/07/2008
• Issued: 12/04/2012
• Est. Priority Date: 05/07/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Holcomb, Thomas W., Mehrotra, Sanjeev, Smirnov, Serge, Siravara, Bharath
• Primary Examiner(s): Sheikh, Ayaz
• Assistant Examiner(s): CHAU, PETER P
• Application Number:
  US12/116,878
• Publication Number:
  US 20090279605A1
• Time in Patent Office: 1,672 Days
• Field of Search: None
• US Class Current: 375/240.03
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/166 concerning the amount of transmission errors, e.g. bit error rate [BER]
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]

Abstract

A method of encoding an input video stream comprising a video component and an audio component is disclosed. The input video stream is split into a plurality of segments, each comprising a plurality of frames. Each of the segments is encoded as a low bit rate layer, a high bit rate layer, and one or more intermediate bit rate layers. The bit rate of the low bit rate layer is selected such that a network streaming the segment will always be able to stream the segment encoded as the low bit rate layer. The bit rate of the high bit rate layer is selected such that the segment is able to be decoded and played back at or above a quality threshold. The bit rates of the intermediate bit rate layers are produced by applying a bit rate factor to another bit rate.

Claims

1. A computer system comprising a processing unit, memory and storage containing instructions which, when executed by the processing unit, cause the system to perform a method of encoding a sequence of frames organized as plural segments, each of the plural segments comprising a plurality of the frames, the method comprising:

selecting a low bit rate at which to encode a given segment of the plural segments for a low bit rate layer, wherein the low bit rate is identical for each of the plural segments;

selecting a high bit rate at which to encode the given segment for a high bit rate layer, wherein the high bit rate is set for the plural segments on a segment-by-segment basis depending on complexity of the given segment and a quality threshold for encoding;

selecting one or more intermediate bit rates between the low bit rate and the high bit rate at which to encode the given segment for one or more intermediate bit rate layers, respectively, wherein a number of intermediate bit rate layers is variable depending on the high bit rate for the given segment;

encoding the given segment at the high bit rate to produce a high bit rate chunk for the given segment;

encoding the given segment at the low bit rate to produce a low bit rate chunk for the given segment;

encoding the given segment at each of the one or more intermediate bit rates to produce one or more intermediate bit rate chunks for the given segment; and

outputting the high bit rate chunk, the low bit rate chunk, and the one or more intermediate bit rate chunks for the given segment.

2. The system of claim 1 wherein the low bit rate is selected based on a lowest supported bit rate regardless of complexity of the given segment.

3. The system of claim 1 wherein the high bit rate is selected based upon a bit rate that provides quality at or above the quality threshold.

4. The system of claim 3 wherein each of the one or more intermediate bit rates is produced by applying a bit rate factor to a next lower bit rate to obtain the intermediate bit rate.

5. The system of claim 4 wherein the bit rate factor is in the range of 1.5 to 2.0.

6. The system of claim 1 wherein the one or more intermediate bit rates are chosen such that a highest intermediate bit rate is between one-half to two-thirds of the high bit rate.

7. The system of claim 1 wherein the plurality of frames in the given segment is produced based upon a pre-determined number of frames.

8. The system of claim 1 wherein the plurality of frames in the given segment is a variable number of frames from segment to segment.

9. The system of claim 1 wherein bit rate is decreased by performing at least one of:

increasing a quantization parameter of the given segment; and

decreasing a spatial resolution parameter of the given segment.

10. The system of claim 1 wherein the method further comprises delivering to a client one of the low bit rate chunk, the high bit rate chunk and the one or more intermediate bit rate chunks for the given segment.

11. A method of encoding a plurality of frames comprising:

selecting plural segments each comprising a portion of the plurality of frames;

selecting a lowest bit rate at which to encode each of the plural segments for a lowest bit rate layer, wherein the lowest bit rate is identical for each of the plural segments;

for each given segment of the plural segments;

selecting a highest bit rate at which to encode the given segment for a highest bit rate layer, wherein the highest bit rate is set on a segment-by-segment basis depending on complexity of the given segment and a quality threshold for encoding, and wherein the highest bit rate is variable among the plural segments; and

selecting one or more intermediate bit rates between the lowest bit rate and the highest bit rate at which to encode the given segment for one or more intermediate bit rate layers, respectively, wherein a number of intermediate bit rate layers is variable depending on the highest bit rate for the given segment;

encoding each of the plural segments according to the lowest bit rate and their respective highest bit rate and one or more intermediate bit rates to produce a lowest bit rate chunk, a highest bit rate chunk and one or more intermediate bit rate chunks for the segment;

and for each of the plural segments, outputting the lowest bit rate chunk, the highest bit rate chunk and the one or more intermediate bit rate chunks for the segment.

12. The method of claim 11 wherein each of the plural segments generally comprises the same number of frames.

13. The method of claim 11 wherein number of frames per segment is variable for the plural segments.

14. The method of claim 11 wherein the lowest bit rate is selected based upon a network connection and regardless of complexity of the plural segments, respectively.

15. The method of claim 11 wherein the highest bit rate for a given segment is a bit rate that produces quality at or above a quality threshold.

16. The method of claim 15 wherein the quality threshold is signaled by a user.

17. The method of claim 15 wherein the quality threshold is signaled in source code.

18. The method of claim 11 wherein the one or more intermediate bit rates for a given segment are chosen such that each one of the one or more intermediate bit rates is in the range of 1.5 to 2.0 times a lower bit rate.

19. The method of claim 11 wherein bit rate is increased by performing at least one of:

decreasing a quantization parameter; and

increasing a spatial resolution parameter.

20. The method of claim 11 wherein each of the plural segments is encoded independent of other ones of the plural segments.

21. The method of claim 11 wherein the lowest bit rate is set regardless of complexity of the plural segments, respectively.

22. The method of claim 11 further comprising, for each of the plural segments, delivering to a client one of the lowest bit rate chunk, the highest bit rate chunk and the one or more intermediate bit rate chunks for the segment.

23. A system for encoding a plurality of frames comprising:

a buffer for storing digital media data;

a segmenter for splitting the plurality of frames into plural segments; and

an encoder operable to;

encode each of the plural segments at a low bit rate to produce a low bit rate chunk for that segment for a low bit rate layer, wherein the low bit rate is identical for each of the plural segments and set regardless of complexity of the plural segments, respectively;

encode each of the plural segments at a high bit rate at which that segment has quality above a threshold for encoding to produce a high bit rate chunk for that segment for a high bit rate layer, wherein the high bit rate is set on a segment-by-segment basis depending on the threshold for encoding and the complexity of the plural segments, respectively; and

encode each of the plural segments at one or more intermediate bit rates to produce one or more intermediate bit rate chunks for that segment for one or more intermediate bit rate layers, respectively, wherein a number of intermediate bit rate layers varies depending on the high bit rate for each of the plural segments.

24. The system of claim 23 wherein, for each of the plural segments, an intermediate bit rate is in the range of 1.5 to 2.0 times a lower bit rate, and bit rate is increased by performing at least one of decreasing a quantization parameter and increasing a spatial resolution parameter of the segment.