MULTIPLE BIT RATE VIDEO ENCODING USING VARIABLE BIT RATE AND DYNAMIC RESOLUTION FOR ADAPTIVE VIDEO STREAMING

• US 20100189183A1
• Filed: 01/29/2009
• Published: 07/29/2010
• Est. Priority Date: 01/29/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Mehrotra, Sanjeev, Holcomb, Thomas W., Lin, Chih-Lung, Chen, William, Spears, Stacey, Gu, Chuang, Chan, Chun-Wei
• Granted Patent: US 8,396,114 B2
• Time in Patent Office: Days
• US Class Current: 375/240.280
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments

Abstract

A video encoding system encodes video streams for multiple bit rate video streaming using an approach that permits the encoded bit rate to vary subject to a peak bit rate and average bit rate constraints for higher quality streams, while a bottom bit rate stream is encoded to achieve a constant chunk rate. The video encoding system also dynamically decides an encoding resolution for segments of the multiple bit rate video streams that varies with video complexity so as to achieve a better visual experience for multiple bit rate streaming.

Claims

1. A method of processing input video for variable bit rate encoding of segmented compressed video streams for multi bit rate video streaming, the method comprising:

receiving input of raw video content to be encoded for multi bit rate video streaming;

encoding the raw video content in segments comprising groups of pictures as a plurality of compressed video streams ranging in video quality from a top video stream to a bottom video stream, wherein said encoding comprises;

for the top video stream, encoding the raw video content at a variable bit rate subject to peak bit rate and average bit rate constraints;

for the bottom video stream, encoding the raw video content at a constant chunk rate, wherein the constant chunk rate is a constant number of pictures per unit time; and

producing an encoded set of the video streams segmented into the groups of pictures.

2. The method of claim 1 wherein said encoding the top video stream comprises encoding with a variable bit rate approach targeting a designated average bit rate and constrained to remain under a peak bit rate.

3. The method of claim 2 wherein said encoding the raw video content further comprises, for one or more intermediate video streams, also encoding the raw video content at a variable bit rate subject to peak bit rate and average bit rate constraints, wherein the peak bit rate and average bit rate constraints of the top video stream and intermediate video streams decrease progressively.

4. The method of claim 3 further comprising:

receiving input designating maximum and minimum bit rates, as well as a number of compressed video streams to be encoded for multi bit rate video streaming;

selecting peak bit rate and average bit rate for the top video stream and intermediate video streams to progressively decrease.

5. The method of claim 4 wherein said selecting peak bit rate and average bit rate comprises selecting the peak and average bit rates according to a uniform distribution.

6. The method of claim 4 wherein said selecting peak bit rate and average bit rate comprises selecting the peak and average bit rates according to a logarithmic distribution.

7. The method of claim 4 wherein said selecting peak bit rate and average bit rate comprises selecting the peak and average bit rates to decrease proportionally from top to bottom video streams.

8. The method of claim 1 further comprising:

selectively streaming the segments of the encoded video streams over a network to a client video playing device, wherein the streamed segments are selected from among the encoded video streams according to client-side rate control decisions based on network conditions.

9. A method of processing input video for dynamic resolution encoding of segmented compressed video streams for multi bit rate video streaming, the method comprising:

receiving input of raw video content to be encoded for multi bit rate video streaming;

encoding the raw video content in segments comprising groups of pictures as a plurality of compressed video streams ranging in video quality from a top video stream to a bottom video stream, wherein said encoding comprises;

for a group of pictures of a video stream to be encoded, obtaining at least three encoding samples for varying resolution and quantization step size;

establishing a model relating resolution, quantization step size and resulting encoded size for the group of pictures;

performing a texture measurement on the group of pictures;

determining a quantization threshold for resizing resolution based on the texture measurement;

dynamically deciding a video resolution and quantization step size for encoding the group of pictures based on the model and quantization threshold for resizing; and

encoding the group of pictures with the dynamically decided video resolution and quantization step size.

10. The method of claim 9 further comprises:

detecting scene changes in the raw video content; and

wherein said dynamically deciding video resolution comprises applying dynamic video resolution changes at scene change boundaries between groups of pictures.

11. The method of claim 9 wherein said establishing the model comprises:

establishing a linear model for a relation of quantization step size to encoded size for a given video resolution based on at least two out of the at least three encoding samples of the group of pictures where the encoded size is sampled at the given video resolution and varying quantization step size.

12. The method of claim 11 wherein said dynamically deciding the video resolution comprises:

determining a quantization step size that yields an encoded size corresponding to a desired bit rate for the group of pictures when encoding at the given video resolution according to the established linear model relating quantization step size to encoded size for the given video resolution;

comparing the determined quantization step size to the quantization threshold for resizing resolution;

if the comparison is not indicative of resizing resolution, deciding to encode the group of pictures using the given video resolution and determined quantization step size; and

otherwise, deciding to resize the video resolution for encoding the group of pictures.

13. The method of claim 12 wherein said establishing the model further comprises:

establishing a linear model for a relation of video resolution to encoded size for a given quantization step size based on at least two out of the at least three encoding samples of the group of pictures where the encoded size is sampled at the given quantization step size and varying video resolution.

14. The method of claim 13 wherein said dynamically deciding the video resolution comprises:

in the case where the video resolution for encoding the group of pictures is to be resized, determining a resized video resolution that yields an encoded size corresponding to a desired bit rate for the group of pictures according to the established linear model relating video resolution to encoded size for the given quantization step size.

15. The method of claim 14 wherein, in the case where the video resolution for encoding the group of pictures is to be resized, said dynamically deciding the quantization step size comprises:

establishing a linear model relating quantization step size to resolution at the desired bit rate for the group of pictures based on the at least three encoding samples; and

deciding the quantization step size for the resized video resolution according to the linear model relating quantization step size to resolution.

16. The method of claim 9 further comprising:

selectively streaming the segments of the encoded video streams over a network to a client video playing device, wherein the streamed segments are selected from among the encoded video streams according to client-side rate control decisions based on network conditions.

17. A video processing system for encoding compressed video streams for multiple bit rate video streaming, the system comprising:

a memory for storing raw video content to be encoded;

a computer processing means for encoding the raw video content using a variable bit rate and dynamic resolution approach in segments comprising groups of pictures as a plurality of compressed video streams, wherein said encoding by the computer processing means comprises;

analyzing the raw video content to determine scene change boundaries in the video content, divide the video content between said scene change boundaries in the segments comprising groups of pictures, and determine a measurement of video complexity of the segments;

dynamically deciding encoding parameters for variable bit rate encoding of the raw video content of each segment into one ore more higher video streams each having a target average bit rate and a peak bit rate constraint that yield a progression of decreasing coded bit rates of the video streams, and for encoding the raw video content of each segment into a bottom video stream at a constant chunk rate, wherein the encoding parameters comprise at least video resolution; and

encoding the higher and bottom video streams using the dynamically decided encoding parameters.

18. The video processing system of claim 17 wherein said dynamically deciding the video resolution for a video stream segment by the computer processing means comprises:

obtaining at least three samples of encoding a segment for varying resolution and quantization step size;

establishing a model of a relation between quantization step size, video resolution and encoded size; and

deciding the video resolution for the segment based on the established model and measurement of video complexity of the segment.

19. The video processing system of claim 18 wherein said dynamically deciding the video resolution by the computer processing means comprises applying decisions to resize video resolution of the segments at scene change boundaries.

20. The video processing system of claim 17 further comprising said computer processing means automatically selecting the peak bit rate and average bit rate for the upper video streams to have a desired numeric distribution between designated maximum and minimum bit rates for a designated number of the video streams.