Enhancement layer switching for scalable video coding

• US 20050117641A1
• Filed: 12/01/2003
• Published: 06/02/2005
• Est. Priority Date: 12/01/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Shipeng, Wu, Feng, Xu, Jizheng
• Granted Patent: US 7,369,610 B2
• Time in Patent Office: Days
• US Class Current: 375/240.80
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 19/37 with arrangements for assigning different transmission priorities to video input data or to video coded data
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/8451 using Advanced Video Coding [AVC]

Abstract

An exemplary system includes a data encoder generating a base layer bitstream encoded at a base bit-rate, and a plurality of enhancement layer bitstreams encoded at different enhancement layer bit-rates, and a bitstream selection module selecting one of the enhancement layer bitstreams every video frame based on available channel bandwidth. A method includes transmitting a first enhancement layer bitstream encoded at a first bit-rate, detecting a transition in network bandwidth through a switching bit-rate, and transmitting a second enhancement layer bitstream encoded at a second bit-rate based on the transition in network bandwidth.

Claims

1. A method comprising:

generating a base layer bitstream and a plurality of independent enhancement layer bitstreams of a first video frame, each bitstream encoded from the first video frame and an associated high-quality reference image having an associated high quality reference bit-rate;

determining a switching bit-rate associated with an available bandwidth of a network; and

selecting a first independent enhancement layer bitstream from the plurality of independent enhancement layer bitstreams of the first video frame based on the switching bit-rate.

2. A method as recited in claim 1 wherein the generating operation comprises concurrently generating the plurality of independent enhancement layer bitstreams.

3. A method as recited in claim 1 further comprising:

generating a plurality of independent enhancement layer bitstreams of a second video frame, each bitstream being generated from the second video frame using the associated high quality reference bit-rates;

selecting a second independent enhancement layer bitstream from the plurality of independent enhancement layer bitstreams of the second video frame; and

generating a difference bitstream representing a difference between the first selected independent enhancement layer bitstream and the second selected independent enhancement layer bitstream.

4. A method as recited in claim 1 wherein the determining operation comprises estimating a switching bit-rate associated with two of the plurality of independent enhancement layer bitstreams, such that distortion values associated with transmission of the two enhancement layer bitstreams are substantially equal.

5. A method as recited in claim 1 wherein the determining operation comprises:

estimating a switching bit-rate between a pair of the high quality reference bit-rates of two of the plurality of independent enhancement layer bitstreams, the estimated switching bit-rate based on distortion values associated with transmission of the two enhancement layer bitstreams; and

adding a buffer bit-rate to the estimated switching bit-rate, the buffer bit-rate corresponding to a level of fluctuation of the network bandwidth.

6. A method as recited in claim 1 wherein the high quality reference bit-rates associated with the enhancement layer bitstreams are successively higher.

7. A method as recited in claim 1 wherein the determining operation comprises computing a switching bit-rate as a function of at least two high quality reference bit-rates associated with two adjacent enhancement layer bitstreams in the plurality of enhancement layer bitstreams.

8. A method as recited in claim 7 wherein the computing operation comprises computing an average bit-rate between the at least two high quality reference bit-rates.

9. A method as recited in claim 1 further comprising encoding the base layer bitstream and the plurality of enhancement layer bitstreams according to an H.26 video standard.

10. A method as recited in claim 1 further comprising transmitting the base layer bitstream and the first independent enhancement layer bitstreams.

11. A method as recited in claim 1 further comprising:

receiving the base layer bitstream and the first independent enhancement layer bitstream; and

decoding the base layer bitstream and the first independent enhancement layer bitstream to display the first video frame on a display device.

12. A method as recited in claim 1 wherein the generating operation further comprises:

generating the base layer bitstream by applying motion estimation to the first video frame and a plurality of high-quality reference images having associated high quality reference bit-rates.

13. A computer-readable medium having stored thereon computer-executable instruction for performing a method comprising:

in a first video frame, transmitting a first enhancement layer bitstream encoded at a first high quality reference bit-rate (HQRB);

detecting a change in network bandwidth from a first bandwidth that is less than a predetermined switching bit-rate to a second bandwidth that is greater than the predetermined switching bit-rate; and

in response to the detecting, switching up to a second enhancement layer bitstream in a second video frame, the second enhancement layer bitstream encoded at a second high quality reference bit-rate (HQRB), the second HQRB being greater than the first HQRB.

14. A computer-readable medium as recited in claim 13, the method further comprising:

in the second video frame, transmitting an enhancement layer difference bitstream representing the difference between the first enhancement layer bitstream and the second enhancement layer bitstream.

15. A computer-readable medium as recited in claim 13, the method further comprising:

transmitting a base layer bitstream in both the first video frame and the second video frame, the base layer bitstream encoded at a base bit-rate that is less than the first HQRB, the base layer bitstream serving as a reference for the first enhancement layer bitstream and the second enhancement layer bitstream.

16. A computer-readable medium as recited in claim 13, the method further comprising:

in a third video frame, switching down to the first enhancement layer bitstream in response to detecting a change in network bandwidth from the second bandwidth to the first bandwidth.

17. A computer-readable medium as recited in claim 13, further comprising encoding the first enhancement layer bitstream and the second enhancement layer bitstream according to a Motion Picture Experts Group (MPEG) video encoding standard.

18. A computer-readable medium as recited in claim 13, wherein the predetermined switching bit-rate is a function of the first HQRB and the second HQRB.

19. A computer-readable medium as recited in claim 13, wherein the predetermined switching bit-rate is a bit-rate at which a distortion value associated with transmission of the first enhancement layer bitstream is substantially equal to a distortion value associated with transmission of the second enhancement layer bitstream.

20. A computer-readable medium as recited in claim 15 further comprising applying motion estimation to at least one of a high quality reference of the first enhancement layer bitstream and a high quality reference of the second high quality reference bitstream to generate the base layer bitstream.

21. A system comprising:

a data encoder generating a base layer bitstream encoded at a base bit-rate, and a plurality of enhancement layer bitstreams, each enhancement layer bitstream encoded at a different enhancement layer bit-rate; and

an enhancement layer bitstream selection module selecting one of the enhancement layer bitstreams in each of a plurality of video frames based on a switching bit-rate, the switching bit-rate being a function of two of the enhancement layer bit-rates.

22. A system as recited in claim 21 wherein each of the enhancement layer bitstreams is independently based on the base layer bitstream.

23. A system as recited in claim 21 wherein the data encoder comprises:

a base layer bitstream encoder receiving video data and generating the base layer bitstream; and

a plurality of enhancement layer bitstream encoders, each enhancement layer bitstream encoder receiving the base layer bitstream and the video data and concurrently generating an associated one of the plurality of enhancement layer bitstreams representing a successively higher level of resolution from the base layer bitstream.

24. A system as recited in claim 21 further comprising a difference layer bitstream encoder generating a difference layer bitstream representing a difference between a first selected one of the plurality of enhancement layer bitstreams transmitted in a first video frame and a second selected one of the plurality of enhancement layer bitstreams transmitted in a second video frame.

25. A system as recited in claim 24 wherein the difference layer bitstream is transmitted between the first selected one of the plurality of enhancement layer bitstreams and the second selected one of the plurality of enhancement layer bitstreams.

26. A system as recited in claim 23 wherein the base layer bitstream encoder comprises a motion estimator generating a base layer bitstream based on at least one high quality reference bitstream and a current video frame.

27. A method comprising:

in a first video frame, receiving a first enhancement layer bitstream encoded at a first high quality reference bit-rate (HQRB);

in a second video frame, receiving a second enhancement layer bitstream encoded at a second high quality reference bit-rate (HQRB), the second HQRB being greater than the first HQRB;

receiving a difference bitstream representing a difference between the first enhancement layer bitstream and the second enhancement layer bitstream; and

decoding the second enhancement layer bitstream based on the first enhancement layer bitstream and the difference bitstream.

28. A method comprising:

encoding a plurality of video frames from a video frame sequence into a base layer and a plurality of enhancement layers;

for each video frame, selecting one of the enhancement layers to transmit based on a relationship between a switching bit-rate and high quality reference bit-rates associated with the enhancement layers and a detected network bandwidth; and

transmitting a difference bitstream if a currently selected one of the enhancement layers is different from a previously selected one of the enhancement layers, the difference bitstream comprising an encoded difference between the currently selected enhancement layer and the previously selected enhancement layer.

29. A method as recited in claim 28 further comprising:

estimating distortions associated with two of the plurality of enhancement layers;

choosing the switching bit-rate such that the estimated distortions are substantially equal.

30. A method as recited in claim 28 further comprising:

estimating distortions associated with two of the plurality of enhancement layers;

choosing the switching bit-rate based on the estimated distortions are substantially equal;

adjusting the switching bit-rate based on a buffer bit-rate.

31. A method as recited in claim 28 further comprising truncating the difference bitstream prior to transmission.