Video rate adaptation to reverse link conditions

• US 20090034610A1
• Filed: 05/31/2006
• Published: 02/05/2009
• Est. Priority Date: 10/21/2005
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Lee, Yen-Chi, Tinnakornsrisuphap, Peerapol, Gupta, Vikram, Lott, Christopher Gerard
• Granted Patent: US 8,406,309 B2
• Time in Patent Office: Days
• US Class Current: 375/240.20
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/169 characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/188 the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/194 involving only two passes
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/23805 Controlling the feeding rate to the network, e.g. by controlling the video pump
• CPC Class Codes: H04N 21/2381 Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/2662 : Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4341 : Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/44004 : involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6131 : involving transmission via a mobile phone network wireless downlink channel access H04W74/006
• CPC Class Codes: H04N 21/6175 : involving transmission via Internet broadcast-related systems characterised by the transmission system being the Internet H04H60/82
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/64322 : IP
• CPC Class Codes: H04N 21/6437 : Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/64769 : for rate control flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/64792 : Controlling the complexity of the content stream, e.g. by dropping packets intermediate media network packet handling H04L65/765; proxy provisioning conversion or adaptation for reducing the amount or size of exchanged application data H04L67/5651; negotiation of resources in wireless networks H04W28/16
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04W 52/04 : TPC

Abstract

The disclosure relates to video rate adaptation techniques that may use information from a medium access control (MAC) layer and radio link protocol (RLP) layer. The techniques may greatly reduce video delay by adjusting video encoding rate. For real-time video telephony (VT) applications, these techniques may provide graceful quality degradation and improve user experience, especially when the channel conditions degrade.

Claims

1. A method comprising:

estimating video throughput based on a size of a video flow radio link protocol (RLP) queue at an access terminal; and

encoding video data using the estimated video throughput.

2. A method comprising:

determining a first size V_n of a video queue in a radio link protocol (RLP) layer at a first time tn based on a video frame rate;

determining a second size V_m of the video queue at a second time t_m based on an audio frame rate;

if the first size V_n or the second size V_m is greater than zero, then using the first size Vn, a previous size V_n-1 of the video queue associated with a previous video frame, a previous video frame size B_n-1, the first time t_n, and a time t_n-1 associated with the previous size of the video queue to determine a video throughput VTP;

if the first size V_n and the second size V_m are equal to zero, then searching for an earlier time based on the audio frame rate when the video queue size was greater than zero;

after finding the earlier time based on the audio frame rate when the video queue size was greater than zero, using an earlier queue size V_m-i based on the audio frame rate, the previous size V_n-1 of the video queue associated with the previous video frame, the previous video frame size B_n-1, the earlier time t_m-1, and the time t_n-1 associated with the previous size of the video queue to determine a video throughput VTP;

using the determined video throughput VTP to determine a channel-constrained video frame size; and

using the channel-constrained video frame size to control a video encoding rate.

3. The method of claim 2, wherein using the first size (V_x), a previous size (V_n-1) of the video queue associated with a previous video frame, a previous video frame size (B_n-1), the first time (t_x), and a time (t_n-1) associated with the previous size of the video queue to determine a video throughput (VTP) comprises:

$V<br><br><br><br>T<br><br><br><br>P=\frac{B_{n-1}+V_{n-1}-V_x}{t_x-t_{n-1}},$

4. The method of claim 2, wherein using the earlier size (V_x), the previous size (V_n-1) of the video queue associated with the previous video frame, the previous video frame size (B_n-1), the earlier time (t_x), and the time (t_n-1) associated with the previous size of the video queue to determine a video throughput (VTP) comprises:

$V<br><br><br><br>T<br><br><br><br>P=\frac{B_{n-1}+V_{n-1}-V_x}{t_x-t_{n-1}},$

5. The method of claim 2, further comprising if the first size and the second size are equal to zero, and a third size of the video queue associated with a previous audio frame is greater than zero, then using the third size, the previous size of the video queue associated with a previous video frame, the previous video frame size, a time associated with the third size, and the time associated with the previous size of the video queue to determine a video throughput.

6. The method of claim 2, wherein the video queue is assigned to only store video packets.

7. The method of claim 2, wherein using the determined video throughput to determine a channel-constrained video frame size further uses the video frame rate, the first size and an adjustment factor, which controls how much the video encoding rate reacts to channel conditions.

8. The method of claim 7, wherein the adjustment factor has a first value for video conferencing and a second value for video sharing.

9. The method of claim 2, further comprising:

using the video encoding rate to encode video; and

transmitting the encoded video across a wireless channel using Code Division Multiple Access Evolution Data Optimized Revision A (CDMA EV-DO Rev A) protocols.

10. The method of claim 2, further comprising:

determining whether the channel-constrained video frame size is greater than a minimum frame size threshold;

if the channel-constrained video frame size is greater than the minimum frame size threshold, determining whether the channel-constrained video frame size is smaller than a frame size constrained by a virtual buffer;

if the channel-constrained video frame size is smaller than the frame size constrained by the virtual buffer, then using the channel-constrained video frame size to control the video encoding rate; and

if the channel-constrained video frame size is greater than the frame size constrained by the virtual buffer, then using the frame size constrained by the virtual buffer to control the video encoding rate.

11. A method comprising:

determining a size of a video queue in a radio link protocol (RLP) layer;

determining a power headroom limitation from a medium access control (MAC) layer;

using the determined power headroom limitation to determine a MAC payload size;

using the determined MAC payload size and an estimate of how many transmission opportunities are given to video in a time period to determine video throughput;

using the determined video throughput and the determined size of the video queue in the RLP layer to determine a channel-constrained video frame size; and

using the channel-constrained video frame size to control a video encoding rate.

12. The method of claim 11, wherein using the determined video throughput and the determined size of the video queue in the RLP layer to determine a channel-constrained video frame size further uses an adjustment factor, which controls how much the video encoding rate reacts to channel conditions.

13. The method of claim 12, wherein the adjustment factor has a first value for video conferencing and a second value for video sharing.

14. The method of claim 11, further comprising:

determining whether the channel-constrained video frame size is greater than a minimum frame size threshold;

if the channel-constrained video frame size is greater than the minimum frame size threshold, determining whether the channel-constrained video frame size is smaller than a frame size constrained by a virtual buffer;

if the channel-constrained video frame size is smaller than the frame size constrained by the virtual buffer, then using the channel-constrained video frame size to control the video encoding rate; and

if the channel-constrained video frame size is greater than the frame size constrained by the virtual buffer, then using the frame size constrained by the virtual buffer to control the video encoding rate.

15. The method of claim 11, further comprising:

using the video encoding rate to encode video; and

transmitting the encoded video across a wireless channel using Code Division Multiple Access Evolution Data Optimized Revision A (CDMA EV-DO Rev A) protocols.

16. The method of claim 11, wherein the video queue is assigned to only store video packets.

17. An apparatus comprising a machine-readable memory storing a set of instructions configured to:

determine a first size V_n of a video queue in a radio link protocol (RLP) layer at a first time tn based on a video frame rate;

determine a second size V_m of the video queue at a second time t_m based on an audio frame rate;

if the first size V_n or the second size V_m is greater than zero, then use the first size V_n, a previous size V_n-1 of the video queue associated with a previous video frame, a previous video frame size B_n-1, the first time tn, and a time t_n-1 associated with the previous size of the video queue to determine a video throughput VTP;

if the first size V_n and the second size V_m are equal to zero, then searching for an earlier time based on the audio frame rate when the video queue size was greater than zero;

after finding the earlier time based on the audio frame rate when the video queue size was greater than zero, use an earlier queue size V_m-i based on the audio frame rate, the previous size V_n-1 of the video queue associated with the previous video frame, the previous video frame size B_n-1, the earlier time t_m-1, and the time t_n-1 associated with the previous size of the video queue to determine a video throughput VTP;

use the determined video throughput VTP to determine a channel-constrained video frame size; and

use the channel-constrained video frame size to control a video encoding rate.

18. The apparatus of claim 17, wherein instructions operate on an application layer above the RLP layer.

19. The apparatus of claim 17, wherein a medium access control (MAC) layer for video flows of the apparatus does not react to sector loading.

20. The apparatus of claim 17, wherein using the first size (V_x), a previous size (V_n-1) of the video queue associated with a previous video frame, a previous video frame size (B_n-1), the first time (t_x), and a time (t_n-1) associated with the previous size of the video queue to determine a video throughput (VTP) comprises:

$V<br><br><br><br>T<br><br><br><br>P=\frac{B_{n-1}+V_{n-1}-V_x}{t_x-t_{n-1}},$

21. The apparatus of claim 17, wherein using the earlier size (V_x), the previous size (V_n-1) of the video queue associated with the previous video frame, the previous video frame size (B_n-1), the earlier time (t_x), and the time (t_n-1) associated with the previous size of the video queue to determine a video throughput (VTP) comprises:

$V<br><br><br><br>T<br><br><br><br>P=\frac{B_{n-1}+V_{n-1}-V_x}{t_x-t_{n-1}},$

22. The apparatus of claim 17, wherein the instructions further comprise if the first size and the second size are equal to zero, and a third size of the video queue associated with a previous audio frame is greater than zero, then using the third size, the previous size of the video queue associated with a previous video frame, the previous video frame size, a time associated with the third size, and the time associated with the previous size of the video queue to determine a video throughput.

23. The apparatus of claim 17, wherein the video queue is assigned to only store video packets.

24. The apparatus of claim 17, wherein using the determined video throughput to determine a channel-constrained video frame size further uses the video frame rate, the first size and an adjustment factor, which controls how much the video encoding rate reacts to channel conditions.

25. The apparatus of claim 24, wherein the adjustment factor has a first value for video conferencing and a second value for video sharing.

26. The apparatus of claim 17, further comprising:

a video encoder configured to use the video encoding rate to encode video; and

a transmitter configured to transmit the encoded video across a wireless channel using Code Division Multiple Access Evolution Data Optimized Revision A (CDMA EV-DO Rev A) protocols.

27. The apparatus of claim 17, wherein the instructions further comprise:

determining whether the channel-constrained video frame size is greater than a minimum frame size threshold;

if the channel-constrained video frame size is greater than the minimum frame size threshold, determining whether the channel-constrained video frame size is smaller than a frame size constrained by a virtual buffer;

if the channel-constrained video frame size is smaller than the frame size constrained by the virtual buffer, then using the channel-constrained video frame size to control the video encoding rate; and

if the channel-constrained video frame size is greater than the frame size constrained by the virtual buffer, then using the frame size constrained by the virtual buffer to control the video encoding rate.

28. An apparatus comprising a machine-readable memory storing a set of instructions configured to:

determine a size of a video queue in a radio link protocol (RLP) layer;

determine a power headroom limitation from a medium access control (MAC) layer;

use the determined power headroom limitation to determine a MAC payload size;

use the determined MAC payload size and an estimate of how many transmission opportunities are given to video in a time period to determine video throughput;

use the determined video throughput and the determined size of the video queue in the RLP layer to determine a channel-constrained video frame size; and

use the channel-constrained video frame size to control a video encoding rate.

29. The apparatus of claim 28, wherein using the determined video throughput and the determined size of the video queue in the RLP layer to determine a channel-constrained video frame size further uses an adjustment factor, which controls how much the video encoding rate reacts to channel conditions.

30. The apparatus of claim 28, wherein the instructions further comprise:

determining whether the channel-constrained video frame size is greater than a minimum frame size threshold;

if the channel-constrained video frame size is greater than the minimum frame size threshold, determining whether the channel-constrained video frame size is smaller than a frame size constrained by a virtual buffer;

if the channel-constrained video frame size is smaller than the frame size constrained by the virtual buffer, then using the channel-constrained video frame size to control the video encoding rate; and

if the channel-constrained video frame size is greater than the frame size constrained by the virtual buffer, then using the frame size constrained by the virtual buffer to control the video encoding rate.

31. An apparatus comprising:

a radio link protocol (RLP) layer queue configured to store video data;

a first unit configured to receive a size of the RLP video queue and a power headroom limitation from a medium access control (MAC) layer, use the power headroom limitation to determine a MAC payload size, use the determined MAC payload size and an estimate of how many transmission opportunities are given to video in a time period to determine video throughput, and use the determined video throughput and the size of the video queue in the RLP layer to determine a channel-constrained video frame size;

a second unit to use the channel-constrained video frame size to control a video encoding rate; and

a video encoder to use the video encoding rate to encode video.

32. The apparatus of claim 31, further comprising:

a virtual buffer; and

a third unit configured to;

determine whether the channel-constrained video frame size is greater than a minimum frame size threshold;

if the channel-constrained video frame size is greater than the minimum frame size threshold, determine whether the channel-constrained video frame size is smaller than a frame size constrained by a virtual buffer;

if the channel-constrained video frame size is smaller than the frame size constrained by the virtual buffer, then use the channel-constrained video frame size to control the video encoding rate; and

if the channel-constrained video frame size is greater than the frame size constrained by the virtual buffer, then use the frame size constrained by the virtual buffer to control the video encoding rate.

33. An apparatus comprising:

means to determine a size of a video queue in a radio link protocol (RLP) layer;

means to determine a power headroom limitation from a medium access control (MAC) layer;

means to use the determined power headroom limitation to determine a MAC payload size;

means to use the determined MAC payload size and an estimate of how many transmission opportunities are given to video in a time period to determine video throughput;

means to use the determined video throughput and the determined size of the video queue in the RLP layer to determine a channel-constrained video frame size; and

means to use the channel-constrained video frame size to control a video encoding rate.