Video packet shaping for video telephony

• US 8,102,878 B2
• Filed: 09/29/2005
• Issued: 01/24/2012
• Est. Priority Date: 09/29/2005
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Lee, Yen-Chi
• Primary Examiner(s): Pham, Chi
• Assistant Examiner(s): Faroul, Farah
• Application Number:
  US11/240,133
• Publication Number:
  US 20070071030A1
• Time in Patent Office: 2,308 Days
• Field of Search: 370/468, 370/528, 370/538, 348/385, 348/387
• US Class Current: 370/468
• CPC Class Codes: H04L 1/0007 by modifying the frame length
• CPC Class Codes: H04L 1/0014 by adapting the source coding
• CPC Class Codes: H04L 1/0015 characterised by the adaptation strategy
• CPC Class Codes: H04L 1/1877 for semi-reliable protocols, e.g. for less sensitive applications like streaming video buffer level management for video bitstream control arrangements H04N21/44004
• CPC Class Codes: H04L 2001/0092 Error control systems characterised by the topology of the transmission link
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/2433 Allocation of priorities to traffic types
• CPC Class Codes: H04L 47/365 Dynamic adaptation of the packet size
• CPC Class Codes: H04N 21/23655 Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/2381 Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/4341 Demultiplexing of audio and video streams
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6131 involving transmission via a mobile phone network wireless downlink channel access H04W74/006
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 21/6437 Real-time Transport Protocol [RTP]
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server
• CPC Class Codes: H04N 7/148 : Interfacing a video terminal to a particular transmission medium, e.g. ISDN
• CPC Class Codes: H04W 28/0231 : based on communication conditions dynamic wireless traffic scheduling definition based on channel quality criteria H04W72/54
• CPC Class Codes: H04W 28/06 : Optimizing the usage of the radio link, e.g. header compression, information sizing , discarding information system modifying transmission characteristic according to link quality by modifying frame length H04L1/0007; dynamic adaptation of the packet size for flow control or congestion control H04L47/365
• CPC Class Codes: H04W 72/12 : Wireless traffic scheduling
• CPC Class Codes: H04W 8/04 : Registration at HLR or HSS [Home Subscriber Server]

Abstract

The disclosure relates to techniques for video packet shaping for video telephony (VT). The techniques can be used to prioritize audio packets to reduce audio delay. Channel conditions, excessive video content, or both can cause delays in audio transmission. When reverse link (RL) throughput is reduced, video packet size can overwhelm the RL and increase audio delay. The video packet may consume an excessive number of MAC RLP packets, resulting in delays between successive audio packets. The size of each video packet is adjusted so that audio packets are prioritized for transmission without substantial delay. The video packet size may be controlled based on channel conditions. The audio can be conveyed without substantial delay, even though the video may suffer from delay due to channel conditions. Although video may be compromised by channel conditions, video packet shaping ensures that the VT parties are able to smoothly carry on verbal conversation.

Claims

1. A method comprising:

generating an audio packet by an audio buffer;

estimating throughput of a wireless channel by a video packetizer; and

generating a video packet by the video packetizer with a video packet size determined based on the estimated throughput and a size of the audio packet.

2. The method of claim 1, further comprising placing the audio packet and the video packet in an output queue, wherein the video packet size is determined based on the estimated throughput, the size of the audio packet, and a size of buffered data in the output queue prior to placement of the video packet in the output queue.

3. The method of claim 1, further comprising placing the audio packet and the video packet in an output queue, and generating a medium access control (MAC) layer packet based on at least some of the contents of the output queue.

4. The method of claim 1, further comprising receiving audio frames at first periodic intervals, and generating medium access control (MAC) layer packets at second periodic intervals, wherein generating an audio packet includes generating an audio packet for each of the audio frames, and adding the audio packet to a next available MAC layer packet.

5. The method of claim 1, further comprising receiving audio frames at first periodic intervals, and generating medium access control (MAC) layer packets at second periodic intervals, wherein generating an audio packet includes generating an audio packet that bundles at least two of the audio frames, and adding the audio packet to a next available MAC layer packet.

6. The method of claim 1, further comprising placing the audio packet and the video packet in an output queue, and generating a medium access control (MAC) layer packet based on at least some of the contents of the output queue, wherein the video packet size is based on the size of the audio packet, the estimated throughput, and a size of buffered data in the output queue prior to placement of the video packet in the output queue.

7. The method of claim 1, further comprising estimating the throughout based on one or more of a current wireless channel transmit rate, wireless base station activity, and transmit power limitations.

8. The method of claim 1, wherein the video packet size is sized to avoid substantial delay in transmission of the audio packet over the wireless channel.

9. The method of claim 1, further comprising receiving a plurality of video frames, and adjusting a video encoding rate based on an amount of unpacketized video frame data.

10. The method of claim 1, wherein the video packet size is determined at least in part based on a priority value that assigns a priority to the audio packet over the video packet, and wherein the video packet size for a given estimated throughput increases as the priority value decreases.

11. The method of claim 1, wherein the video packet size is greater than or equal to a minimum video packet size.

12. The method of claim 1, further comprising generating a physical layer packet based on the audio packet and the video packet, and transmitting the physical layer packet to a remote device to support a mobile wireless video telephony conference.

13. The method of claim 12, wherein the physical layer packet conforms to the CDMA2000 1x EV-DO, Release 0, standard.

14. A system comprising:

an audio encoder that generates audio data;

an audio buffer that receives the audio data and outputs an audio packet;

a video encoder that generates video data;

a packetizer that estimates throughput of a wireless channel, and generates a video packet from the video data with a video packet size determined based on the estimated throughput and a size of the audio packet.

15. The system of claim 14, further comprising an output queue that receives the audio packet and the video packet, wherein the video packet size is determined based on the estimated throughput, the size of the audio packet, and a size of buffered data in the output queue prior to placement of the video packet in the output queue.

16. The system of claim 14, further comprising an output queue that receives the audio packet and the video packet, and a medium access control (MAC) layer module that generates a MAC layer packet based on at least some of the contents of the output queue.

17. The system of claim 14, wherein the audio buffer receives audio frames at first periodic intervals, and generates an audio packet for each of the audio frames, and the MAC layer module generates medium access control (MAC) layer packets at second periodic intervals, and adds the audio packet to a next available MAC layer packet.

18. The system of claim 14, wherein the audio buffer receives audio frames at first periodic intervals, and bundles at least two of the audio frames to form the audio packet, and the MAC layer module generates medium access control (MAC) layer packets at second periodic intervals, and adds the audio packet to a next available MAC layer packet.

19. The system of claim 14, wherein the packetizer estimates the throughout based on one or more of a current wireless channel transmit rate, wireless base station activity, and transmit power limitations.

20. The system of claim 14, wherein the video packet size is sized to avoid substantial delay in transmission of the audio packet over the wireless channel.

21. The system of claim 14, wherein the video encoder generates a plurality of video frames, and adjusts a video encoding rate based on an amount of unpacketized video frame data.

22. The system of claim 14, wherein the packetizer determines the video packet size at least in part based on a priority value that assigns a priority to the audio packet over the video packet, wherein the video packet size for a given estimated throughput increases as the priority value decreases.

23. The system of claim 14, wherein the video packet size is greater than or equal to a minimum video packet size.

24. The system of claim 14, further comprising a physical layer module that generates a physical layer packet based on the audio packet and the video packet, and transmits the physical layer packet to a remote device to support a mobile wireless video telephony conference.

25. The system of claim 24, wherein the physical layer packet conforms to the CDMA2000 1x EV-DO, Release 0, standard.

26. A non-transitory computer-readable storage medium comprising computer-executable instructions to cause a processor to:

generate an audio packet;

estimate throughput of a wireless channel; and

generate a video packet with a video packet size determined based on the estimated throughput and a size of the audio packet.

27. The non-transitory computer-readable storage medium of claim 26, further comprising computer-executable instructions to cause a processor to place the audio packet and the video packet in an output queue, wherein the video packet size is determined based on the estimated throughput, the size of the audio packet, and a size of buffered data in the output queue prior to placement of the video packet in the output queue.

28. The non-transitory computer-readable storage medium of claim 26, further comprising computer-executable instructions to cause a processor to place the audio packet and the video packet in an output queue, and generate a medium access control (MAC) layer packet based on at least some of the contents of the output queue.

29. The non-transitory computer-readable storage medium of claim 26, further comprising computer-executable instructions to cause a processor to receive receiving audio frames at first periodic intervals, and generate medium access control (MAC) layer packets at second periodic intervals, wherein generation of an audio packet includes generation of an audio packet for each of the audio frames, and add the audio packet to a next available MAC layer packet.

30. The non-transitory computer-readable storage medium of claim 26, further comprising computer-executable instructions to cause a processor to receive audio frames at first periodic intervals, and generate medium access control (MAC) layer packets at second periodic intervals, wherein generation of an audio packet includes generation of an audio packet that bundles at least two of the audio frames, and add the audio packet to a next available MAC layer packet.

31. The non-transitory computer-readable storage medium of claim 26, further comprising computer-executable instructions to cause the processor to estimate the throughout based on one or more of a current wireless channel transmit rate, wireless base station activity, and transmit power limitations.

32. The non-transitory computer-readable storage medium of claim 26, wherein the video packet size is sized to avoid substantial delay in transmission of the audio packet over the wireless channel.

33. The non-transitory computer-readable storage medium of claim 26, further comprising computer-executable instructions to cause the processor to receive a plurality of video frames, and adjust a video encoding rate based on an amount of unpacketized video frame data.

34. The non-transitory computer-readable storage medium of claim 26, wherein the video packet size is determined at least in part based on a priority value that assigns a priority to the audio packet over the video packet, and wherein the video packet size for a given estimated throughput increases as the priority value decreases.

35. The non-transitory computer-readable storage medium of claim 26, wherein the video packet size is greater than or equal to a minimum video packet size.

36. The non-transitory computer-readable storage medium of claim 26, further comprising computer-executable instructions to cause a processor to generate a physical layer packet based on the audio packet and the video packet, and transmit the physical layer packet to a remote device to support a mobile wireless video telephony conference.

37. The non-transitory computer-readable storage medium of claim 36, wherein the physical layer packet conforms to the CDMA2000 1x EV-DO, Release 0, standard.

38. A system comprising:

means for generating an audio packet;

means for estimating throughput of a wireless channel; and

means for generating a video packet with a video packet size determined based on the estimated throughput and a size of the audio packet.

39. The system of claim 38, further comprising means for placing the audio packet and the video packet in an output queue, wherein the video packet size is determined based on the estimated throughput, the size of the audio packet, and a size of buffered data in the output queue prior to placement of the video packet in the output queue.

40. The system of claim 38, further comprising placing the audio packet and the video packet in an output queue, and generating a medium access control (MAC) layer packet based on at least some of the contents of the output queue.