CONTENT- AND LINK-DEPENDENT CODING ADAPTATION FOR MULTIMEDIA TELEPHONY

• US 20090021572A1
• Filed: 01/10/2008
• Published: 01/22/2009
• Est. Priority Date: 01/10/2007
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Srinivasamurthy, Naveen B., Chung, Hyukjune, Vij, Gajinder, Sagetong, Phoom, Sundarraman, Chandru, Garudadri, Harinath
• Granted Patent: US 8,537,197 B2
• Time in Patent Office: Days
• US Class Current: 348/14.10
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 65/1059 End-user terminal functionalities specially adapted for real-time communication
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04W 72/54 based on quality criteria

Abstract

This disclosure describes techniques that can facilitate multimedia telephony. In one example, a method for communication of multimedia data comprises determining a first level of throughput associated with multimedia data communication from a first access terminal to a network, determining a second level of throughput associated with multimedia data communication from the network to a second access terminal based on feedback from the second access terminal to the first access terminal via the network, determining a budget associated with communication of a video unit of the multimedia data, and coding the video unit of the multimedia data based on the budget and the first and second levels of throughput.

Claims

1. A method for communication of multimedia data, the method comprising:

determining a first level of throughput associated with multimedia data communication from a first access terminal to a network;

determining a second level of throughput associated with multimedia data communication from the network to a second access terminal based on feedback from the second access terminal to the first access terminal via the network;

determining a budget associated with communication of a video unit of the multimedia data; and

coding the video unit of the multimedia data based on the budget and the first and second levels of throughput.

2. The method of claim 1, wherein coding the video unit of the multimedia data based on the budget and the first and second levels of throughput comprises:

coding the video unit to have a size that is a function of the budget and the first and second levels of throughput.

3. The method of claim 1, wherein coding the video unit of the multimedia data based on the budget and the first and second levels of throughput comprises:

coding the video unit according to the budget if the budget is less than the first and second levels of throughput.

4. The method of claim 1, wherein the network comprises

a first access point that sends and receives data to and from the first access terminal;

and

a second access point that sends and receives data to and from the second access terminal, wherein the first and second access points are communicatively coupled to one another in the network.

5. The method of claim 1, further comprising sending the coded video unit of the multimedia data from the first access terminal to the second access terminal via the network.

6. The method of claim 1, further comprising:

maintaining at least two token buckets at the first access terminal, wherein a first token bucket stores information indicative of the first level of throughput and the second token bucket stores information indicative of the second level of throughput,

wherein coding the video unit of the multimedia data comprises coding the video unit with a size that is a function of the budget and occupancies of the first and second token buckets.

7. The method of claim 6, further comprising:

updating the first token bucket to reflect the first level of throughput measured at the first access terminal; and

updating the second token bucket to reflect the second level of throughput measured at the second access terminal.

8. The method of claim 7, wherein updating the first and second token buckets is based on negotiated quality of service (QoS) between the first and second access terminals, current buffer occupancy of the first and second token buckets and elapsed time.

9. The method of claim 6, further comprising:

reducing contents from at least one of the first and second token buckets in the first access terminal upon coding the video unit of multimedia data.

10. The method of claim 1, wherein the video unit of multimedia data comprises a video frame.

11. The method of claim 10, further comprising:

coding audio information associated with the video frame and sending the coded audio information from the first access terminal to the second access terminal over the network regardless of whether the video frame is coded.

12. The method of claim 1, further comprising:

skipping coding of the video unit of multimedia data if the budget is greater than either the first level of throughput or the second level of throughput.

13. The method of claim 12, further comprising upon skipping coding of the video unit of multimedia data:

updating the first level of throughput based on measurements at the first access terminal;

updating the second level of throughput based on measurements at the second access terminal;

re-determining the budget associated with communication of a subsequent video unit of multimedia data; and

coding the subsequent video unit of multimedia data if the re-determined budget is less than the updated first and second levels of throughput.

14. The method of claim 13,

wherein the video unit of multimedia data comprises a video frame,

wherein the first and second access terminals define a frame rate for communication of video frames of the multimedia data during a session, and

wherein upon skipping coding of a given video frame of multimedia data, the updating, re-determining and coding of a subsequent video frame occur prior to a next cycle defined by the frame rate of the session.

15. The method of claim 1, further comprising:

determining a third level of throughput associated with multimedia data communication from the network to a third access terminal based on feedback from the third access terminal to the first access terminal via the network; and

coding the video unit of the multimedia data when the budget is less than the first, second and third levels of throughput.

16. The method of claim 15, further comprising:

maintaining at least three token buckets at the first access terminal, wherein a first token bucket stores information indicative of the first level of throughput, the second token bucket stores information indicative of the second level of throughput, and the third token bucket stores information indicative of the third level of throughput.

17. An apparatus for communication of multimedia data, the apparatus comprising:

a first token buffer module that determines a first level of throughput associated with multimedia data communication from the apparatus to a network;

a second token buffer module that determines a second level of throughput associated with multimedia data communication from the network to an access terminal based on feedback from the access terminal to the apparatus;

a residual module that determines a budget associated with communication of a video unit of the multimedia data; and

a coding module that codes the video unit of the multimedia data based on the budget and the first and second levels of throughput.

18. The apparatus of claim 17, wherein the coding module codes the video unit to have a size that is a function of the budget and the first and second levels of throughput.

19. The apparatus of claim 17, wherein the coding module codes the video unit according to the budget if the budget is less than the first and second levels of throughput.

20. The apparatus of claim 17, wherein the network comprises:

a first access point that sends and receives data to and from the apparatus; and

a second access point that sends and receives data to and from the access terminal, wherein the first and second access points are communicatively coupled to one another in the network.

21. The apparatus of claim 17, further comprising a transmitter module that sends the coded video unit of the multimedia data from the apparatus to the access terminal via the network.

22. The apparatus of claim 17, wherein:

the first token bucket stores information indicative of the first level of throughput, and the second token bucket stores information indicative of the second level of throughput, and

wherein the coding module codes the video unit with a size that is a function of the budget and occupancies of the first and second token buckets.

23. The apparatus of claim 22, wherein the apparatus is configured to:

update the first token bucket to reflect the first level of throughput measured at the apparatus; and

update the second token bucket to reflect the second level of throughput measured at the access terminal.

24. The apparatus of claim 23, wherein updating the first and second token buckets is based on negotiated quality of service (QoS) between the apparatus and the other device, current buffer occupancy of the first and second token buckets and elapsed time.

25. The apparatus of claim 23, wherein:

contents from at least one of the first and second token buckets are reduced in the apparatus upon coding the video unit of multimedia data.

26. The apparatus of claim 17, wherein the apparatus comprises one or more integrated circuits.

27. The apparatus of claim 17, wherein the video unit of multimedia data comprises a video frame.

28. The apparatus of claim 27, wherein:

the coding module codes audio information associated with the video frame and a transmitter sends the coded audio information from the apparatus to the access terminal via the network regardless of whether the video frame is coded.

29. The apparatus of claim 17, wherein the coding module:

skips coding of the video unit of multimedia data if the budget is greater than either the first level of throughput or the second level of throughput.

30. The apparatus of claim 29, wherein upon skipping coding of the video unit of multimedia data:

the first token bucket updates the first level of throughput based on measurements at the apparatus;

the second token bucket updates the second level of throughput based on measurements at the access terminal;

the residual module re-determines the budget associated with communication of a subsequent video unit of multimedia data; and

the coding module codes the subsequent video unit of multimedia data if the re-determined budget is less than the updated first and second levels of throughput.

31. The apparatus of claim 30,

wherein the video unit of multimedia data comprises a video frame,

wherein the apparatus and the access terminal define a frame rate for communication of video frames of the multimedia data during a session, and

wherein upon skipping coding of a given video frame of multimedia data, the updating, re-determining and coding of a subsequent video frame occur prior to a next cycle defined by the frame rate of the session.

32. The apparatus of claim 17, further comprising:

a third token buffer module that determines a third level of throughput associated with multimedia data communication from the network to another access terminal based on feedback from the another terminal to the apparatus,

wherein the coding module codes the video unit of the multimedia data when the budget is less than the first, second and third levels of throughput.

33. The apparatus of claim 32, wherein

the first token bucket stores information indicative of the first level of throughput, the second token bucket stores information indicative of the second level of throughput, and the third token bucket stores information indicative of the third level of throughput.

34. An apparatus for communication of multimedia data, the apparatus comprising:

means for determining a first level of throughput associated with multimedia data communication from the apparatus to a network;

means for determining a second level of throughput associated with multimedia data communication from the network to an access terminal based on feedback from the access terminal to the apparatus via the network;

means for determining a budget associated with communication of a video unit of the multimedia data; and

means for coding the video unit of the multimedia data based on the budget and the first and second levels of throughput.

35. The apparatus of claim 34, wherein means for coding the video unit of the multimedia data based on the budget and the first and second levels of throughput comprises:

means for coding the video unit to have a size that is a function of the budget and the first and second levels of throughput.

36. The apparatus of claim 34, wherein means for coding the video unit of the multimedia data based on the budget and the first and second levels of throughput comprises:

means for coding the video unit according to the budget if the budget is less than the first and second levels of throughput.

37. The apparatus of claim 34, wherein the network comprises:

a first access point that sends and receives data to and from the apparatus; and

a second access point that sends and receives data to and from the access terminal, wherein the first and second access points are communicatively coupled to one another in the network.

38. The apparatus of claim 34, further comprising means for sending the coded video unit of the multimedia data from the apparatus to the access terminal via the network.

39. The apparatus of claim 34, further comprising:

means for maintaining at least two token buckets at the apparatus, wherein a first token bucket stores information indicative of the first level of throughput and the second token bucket stores information indicative of the second level of throughput,

wherein means for coding the video unit of the multimedia data comprises means for coding the video unit with a size that is a function of the budget and occupancies of the first and second token buckets.

40. The apparatus of claim 39, further comprising:

means for updating the first token bucket to reflect the first level of throughput measured at the apparatus; and

means for updating the second token bucket to reflect the second level of throughput measured at the access terminal.

41. The apparatus of claim 40, wherein means for updating the first and second token buckets comprises means for updating based on negotiated quality of service (QoS) between the apparatus and the access terminal, current buffer occupancy of the first and second token buckets and elapsed time.

42. The apparatus of claim 34, wherein the apparatus comprises one or more integrated circuits.

43. The apparatus of claim 39, further comprising:

means for reducing contents from at least one of the first and second token buckets in the apparatus upon coding the video unit of multimedia data.

44. The apparatus of claim 34, wherein the video unit of multimedia data comprises a video frame.

45. The apparatus of claim 44, further comprising:

means for coding audio information associated with the video frame and means for sending the coded audio information from the apparatus to the access terminal over the network regardless of whether the video frame is coded.

46. The apparatus of claim 34, further comprising:

means for skipping coding of the video unit of multimedia data if the budget is greater than either the first level of throughput or the second level of throughput.

47. The apparatus of claim 46, further comprising upon skipping coding of the video unit of multimedia data:

means for updating the first level of throughput based on measurements at the apparatus;

means for updating the second level of throughput based on measurements at the access terminal;

means for re-determining the budget associated with communication of a subsequent video unit of multimedia data; and

means for coding the subsequent video unit of multimedia data if the re-determined budget is less than the updated first and second levels of throughput.

48. The apparatus of claim 47,

wherein the video unit of multimedia data comprises a video frame,

wherein the apparatus and the access terminal define a frame rate for communication of video frames of the multimedia data during a session, and

wherein upon skipping coding of a given video frame of multimedia data, the updating, re-determining and coding of a subsequent video frame occur prior to a next cycle defined by the frame rate of the session.

49. The apparatus of claim 34, further comprising:

means for determining a third level of throughput associated with multimedia data communication from the network to another access terminal based on feedback from the another access terminal to the apparatus via the network; and

means for coding the video unit of the multimedia data when the budget is less than the first, second and third levels of throughput.

50. The apparatus of claim 49, further comprising:

means for maintaining at least three token buckets at the apparatus, wherein a first token bucket stores information indicative of the first level of throughput, the second token bucket stores information indicative of the second level of throughput, and the third token bucket stores information indicative of the third level of throughput.

51. A computer program product including a computer readable medium comprising instructions that are executable to:

determine a first level of throughput associated with multimedia data communication from a first access terminal to a network;

determine a second level of throughput associated with multimedia data communication from the network to a second access terminal based on feedback from the second access terminal to the first access terminal via the network;

determine a budget associated with communication of a video unit of the multimedia data; and

code the video unit of the multimedia data based on the budget and the first and second levels of throughput.

52. An access terminal (AT) comprising:

an antenna that sends and receives wireless communications to and from a network;

a first token buffer module that determines a first level of throughput associated with multimedia data communication from the AT to the network;

a second token buffer module that determines a second level of throughput associated with multimedia data communication from the network to another AT based on feedback from the other AT;

a residual module that determines a budget associated with communication of a video unit of the multimedia data; and

a coding module that codes the video unit of the multimedia data based on the budget and the first and second levels of throughput.

53. A method for communication of multimedia data, the method comprising:

receiving first multimedia data sent from a first access terminal (AT) to a second AT, wherein the first multimedia data is communicated to the second AT from a network;

measuring throughput from the network to the second AT over an observation window; and

sending, from the second AT to the first AT, information indicative of a level of throughput measured from the network to the second AT over the observation window.

54. The method of claim 53, further comprising:

sending second multimedia data from the second AT to the first AT; and

piggybacking the information indicative of the level of throughput with the second multimedia data.

55. The method of claim 53, wherein the multimedia data includes video frames.

56. The method of claim 53, wherein the multimedia data includes audio frames.

57. The method of claim 53, further comprising establishing a multimedia telephony session between the first and second ATs prior to receiving, measuring and sending.

58. The method of claim 53, wherein the first multimedia data is communicated to the second AT from the network via a downlink, and

wherein measuring throughput comprises measuring throughput on the downlink.

59. An apparatus for communication of multimedia data, the apparatus comprising:

a receiver module that receives first multimedia data sent from an access terminal (AT) to the apparatus, wherein the first multimedia data is communicated to the apparatus from a network;

a throughput measuring module to measure throughput from the network to the apparatus over an observation window; and

a transmitter module that sends, from the apparatus to the AT, information indicative of a level of throughput measured from the network to the apparatus over the observation window.

60. The apparatus of claim 59, wherein the transmitter:

sends second multimedia data from the apparatus to the AT, and

piggybacks the information indicative of the level of throughput with the second multimedia data.

61. The apparatus of claim 59, wherein the multimedia data includes video frames.

62. The apparatus of claim 59, wherein the multimedia data includes audio frames.

63. The apparatus of claim 59, wherein the apparatus comprises one or more integrated circuits.

64. The apparatus of claim 59, wherein the first multimedia data is communicated to the apparatus from the network via a downlink, and

wherein the throughput measuring module measures throughput on the downlink.

65. An apparatus for communication of multimedia data, the apparatus comprising:

means for receiving first multimedia data sent from an access terminal (AT) to the apparatus, wherein the first multimedia data is communicated to the apparatus from a network;

means for measuring throughput from the network to the apparatus over an observation window; and

means for sending, from the apparatus to the AT, information indicative of a level of throughput measured from the network to the apparatus over the observation window.

66. The apparatus of claim 65, further comprising:

means for sending second multimedia data from the apparatus to the AT; and

means for piggybacking the information indicative of the level of throughput with the second multimedia data.

67. The apparatus of claim 65, wherein the multimedia data includes video frames.

68. The apparatus of claim 65, wherein the multimedia data includes audio frames.

69. The apparatus of claim 65, further comprising means for establishing a multimedia telephony session between the first and second ATs.

70. The apparatus of claim 65, wherein the first multimedia data is communicated to the second AT from the network via a downlink, and

wherein means for measuring throughput comprises means for measuring throughput on the downlink.

71. A computer program product including a computer readable medium comprising instructions that are executable to:

receive first multimedia data sent from a first access terminal (AT) to a second AT, wherein the first multimedia data is communicated to the second AT from a network;

measure throughput from the network to the second AT over an observation window; and

send, from the second AT to the first AT, information indicative of a level of throughput measured from the network to the second AT over the observation window.

72. An access terminal (AT) comprising:

an antenna that sends and receives wireless communications to and from a network;

a receiver module that receives first multimedia data sent from another AT, wherein the first multimedia data is communicated from the other AT to the network and to the AT from the network;

a throughput measuring module to measure throughput from the network to the AT over an observation window; and

a transmitter module that sends, from the AT to the other AT, information indicative of a level of throughput measured from the network to the AT over the observation window.