Packet prioritization and associated bandwidth and buffer management techniques for audio over IP

• US 20040073692A1
• Filed: 09/30/2002
• Published: 04/15/2004
• Est. Priority Date: 09/30/2002
• Status: Active Grant

• Current Assignee: Arlington Technologies, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Avaya Incorporated
• Inventors: Gentle, Christopher R., Michaelis, Paul Roller
• Granted Patent: US 7,359,979 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 47/2408 for supporting different services, e.g. a differentiated services [DiffServ] type of service
• CPC Class Codes: H04L 47/2416 Real-time traffic
• CPC Class Codes: H04L 47/2433 Allocation of priorities to traffic types
• CPC Class Codes: H04L 47/283 in response to processing delays, e.g. caused by jitter or round trip time [RTT]
• CPC Class Codes: H04L 47/31 by tagging of packets, e.g. using discard eligibility [DE] bits
• CPC Class Codes: H04L 47/32 by discarding or delaying data units, e.g. packets or frames
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/70 Media network packetisation

Abstract

The present invention is directed to voice communication devices in which an audio stream is divided into a sequence of individual packets, each of which is routed via pathways that can vary depending on the availability of network resources. All embodiments of the invention rely on an acoustic prioritization agent that assigns a priority value to the packets. The priority value is based on factors such as whether the packet contains voice activity and the degree of acoustic similarity between this packet and adjacent packets in the sequence. A confidence level, associated with the priority value, may also be assigned. In one embodiment, network congestion is reduced by deliberately failing to transmit packets that are judged to be acoustically similar to adjacent packets; the expectation is that, under these circumstances, traditional packet loss concealment algorithms in the receiving device will construct an acceptably accurate replica of the missing packet. In another embodiment, the receiving device can reduce the number of packets stored in its jitter buffer, and therefore the latency of the speech signal, by selectively deleting one or more packets within sustained silences or non-varying speech events. In both embodiments, the ability of the system to drop appropriate packets may be enhanced by taking into account the confidence levels associated with the priority assessments.

Claims

1. A method for processing voice communications over a data network, comprising:

(a) receiving a voice stream from a user, the voice stream comprising a plurality of temporally distinct segments;

(b) processing at least one selected first segment of the voice stream, wherein the processing step comprises at least one of the following substeps;

(i) determining whether or not the contents of the selected first segment are the product of voice activity and, when the contents are determined not to be the product of voice activity, a level of confidence that the voice activity determination is accurate;

(ii) determining a type of voice activity associated with the contents of the first segment; and

(iii) comparing the first segment with a second segment of the voice stream to determine a degree of acoustic similarity between the first and second segments, wherein the processing of the first segment is based on at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity.

2. The method of claim 1, further comprising:

(c) based on the at least one of the level of confidence, type of voice activity and the degree of acoustic similarity, assigning an importance to the first segment.

3. The method of claim 2, wherein the importance is a value marker and further comprising:

incorporating the value marker into a first packet comprising the first segment.

4. The method of claim 2, wherein the importance is a service class assigned to a first packet comprising the first segment.

5. The method of claim 2, wherein the importance is a transmission priority assigned to a first packet comprising the first segment.

6. The method of claim 1, wherein in the processing step a first packet comprising the first segment is not transmitted when the at least one of the level of confidence and the degree of acoustic similarity is one of less than and greater than a predetermined threshold.

7. The method of claim 6, further comprising:

varying the predetermined threshold based on at least one of jitter, latency, a number of missing packets, a number of packets received out-of-order, a processing delay, a propagation delay, a receive buffer delay, and a number of packets enqueued in a receive buffer.

8. The method of claim 1, wherein the processing step comprises substep (i).

9. The method of claim 1, wherein the processing step comprises substep (iii).

10. The method of claim 9, wherein the second segment temporally precedes the first segment and a third segment temporally follows the first segment and wherein substep (ii) comprises:

comparing the first segment with the second segment of the voice stream to determine a first degree of acoustic similarity between the first and second segments; and

comparing the first segment with the third segment of the voice stream to determine a second degree of acoustic similarity between the first and third segments.

11. The method of claim 10, wherein the processing step is based on at least one of the first and second degrees of acoustic similarity one of exceeding or being less than a selected similarity threshold.

12. The method of claim 1, wherein the first segment corresponds to a payload of a first packet.

13. The method of claim 1, wherein the first segment corresponds to a frame of a first packet.

14. The method of claim 1, wherein different classes of services are used for different segments of the voice stream.

15. The method of claim 1, wherein different transmission priorities are used for different segments of the voice stream.

16. The method of claim 1, wherein the processing step comprises substep (ii).

17. The method of claim 16, wherein the type of voice activity is a plosive.

18. The method of claim 9, wherein a first packet associated with the first segment is not transmitted and further comprising:

later reconstructing the first segment with a packet loss concealment algorithm.

19. The method of claim 3, further comprising:

when the value of the value marker is one of less than and greater than a predetermined value threshold, removing the first packet from a receive buffer.

20. A computer readable medium containing instructions to perform the steps of claim 1.

21. a logic circuit configured to perform the steps of claim 1.

22. A method for managing a receive buffer, comprising:

providing a receive buffer, the receive buffer containing a plurality of packets associated with voice communications; and

based on a level of importance associated with at least some of the plurality of packets, removing at least some of the packets from the receive buffer while leaving other packets in the receive buffer.

23. The method of claim 22, wherein the level of importance associated with each packet is indicated by a corresponding value marker.

24. The method of claim 22, further comprising:

determining when at least one of a delay associated with the receive buffer and a length of the receive buffer exceeds a predetermined level;

when the at least one of a delay and length exceeds the predetermined level, performing the removing step; and

when the at least one of a delay and length does not exceed the predetermined level, not performing the removing step.

25. The method of claim 22, further comprising:

for at least some packets remaining in the receive buffer, resetting a packet counter indicating an ordering of the packets.

26. The method of claim 22, further comprising:

assigning a packet counter of a removed packet to a packet remaining in the receive buffer.

27. The method of claim 22, further comprising:

performing time compression around at least one of the removed packets.

28. The method of claim 22 further comprising before the removing step:

receiving a voice stream from a user, the voice stream comprising a plurality of temporally distinct segments;

processing at least one selected first segment of the voice stream, wherein the processing step comprises at least one of the following substeps;

determining whether or not the contents of the selected first segment are the product of voice activity and, when the contents are determined to be a product of voice activity, a level of confidence that the voice activity determination is accurate;

determining a type of voice activity associated with the contents of the first segment; and

comparing the first segment with a second segment of the voice stream to determine a degree of acoustic similarity between the first and second segments, wherein the processing of the first segment is based on at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity.

29. The method of claim 28, further comprising:

based on the at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity, assigning an importance to the first segment.

30. The method of claim 29, wherein the importance is a value marker and further comprising:

incorporating the value marker into a first packet comprising the first segment.

31. The method of claim 29, wherein the importance is a service class assigned to a first packet comprising the first segment.

32. The method of claim 29, wherein the importance is a transmission priority assigned to a first packet comprising the first segment.

33. A computer readable medium to perform the steps of claim 22.

34. A logic circuit configured to perform the steps of claim 22.

35. A system for transmitting voice communications over a data network, comprising:

(a) an input operable to receive a voice stream from a user, the voice stream comprising a plurality of temporally distinct segments;

(b) a packet protocol interface operable to convert at least one selected first segment of the voice stream into at least a first packet; and

(c) an acoustic prioritization agent operable to control processing of at least one of the first segment and the at least a first packet based on at least one of a level of confidence that the contents of the selected first segment are not the product of voice activity, a type of voice activity associated with the contents of the first segment, and a degree of acoustic similarity between the first segment and a second segment of the voice stream.

36. The system of claim 35, wherein the acoustic prioritization agent is operable to assign an importance to the at least one of the first segment and the at least a first packet based on the at least one of the level of confidence, type of voice activity and the degree of acoustic similarity.

37. The system of claim 36, wherein the importance is a value marker and the packet protocol interface is operable to incorporate the value marker into the at least a first packet.

38. The system of claim 36, wherein the importance is a service class assigned to the at least a first packet.

39. The system of claim 36, wherein the importance is a transmission priority assigned to the at least a first packet.

40. The system of claim 35, wherein the packet protocol interface is operable to not transmit the at least a first packet when the at least one of the level of confidence and the degree of acoustic similarity is one of less than and greater than a predetermined threshold.

41. The system of claim 40, wherein the predetermined threshold is varied based on at least one of jitter, latency, a number of missing packets, a number of packets received out-of-order, a processing delay, a propagation delay, and a receive buffer length.

42. The system of claim 35, wherein the at least one of the level of confidence, type of voice activity and the degree of acoustic similarity is the level of confidence.

43. The system of claim 35, wherein the at least one of the level of confidence, type of voice activity and the degree of acoustic similarity is the degree of acoustic similarity.

44. The system of claim 43, wherein the second segment temporally precedes the first segment and a third segment of the voice stream temporally follows the first segment and further comprising:

a codec operable to compare the first segment with the second segment of the voice stream to determine a first degree of acoustic similarity between the first and second segments and compare the first segment with the third segment of the voice stream to determine a second degree of acoustic similarity between the first and third segments.

45. The system of claim 44, wherein the prioritization agent controls processing of the at least one of the first segment and the at least a first packet based on at least one of the first and second degrees of acoustic similarity one of exceeding or being less than a selected similarity threshold.

46. The system of claim 35, wherein the first segment corresponds to a payload of the at least a first packet.

47. The system of claim 35, wherein the first segment corresponds to a frame of the at least a first packet.

48. The system of claim 35, wherein different classes of services are used for different segments of the voice stream.

49. The system of claim 35, wherein the packet protocol interface is operable to use different transmission priorities for different segments of the voice stream.

50. The system of claim 35, wherein the at least one of the level of confidence, type of voice activity and the degree of acoustic similarity is the type of voice activity.

51. The system of claim 50, wherein the type of voice activity is a plosive.

52. The system of claim 43, wherein the at least a packet is not transmitted and further comprising:

a packet loss concealment agent operable to later reconstructing the first segment.

53. The system of claim 37, further comprising:

a buffer manager operable to remove the at least a first packet from a receive buffer when the value of the value marker is one of less than and greater than a predetermined value threshold.

54. A system for managing a receive buffer, comprising:

a receive buffer containing a plurality of packets associated with voice communications; and

a buffer manager operable to remove at least some of the packets from the receive buffer while leaving other packets in the receive buffer based on a level of importance associated with at least some of the plurality of packets.

55. The system of claim 54, wherein the level of importance of the at least some packets is indicated by a corresponding value marker.

56. The system of claim 54, wherein the buffer manager is further operable to:

determine if at least one of a delay associated with the receive buffer and a number of packets enqueued in the receive buffer exceeds a predetermined level;

when the at least one of a delay and number of packets exceeds the predetermined level, performing the removing step; and

when the at least one of a delay and number of packets does not exceed the predetermined level, not performing the removing step.

57. The system of claim 54, wherein the buffer manager is operable, for at least some packets remaining in the receive buffer, to reset a packet counter indicating an ordering of the packets.

58. The system of claim 54, wherein the buffer manager is operable to assign a packet counter of a removed packet to a packet remaining in the receive buffer.

59. The system of claim 54, wherein the buffer manager is operable to perform time compression around at least one of the removed packets.

60. A packet, comprising:

a packet header comprising transmission information and

a payload comprising one or more frames of a voice stream, wherein at least one of the packet header and payload comprises a value of the value marker is indicative of a level of importance of the payload to maintaining a selected quality of voice communication.

61. The packet of claim 60, wherein the transmission information comprises the value marker.

62. The packet of claim 60, wherein the payload comprises the value marker.

63. A method for processing voice communications over a data network, comprising:

(a) receiving a first voice stream from a first user, the voice stream comprising a plurality of temporally distinct segments associated with a plurality of packets and the voice stream being a part of a session between at least the first user and a second user, wherein the session has an associated at least one of a a jitter value, a latency value, a number of missing packets, a number of packets received out-of-order, a processing delay, a propagation delay, a receive buffer delay, and a number of packets enqueued in a receive buffer and

(b) comparing the at least one of a jitter value, a latency value, a number of missing packets, a number of packets received out-of-order, a processing delay, a propagation delay, a receive buffer delay, and a number of packets enqueued in a receive buffer with a predetermined threshold;

(i) when the at least one of a jitter value, a latency value, a number of missing packets, a number of packets received out-of-order, a processing delay, a propagation delay, a receive buffer delay, and a number of packets enqueued in a receive buffer exceeds the predetermined threshold, not transmitting at least some of the plurality of packets and

(ii) when the at least one of a jitter value, a latency value, a number of missing packets, a number of packets received out-of-order, a processing delay, a propagation delay, a receive buffer delay, and a number of packets enqueued in a receive buffer is less than the predetermined threshold, transmitting the at least some of the plurality of packets.

64. The method of claim 63, further comprising at least one of the following steps:

(c) determining whether or not the contents of a selected first segment of the plurality of temporally distinct segments are the product of voice activity and, when the contents are determined not to be the product of voice activity, a level of confidence that the voice activity determination is accurate;

(d) determining a type of voice activity associated with the contents of the first segment; and

(e) comparing the first segment with a second segment of the plurality of temporally distinct segments to determine a degree of acoustic similarity between the first and second segments, wherein the processing of the first segment is based on at least one of the level of confidence, the type of voice activity, and the degree of acoustic similarity.

65. The method of claim 64, further comprising:

(c) based on the at least one of the level of confidence, type of voice activity and the degree of acoustic similarity, assigning an importance to the first segment.

66. The method of claim 64, wherein the importance is a value marker and further comprising:

incorporating the value marker into a first packet comprising the first segment.

67. The method of claim 64, wherein the importance is a service class assigned to a first packet comprising the first segment.

68. The method of claim 64, wherein the importance is a transmission priority assigned to a first packet comprising the first segment.

69. The method of claim 63, wherein in the processing step a first packet comprising the first segment is not transmitted when the at least one of the level of confidence and the degree of acoustic similarity is one of less than and greater than a predetermined threshold.

70. The method of claim 64, wherein step (c) is performed.

71. The method of claim 64, wherein step (d) is performed.

72. The method of claim 64, wherein step (d) is performed.

73. The method of claim 64, wherein the type of voice activity is a plosive.

74. The method of claim 69, wherein a first packet associated with the first segment is not transmitted and further comprising:

(f) later reconstructing the first segment with a packet loss concealment algorithm.