Adaptive jitter buffer-packet loss concealment

• US 8,045,571 B1
• Filed: 05/15/2008
• Issued: 10/25/2011
• Est. Priority Date: 02/12/2007
• Status: Expired due to Fees

• Current Assignee: Marvell Asia Pte Limited (Marvell Technology Group Limited)
• Original Assignee: Marvell International Limited (Marvell Technology Group Limited)
• Inventors: Xu, Beryl, Li, Hongxin
• Primary Examiner(s): Yao; Kwang B
• Assistant Examiner(s): Skripnikov; Alex
• Application Number:
  US12/152,531
• Time in Patent Office: 1,258 Days
• Field of Search: None
• US Class Current: 370/419
• CPC Class Codes: G10L 19/005 Correction of errors induced by the transmission channel, if related to the coding algorithm
• CPC Class Codes: G10L 19/167 Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes

Abstract

An audio decoding system comprises a buffer module that receives packets including encoded audio frames that each store audio parameters. A packet loss concealment module that selectively extracts the audio parameters from ones of the encoded audio frames, determines recovered audio parameters based on the extracted audio parameters, and encodes the recovered audio parameters into recovered audio frames. An audio decoding module that decodes the encoded audio frames and the recovered audio frames and outputs decoded audio samples.

Claims

1. An audio decoding system comprising:

a buffer module that receives packets including encoded audio frames that each store audio parameters;

a packet loss concealment module that selectively extracts the audio parameters from ones of the encoded audio frames, determines recovered audio parameters based on the extracted audio parameters, and encodes the recovered audio parameters into recovered audio frames;

an audio decoding module that decodes the encoded audio frames and the recovered audio frames, and outputs decoded audio samples;

an uncompressed adjustment module that generates an output stream of audio samples, and that incorporates the decoded audio samples into the output stream at a first rate; and

a playout control module that determines a target playout time based on packet delay information of the packets, and regulates the first rate based on the target playout time.

2. The audio decoding system of claim 1, wherein the decoded audio samples and the output stream of output samples comprise pulse-code modulation (PCM) samples.

3. The audio decoding system of claim 1, wherein the playout control module increases the target playout time at a first change rate based on an increase in jitter, and decreases the target playout time at a second change rate based on a decrease in the jitter, wherein the first change rate is greater than the change second rate.

4. The audio decoding system of claim 3, wherein the packet delay information comprises a transmission delay value for each of the packets, and the playout control module determines the jitter based on differences between the transmission delay values of at least two of the packets.

5. The audio decoding system of claim 1, further comprising a silence interval adjust module that, before the audio decoding module decodes the encoded audio frames, at least one of selectively inserts silent encoded audio frames and selectively deletes silent encoded audio frames, wherein the playout control module controls the silence interval adjust module based on the target playout time.

6. The audio decoding system of claim 5, wherein the silence interval adjust module only inserts the silent encoded audio frames adjacent to existing silent encoded audio frames received in the packets.

7. The audio decoding system of claim 5, wherein the playout control module causes the silence interval adjust module to selectively insert the silent encoded audio frames when the target playout time is greater than a threshold, and to selectively delete the silent encoded audio frames when the target playout time is less than the threshold, wherein a number of the silent encoded audio frames being inserted increases as the target playout time increases, and wherein a number of the silent encoded audio frames being deleted increases as the target playout time decreases.

8. The audio decoding system of claim 1, wherein each of the packets includes a monotonic sequence number, and the packet loss concealment module generates one of the recovered audio frames based on a first one of the packets having the sequence number prior to a missing packet.

9. The audio decoding system of claim 8, wherein the packet loss concealment module generates the one of the recovered audio frames based also on a second one of the packets having the sequence number subsequent to the missing packet.

10. The audio decoding system of claim 9, wherein the packet loss concealment module determines the recovered audio parameters by interpolating, for each of the audio parameters, between the corresponding extracted audio parameter from the first and second ones of the packets.

11. The audio decoding system of claim 8, wherein the packet loss concealment module determines the recovered audio parameters by extrapolating, for each of the audio parameters, from the corresponding extracted audio parameter from the first one of the packets.

12. The audio decoding system of claim 11, wherein the packet loss concealment module determines the recovered audio parameters by extrapolating, for each of the audio parameters, from the corresponding extracted audio parameter from the first one of the packets and from the corresponding extracted audio parameter from a second one of the packets having the sequence number prior to the first one of the packets.

13. An audio decoding system comprising:

a buffer module that receives packets including encoded audio frames that each store audio parameters;

a packet loss concealment module that selectively extracts the audio parameters from ones of the encoded audio frames, determines recovered audio parameters based on the extracted audio parameters, and encodes the recovered audio parameters into recovered audio frames;

an audio decoding module that decodes the encoded audio frames and the recovered audio frames, and outputs decoded audio samples;

an uncompressed adjustment module that generates an output stream of audio samples, and that incorporates the decoded audio samples into the output stream at a first rate; and

a playout control module that determines a target playout time based on packet delay information of the packets, and that increases the first rate as the target playout time decreases, wherein the output stream is read from the uncompressed adjustment module at a second rate.

14. An audio playback system comprising:

the audio decoding system of claim 13; and

a digital to analog converter that converts the output stream to analog at the second rate.

15. The audio decoding system of claim 13, wherein the playout control module decreases the first rate as the target playout time increases.

16. The audio decoding system of claim 13, wherein the uncompressed adjustment module selectively inserts at least one of waveform periods and individual audio samples into the output stream when the first rate is less than the second rate.

17. The audio decoding system of claim 16, wherein the uncompressed adjustment module incorporates all of the decoded audio samples into the output stream when the first rate is less than or equal to the second rate.

18. The audio decoding system of claim 16, wherein the uncompressed adjustment module selectively inserts the waveform periods when the output stream comprises voice data, and selectively inserts the individual audio samples otherwise, wherein the individual audio samples comprise at least one of silent audio samples and white noise samples.

19. The audio decoding system of claim 18, wherein the output stream comprises voice data when a rate of zero crossings of the output stream is less than a crossing threshold.

20. The audio decoding system of claim 18, wherein the uncompressed adjustment module inserts one of the waveform periods between first and second groups of audio samples of the output stream, and generates the one of the waveform periods based on the first and second groups.

21. The audio decoding system of claim 20, wherein the uncompressed adjustment module generates the one of the waveform periods by adding the first group multiplied by a first windowing function to the second group multiplied by a second windowing function.

22. The audio decoding system of claim 20, wherein the uncompressed adjustment module selectively inserts multiple copies of the one of the waveform periods between the first and second groups.

23. The audio decoding system of claim 20, wherein the first and second groups have lengths approximately equal to a length of the one of the waveform periods, wherein the length is determined by a periodicity of the output stream.

24. The audio decoding system of claim 23, wherein the uncompressed adjustment module determines the length of the one of the waveform periods by determining a level of periodicity of the output stream for each of a plurality of test periods and selecting one of the plurality of test periods whose level of periodicity is highest.

25. The audio decoding system of claim 24, wherein the uncompressed adjustment module determines the level of periodicity corresponding to a first one of the plurality of test periods by performing a correlation between a first group of the audio samples of the output stream and a second group of the audio samples of the output stream, wherein the first and second groups are adjacent and have lengths equal to the first one of the plurality of test periods.

26. The audio decoding system of claim 20, wherein the uncompressed adjustment module omits inserting the waveform periods when the output stream comprises unstable voice data, and wherein the output stream comprises unstable voice data when the highest level of periodicity is below a periodicity threshold.

27. The audio decoding system of claim 13, wherein, when the first rate is greater than the second rate, the uncompressed adjustment module selectively merges ones of the decoded audio samples and includes the merged audio samples in the output stream.

28. The audio decoding system of claim 27, wherein the uncompressed adjustment module merges the ones of the decoded audio samples when the output stream comprises voice data.

29. The audio decoding system of claim 28, wherein the uncompressed adjustment module merges first and second groups of the decoded audio samples, wherein the first and second groups are adjacent and have a length determined by a periodicity of the decoded audio samples.

30. The audio decoding system of claim 29, wherein the uncompressed adjustment module merges the first and second groups by adding the first group multiplied by a first windowing function to the second group multiplied by a second windowing function.

31. The audio decoding system of claim 13, wherein the second rate is approximately constant.

32. A method of controlling an audio decoding system, the method comprising:

receiving packets including encoded audio frames that each store audio parameters;

selectively extracting the audio parameters from ones of the encoded audio frames;

determining recovered audio parameters based on the extracted audio parameters;

encoding the recovered audio parameters into recovered audio frames;

decoding the encoded audio frames and the recovered audio frames into decoded audio samples;

generating an output stream of audio samples;

incorporating the decoded audio samples into the output stream at a first rate;

determining a target playout time based on packet delay information of the packets; and

regulating the first rate based on the target playout time.

33. The method of claim 32, wherein the decoded audio samples and the output stream of output samples comprise pulse-code modulation (PCM) samples.

34. The method of claim 32, further comprising:

increasing the target playout time at a first change rate based on an increase in jitter; and

decreasing the target playout time at a second change rate based on a decrease in the jitter, wherein the first change rate is greater than the change second rate.

35. The method of claim 34, wherein the packet delay information comprises a transmission delay value for each of the packets, and further comprising determining the jitter based on differences between the transmission delay values of at least two of the packets.

36. The method of claim 32, further comprising, before decoding the encoded audio frames:

at least one of selectively inserting silent encoded audio frames and selectively deleting silent encoded audio frames; and

controlling the inserting and deleting based on the target playout time.

37. The method of claim 36, further comprising inserting the silent encoded audio frames only adjacent to existing silent encoded audio frames received in the packets.

38. The method of claim 36, further comprising:

selectively inserting the silent encoded audio frames when the target playout time is greater than a threshold;

selectively deleting the silent encoded audio frames when the target playout time is less than the threshold;

increasing a number of the silent encoded audio frames being inserted as the target playout time increases; and

increasing a number of the silent encoded audio frames being deleted as the target playout time decreases.

39. The method of claim 32, wherein each of the packets includes a monotonic sequence number, and further comprising generating one of the recovered audio frames based on a first one of the packets having the sequence number prior to a missing packet.

40. The method of claim 39, further comprising generating the one of the recovered audio frames based also on a second one of the packets having the sequence number subsequent to the missing packet.

41. The method of claim 40, further comprising determining the recovered audio parameters by interpolating, for each of the audio parameters, between the corresponding extracted audio parameter from the first and second ones of the packets.

42. The method of claim 39, further comprising determining the recovered audio parameters by extrapolating, for each of the audio parameters, from the corresponding extracted audio parameter from the first one of the packets.

43. The method of claim 42, further comprising determining the recovered audio parameters by extrapolating, for each of the audio parameters, from the corresponding extracted audio parameter from the first one of the packets and from the corresponding extracted audio parameter from a second one of the packets having the sequence number prior to the first one of the packets.

44. A method of controlling an audio decoding system, the method comprising:

receiving packets including encoded audio frames that each store audio parameters;

selectively extracting the audio parameters from ones of the encoded audio frames;

determining recovered audio parameters based on the extracted audio parameters,

encoding the recovered audio parameters into recovered audio frames;

decoding the encoded audio frames and the recovered audio frames into decoded audio samples;

generating an output stream of audio samples;

incorporating the decoded audio samples into the output stream at a first rate;

determining a target playout time based on packet delay information of the packets; and

increasing the first rate as the target playout time decreases, wherein the output stream is read at a second rate.

45. The method of claim 44, further comprising converting the output stream to analog at the second rate.

46. The method of claim 44, further comprising decreasing the first rate as the target playout time increases.

47. The method of claim 44, further comprising selectively inserting at least one of waveform periods and individual audio samples into the output stream when the first rate is less than the second rate.

48. The method of claim 47, further comprising incorporating all of the decoded audio samples into the output stream when the first rate is less than or equal to the second rate.

49. The method of claim 47, further comprising:

selectively inserting the waveform periods when the output stream comprises voice data; and

selectively inserting the individual audio samples when the output stream comprises other than voice data, wherein the individual audio samples comprise at least one of silent audio samples and white noise samples.

50. The method of claim 49, wherein the output stream comprises voice data when a rate of zero crossings of the output stream is less than a crossing threshold.

51. The method of claim 49, further comprising:

inserting one of the waveform periods between first and second groups of audio samples of the output stream; and

generating the one of the waveform periods based on the first and second groups.

52. The method of claim 51, further comprising generating the one of the waveform periods by adding the first group multiplied by a first windowing function to the second group multiplied by a second windowing function.

53. The method of claim 51, further comprising selectively inserting multiple copies of the one of the waveform periods between the first and second groups.

54. The method of claim 51, wherein the first and second groups have lengths approximately equal to a length of the one of the waveform periods, and wherein the length is determined by a periodicity of the output stream.

55. The method of claim 54, further comprising determining the length of the one of the waveform periods by:

determining a level of periodicity of the output stream for each of a plurality of test periods; and

selecting one of the plurality of test periods whose level of periodicity is highest.

56. The method of claim 55, further comprising determining the level of periodicity corresponding to a first one of the plurality of test periods by performing a correlation between a first group of the audio samples of the output stream and a second group of the audio samples of the output stream, wherein the first and second groups are adjacent and have lengths equal to the first one of the plurality of test periods.

57. The method of claim 51, further comprising omitting inserting the waveform periods when the output stream comprises unstable voice data, wherein the output stream comprises unstable voice data when the highest level of periodicity is below a periodicity threshold.

58. The method of claim 44, further comprising, when the first rate is greater than the second rate, selectively merging ones of the decoded audio samples and includes the merged audio samples in the output stream.

59. The method of claim 58, further comprising merging the ones of the decoded audio samples when the output stream comprises voice data.

60. The method of claim 59, further comprising merging first and second groups of the decoded audio samples, wherein the first and second groups are adjacent and have a length determined by a periodicity of the decoded audio samples.

61. The method of claim 60, further comprising merging the first and second groups by adding the first group multiplied by a first windowing function to the second group multiplied by a second windowing function.

62. The method of claim 44, wherein the second rate is approximately constant.