Audio Packet Loss Concealment by Transform Interpolation

• US 20110191111A1
• Filed: 01/29/2010
• Published: 08/04/2011
• Est. Priority Date: 01/29/2010
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Polycom Incorporated (Plantronics, Inc.)
• Inventors: Tu, Zhemin, Chu, Peter
• Granted Patent: US 8,428,959 B2
• Time in Patent Office: Days
• US Class Current: 704/500
• CPC Class Codes: G10L 19/005 Correction of errors induced by the transmission channel, if related to the coding algorithm
• CPC Class Codes: G10L 19/0212 using orthogonal transformation

Abstract

In audio processing for an audio or video conference, a terminal receives audio packets having transform coefficients for reconstructing an audio signal that has undergone transform coding. When receiving the packets, the terminal determines whether there are any missing packets and interpolates transform coefficients from the preceding and following good frames. To interpolate the missing coefficients, the terminal weights first coefficients from the preceding good frame with a first weighting, weights second coefficients from the following good frame with a second weighting, and sums these weighted coefficients together for insertion into the missing packets. The weightings can be based on the audio frequency and/or the number of missing packets involved. From this interpolation, the terminal produces an output audio signal by inverse transforming the coefficients.

Claims

1. An audio processing method, comprising:

receiving sets of packets at an audio processing device via a network, each set having one or more of the packets, each packet having transform coefficients in a frequency domain for reconstructing an audio signal in a time domain that has undergone transform coding;

determining one or more missing packets in a given one of the sets received;

applying a first weight to first transform coefficients of one or more first packets in a first set sequenced before the given set;

applying a second weight to second transform coefficients of one or more second packets in a second set sequenced after the given set;

interpolating transform coefficients by summing the first and second weighted transform coefficients;

inserting the interpolated transform coefficients into the one or more missing packets; and

producing an output audio signal for the audio processing device by performing an inverse transform on the transform coefficients.

2. The method of claim 1, wherein the audio processing device is selected from the group consisting of an audio conferencing endpoint, a videoconferencing endpoint, an audio playback device, a personal music player, a computer, a server, a telecommunications device, a cellular telephone, and a personal digital assistant.

3. The method of claim 1, wherein the network comprises an Internet Protocol network.

4. The method of claim 1, wherein the transform coefficients comprise coefficients of a Modulated Lapped Transform.

5. The method of claim 1, wherein each set has one packet, and wherein the one packet encompasses a frame of input audio.

6. The method of claim 1, wherein receiving comprises decoding the packets.

7. The method of claim 6, wherein receiving comprising de-quantizing the decoded packets.

8. The method of claim 1, wherein determining the one or more missing packets comprises sequencing the packets received in a buffer and finding gaps in the sequencing.

9. The method of claim 1, wherein interpolating the transform coefficients comprises assigning a random positive or negative sign to the summed first and second weighted transform coefficients.

10. The method of claim 1, wherein the first and second weights applied to the first and second transform coefficients are based on audio frequencies.

11. The method of claim 10, wherein if the audio frequencies fall below a threshold, the first weight emphasizes the first transform coefficients, and the second weight de-emphasizes the second transform coefficients.

12. The method of claim 11, wherein the threshold is 1 kHz.

13. The method of claim 11, wherein the first transform coefficients are weighted at 75 percent, and wherein the second transform coefficients are zeroed.

14. The method of claim 10, wherein if the audio frequencies exceed a threshold, the first and second weights equally emphasize the first and second transform coefficients.

15. The method of claim 14, wherein the first and second transform coefficients are both weighted at 50 percent.

16. The method of claim 1, wherein the first and second weights applied to the first and second transform coefficients are based on a number of the missing packets.

17. The method of claim 16, wherein if one of the packets is missing in the given set,

the first weight emphasizes the first transform coefficients and the second weight de-emphasizes the second transform coefficients if an audio frequency related to the missing packet falls below a threshold;

and

the first and second weights equally emphasize the first and second transform coefficients if the audio frequency exceeds the threshold.

18. The method of claim 16, wherein if two of the packets are missing in the given set,

the first weighting emphasizes the first transform coefficients for a preceding one of the two packets and de-emphasizes the first transform coefficients for a following one of the two packets;

and

the second weighting de-emphasizes the second transform coefficients for the preceding packet and emphasizes the second transform coefficients for the following packet.

19. The method of claim 18, wherein the emphasized coefficients are weighted at 90 percent, and wherein the de-emphasized coefficients are zeroed.

20. The method of claim 16, wherein if three or more packets are missing in the given set,

the first weighting emphasizes the first transform coefficients for a first one of the packets and de-emphasizes the first transform coefficients for a last one of the packets;

the first and second weightings equally emphasizes the first and second transform coefficients for one or more intermediate ones of the packets; and

the second weighting de-emphasizes the second transform coefficients for the first one of the packets and emphasizes the second transform coefficients for the last of the packets.

21. The method of claim 20, wherein the emphasized coefficients are weighted at 90 percent, wherein the de-emphasized coefficients are zeroed, and wherein the equally emphasized coefficients are weighted at 40 percent.

22. A program storage device having instructions stored thereon for causing a programmable control device to perform an audio processing method of claim 1.

23. An audio processing device, comprising:

an audio output interface;

a network interface in communication with at least one network and receiving sets of packets of audio, each set having one or more of the packets, each packet having transform coefficients in a frequency domain;

memory in communication with the network interface and storing the received packets;

a processing unit in communication with the memory and the audio output interface, the processing unit programmed with an audio decoder configured to;

determine one or more missing packets in a given one of the sets received;

apply a first weighting to first transform coefficients of one or more first packets from a first set sequenced before the given set;

apply a second weighting to second transform coefficients of one or more second packets from a second set sequenced after the given set;

interpolate transform coefficients by summing the first and second weighted transform coefficients;

insert the interpolated transform coefficients into the one or more missing packets; and

perform an inverse transform on the transform coefficients to produce an output audio signal in a time domain for the audio output interface.

24. The device of claim 23, wherein the device comprises a conferencing endpoint.

25. The device of claim 23, further comprising a speaker communicably coupled to the audio output interface.

26. The device of claim 23, further comprising an audio input interface and a microphone communicably coupled to the audio input interface.

27. The device of claim 26, wherein the processing unit is in communication with the audio input interface and is programmed with an audio encoder configured to:

transform frames of time domain samples of an audio signal to frequency domain transform coefficients;

quantize the transform coefficients; and

code the quantized transform coefficients.