Encoder-assisted frame loss concealment techniques for audio coding

• US 8,620,644 B2
• Filed: 05/10/2006
• Issued: 12/31/2013
• Est. Priority Date: 10/26/2005
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Ryu, Sang-Uk, Choy, Eddie L. T., Gupta, Samir Kumar
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US11/431,733
• Publication Number:
  US 20070094009A1
• Time in Patent Office: 2,792 Days
• Field of Search: 704/201, 704/205, 704/206
• US Class Current: 704/201
• CPC Class Codes: G10L 19/005 Correction of errors induced by the transmission channel, if related to the coding algorithm
• CPC Class Codes: G10L 19/02 using spectral analysis, e.g. transform vocoders or subband vocoders

Abstract

Encoder-assisted frame loss concealment (FLC) techniques for decoding audio signals are described. A decoder may discard an erroneous frame of an audio signal and may implement the encoder-assisted FLC techniques in order to accurately conceal the discarded frame based on neighboring frames and side-information transmitted from the encoder. The encoder-assisted FLC techniques include estimating magnitudes of frequency-domain data for the frame based on frequency-domain data of neighboring frames, and estimating signs of the frequency-domain data based on a subset of signs transmitted from the encoder as side-information. Frequency-domain data for a frame of an audio signal includes tonal components and noise components. Signs estimated from a random signal may be substantially accurate for the noise components of the frequency-domain data. However, to achieve highly accurate sign estimation for the tonal components, the encoder transmits signs for the tonal components of the frequency-domain data as side-information.

Claims

1. A method of concealing a frame of an audio signal comprising:

receiving the frame at a decoder, the frame including frequency-domain data of the audio signal;

the decoder detecting one or more errors in the frame and discarding the frequency-domain data as a result of detecting the errors;

the decoder estimating magnitudes of replacement frequency-domain data for the frame based on frequency-domain data included in neighboring frames of the frame;

the decoder estimating signs of the replacement frequency-domain data for the frame based on a subset of signs for the frame transmitted from an encoder as side-information of a neighboring frame of the frame; and

the decoder combining the magnitude estimates and the sign estimates to estimate the replacement frequency-domain data for the frame.

2. The method of claim 1, further comprising:

receiving an audio bitstream for the frame including frequency-domain data from the encoder; and

receiving the side-information for the frame with an audio bitstream for a neighboring frame from the encoder.

3. The method of claim 1, further comprising:

performing error detection on an audio bitstream for the frame transmitted from the encoder; and

discarding frequency-domain data for the frame when one or more errors are detected.

4. The method of claim 1, wherein estimating magnitudes of the replacement frequency-domain data for the frame comprises performing energy interpolation based on the energy of a preceding frame of the frame and a subsequent frame of the frame.

5. The method of claim 1, wherein estimating signs of the replacement frequency-domain data for the frame comprises:

estimating signs for noise components of the replacement frequency-domain data for the frame from a random signal; and

estimating signs for tonal components of the replacement frequency-domain data for the frame based on the subset of signs for the frame transmitted from the encoder as the side-information.

6. The method of claim 1, wherein estimating signs of the replacement frequency-domain data for the frame comprises:

selecting tonal components of the frequency-domain data for the frame;

generating an index subset that identifies locations of the tonal components within the frame; and

estimating signs for the tonal components from the subset of signs for the frame based on the index subset.

7. The method of claim 6, wherein selecting tonal components comprises:

sorting the frequency-domain data in order of magnitudes; and

selecting a predetermined number of the frequency-domain data with the highest magnitudes as the tonal components.

8. The method of claim 1, wherein estimating signs of the replacement frequency-domain data for the frame comprises:

selecting tonal components from the magnitude estimates of the frequency-domain data for the frame;

generating an estimated index subset that identifies locations of the tonal components selected from the magnitude estimates of the frequency-domain data for the frame; and

estimating signs for the tonal components from the subset of signs for the frame based on the estimated index subset for the frame.

9. The method of claim 1, wherein estimating signs of the replacement frequency-domain data for the frame comprises:

selecting tonal components from magnitudes of frequency-domain data for a neighboring frame of the frame;

generating an index subset that identifies locations of the tonal components selected from the magnitudes of the frequency-domain data for the neighboring frame; and

estimating signs for the tonal components from the subset of signs for the frame based on the index subset for the neighboring frame.

10. The method of claim 1, further comprising:

transmitting an audio bitstream for the frame including frequency-domain data to a decoder; and

transmitting the side-information for the frame with an audio bitstream for a neighboring frame to a decoder.

11. The method of claim 10, wherein transmitting the side-information comprises:

extracting the subset of signs from the frequency-domain data for the frame; and

attaching the subset of signs to the audio bitstream for the neighboring frame as the side-information.

12. The method of claim 11, wherein extracting the subset of signs for the frame comprises:

selecting tonal components of the frequency-domain data for the frame;

generating an index subset that identifies locations of the tonal components within the frame; and

extracting the subset of signs for the tonal components from the frequency-domain data for the frame based on the index subset.

13. The method of claim 12, wherein selecting tonal components comprises:

sorting the frequency-domain data in order of magnitudes; and

selecting a predetermined number of the frequency-domain data with the highest magnitudes as the tonal components.

14. The method of claim 11, wherein extracting the subset of signs for the frame comprises:

estimating magnitudes of the frequency-domain data for the frame based on neighboring frames of the frame;

selecting tonal components from the frequency-domain data magnitude estimates for the frame;

generating an estimated index subset that identifies locations of the tonal components selected from the frequency-domain data magnitude estimates for the frame; and

extracting the subset of signs for the tonal components from the frequency-domain data for the frame based on the estimated index subset for the frame.

15. The method of claim 11, wherein extracting the subset of signs for the frame comprises:

selecting tonal components from frequency-domain data magnitudes for the neighboring frame;

generating an index subset that identifies locations of the tonal components selected from the frequency-domain data magnitudes for the neighboring frame; and

extracting the subset of signs for the tonal components from the frequency-domain data for the frame based on the index subset for the neighboring frame.

16. The method of claim 1, further comprising:

encoding a time-domain audio signal for the frame into frequency-domain data for the frame with a transform unit included in the encoder; and

decoding the replacement frequency-domain data for the frame into estimated time-domain data for the frame with an inverse transform unit included in a decoder.

17. The method of claim 1, wherein the side-information comprises a subset of signs for tonal components of frequency-domain data for the frame, the method further comprising:

generating an index subset that identifies locations of the tonal components within the frame with the encoder;

extracting the subset of signs for the tonal components from the frequency-domain data for the frame based on the index subset with the encoder;

transmitting the subset of signs for the tonal components as the side-information to a decoder;

generating an index subset that identifies locations of the tonal components within the frame with the decoder using the same process as the encoder; and

estimating signs for the tonal components from the subset of signs based on the index subset.

18. A non-transitory computer-readable medium comprising instructions for concealing a frame of an audio signal that cause a programmable processor to:

receive the frame, the frame including frequency-domain data of the audio signal;

detect one or more errors in the frame;

discard the frequency-domain data as a result of detecting the errors;

estimate magnitudes of replacement frequency-domain data for the frame based on frequency-domain data included in neighboring frames of the frame;

estimate signs of the replacement frequency-domain data for the frame based on a subset of signs for the frame transmitted from an encoder as side-information of a neighboring frame of the frame; and

combine the magnitude estimates and the sign estimates to estimate the replacement frequency-domain data for the frame.

19. The computer-readable medium of claim 18, wherein the instructions cause the programmable processor to:

estimate signs for noise components of the replacement frequency-domain data for the frame from a random signal; and

estimate signs for tonal components of the replacement frequency-domain data for the frame based on the subset of signs for the frame transmitted from the encoder as the side-information.

20. The computer-readable medium of claim 18, wherein the instructions cause the programmable processor to:

sort the frequency-domain data for the frame in order of magnitudes;

select a predetermined number of the frequency-domain data with the highest magnitudes as tonal components of the frequency-domain data for the frame;

generate an index subset that identifies locations of the tonal components within the frame; and

estimate signs for the tonal components from the subset of signs for the frame based on the index subset.

21. The computer-readable medium of claim 18, further comprising instructions that cause the programmable processor to:

extract the subset of signs from the frequency-domain data for the frame;

attach the subset of signs to an audio bitstream for a neighboring frame as the side-information; and

transmit the side-information for the frame with the audio bitstream for the neighboring frame to a decoder.

22. The computer-readable medium of claim 21, wherein the instructions cause the programmable processor to:

sort the frequency-domain data for the frame in order of magnitudes;

select a predetermined number of the frequency-domain data with the highest magnitudes as tonal components of the frequency-domain data for the frame;

generate an index subset that identifies locations of the tonal components within the frame; and

extract the subset of signs for the tonal components from the frequency-domain data for the frame based on the index subset.

23. A system for concealing a frame containing frequency-domain data of an audio signal comprising:

an encoder that transmits a subset of signs for the frame as side-information of a neighboring frame of the frame; and

a decoder including a frame loss concealment (FLC) module that receives the side-information for the frame from the encoder, and an error detection module that detects one or more errors in the frame and discards the frequency-domain data as a result of detecting the errors,

wherein the FLC module estimates magnitudes of replacement frequency-domain data for the frame based on frequency-domain data of neighboring frames of the frame, estimates signs of the replacement frequency-domain data for the frame based on the subset of signs received as side-information, and combines the magnitude estimates and the sign estimates to estimate the replacement frequency-domain data for the frame.

24. The system of claim 23, wherein the error detection module performs error detection on an audio bitstream for the frame transmitted from the encoder.

25. The system of claim 23, wherein the FLC module includes a magnitude estimator that performs energy interpolation based on the energy of a preceding frame of the frame and a subsequent frame of the frame to estimate the magnitudes of the replacement frequency-domain data for the frame.

26. The system of claim 23, wherein the FLC module includes a sign estimator that:

estimates signs for noise components of the replacement frequency-domain data for the frame from a random signal; and

estimates signs for tonal components of the replacement frequency-domain data for the frame based on the subset of signs for the frame transmitted from the encoder as the side-information.

27. The system of claim 23,

wherein the FLC module includes a component selection module that sorts the frequency-domain data for the frame in order of magnitudes, selects a predetermined number of the frequency-domain data with the highest magnitudes as tonal components of the frequency-domain data for the frame, and generates an index subset that identifies locations of the tonal components within the frame;

and

wherein the sign estimator estimates signs for the tonal components from the subset of signs for the frame based on the index subset.

28. The system of claim 23, wherein the encoder includes a sign extractor that extracts the subset of signs from the frequency-domain data for the frame, and attaches the subset of signs to an audio bitstream for a neighboring frame as the side-information, wherein the encoder transmits the side-information for the frame with the audio bitstream for the neighboring frame to the decoder.

29. The system of claim 28,

wherein the encoder includes a component selection module that sorts the frequency-domain data for the frame in order of magnitudes, selects a predetermined number of the frequency-domain data with the highest magnitudes as tonal components of the frequency-domain data for the frame, and generates an index subset that identifies locations of the tonal components within the frame;

and

wherein the sign extractor extracts the subset of signs for the tonal components from the frequency-domain data for the frame based on the index subset.

30. The system of claim 23, wherein frequency-domain data for the frame is represented by modified discrete cosine transform (MDCT) coefficients.

31. The system of claim 23,

wherein the encoder includes a transform unit that encodes a time-domain audio signal for the frame into frequency-domain data for the frame;

and

wherein the decoder includes an inverse transform unit that decodes the replacement frequency-domain data for the frame into replacement time-domain data for the frame.

32. The system of claim 31, wherein the transform unit included in the encoder comprises a modified discrete cosine transform unit, and wherein the inverse transform unit included in the decoder comprises an inverse modified discrete cosine transform unit.

33. The system of claim 23, wherein the side-information comprises a subset of signs for tonal components of frequency-domain data for the frame,

wherein the encoder generates an index subset that identifies locations of the tonal components within the frame with the encoder, extracts the subset of signs for the tonal components from the frequency-domain data for the frame based on the index subset with the encoder, and transmits the subset of signs for the tonal components as the side-information to the decoder;

and

wherein the decoder generates an index subset that identifies locations of the tonal components within the frame with the decoder using the same process as the encoder, and estimates signs for the tonal components from the subset of signs based on the index subset.

34. An encoder comprising:

a component selection module that selects components of frequency-domain data for a frame of an audio signal; and

a sign extractor that extracts a subset of signs for the selected components from the frequency-domain data for the frame,

wherein the encoder transmits the subset of signs for the frame to a decoder as side-information of a neighboring frame of the frame.

35. The encoder of claim 34, wherein the encoder transmits an audio bitstream for the frame including frequency-domain data to the decoder and transmits the side-information for the frame with an audio bitstream for a neighboring frame to the decoder, wherein the sign extractor attaches the side-information for the frame to the audio bitstream for the neighboring frame.

36. The encoder of claim 34, wherein the component selection module generates an index subset that identifies locations of the components within the frame.

37. The encoder of claim 34, wherein the selected components comprise tonal components of the frequency-domain data for the frame, wherein the component selection module sorts the frequency-domain data for the frame in order of magnitudes, and selects a predetermined number of the frequency-domain data with the highest magnitudes as the tonal components.

38. The encoder of claim 34, further comprising a FLC module including:

a magnitude estimator that estimates magnitudes of the frequency-domain data for the frame based on neighboring frames of the frame;

the component selection module that selects tonal components from the frequency-domain data magnitude estimates for the frame, and generates an estimated index subset that identifies locations of the tonal components selected from the frequency-domain data magnitude estimates for the frame; and

the sign extractor that extracts the subset of signs for the tonal components from the frequency-domain data for the frame based on the estimated index subset for the frame.

39. The encoder of claim 34,

wherein the component selection module selects tonal components from frequency-domain data magnitudes for the neighboring frame, and generates an index subset that identifies locations of the tonal components selected from the frequency-domain data magnitudes for the neighboring frame;

and

wherein the sign extractor extracts the subset of signs for the tonal components from the frequency-domain data for the frame based on the index subset for the neighboring frame.

40. A decoder comprising:

an error detection module that detects one or more errors in a frame of an audio signal and discards frequency-domain data of the frame as a result of detecting the errors; and

a frame loss concealment (FLC) module including;

a magnitude estimator that estimates magnitudes of replacement frequency-domain data for the frame based on neighboring frames of the frame; and

a sign estimator that estimates signs of the replacement frequency-domain data for the frame based on a subset of signs for the frame transmitted from an encoder as side-information of a neighboring frame of the frame,

wherein the decoder combines the magnitude estimates and the sign estimates to estimate the replacement frequency-domain data for the frame.

41. The decoder of claim 40, wherein the decoder receives an audio bitstream for the frame including frequency-domain data from the encoder, and receives the side-information for the frame with an audio bitstream for a neighboring frame from the encoder.

42. The decoder of claim 40, wherein the error detection module performs error detection on an audio bitstream for the frame transmitted from the encoder.

43. The decoder of claim 40, wherein the FLC module includes a magnitude estimator that performs energy interpolation based on the energy of a preceding frame of the frame and a subsequent frame of the frame to estimate the magnitudes of the replacement frequency-domain data for the frame.

44. The decoder of claim 40, wherein the sign estimator estimates signs for noise components of the replacement frequency-domain data for the frame from a random signal, and estimates signs for tonal components of the replacement frequency-domain data for the frame based on the subset of signs for the frame transmitted from the encoder as the side-information.

45. The decoder of claim 40,

wherein the FLC module includes a component selection module that selects tonal components of the frequency-domain data for the frame, and generates an index subset that identifies locations of the tonal components within the frame;

and

wherein the sign estimator estimates signs for the tonal components from the subset of signs for the frame based on the index subset.

46. The decoder of claim 45, wherein the component selection module sorts the frequency-domain data in order of magnitudes, and selects a predetermined number of the frequency-domain data with the highest magnitudes as the tonal components.

47. The decoder of claim 40,

wherein the FLC module includes a component selection module that selects tonal components from the magnitude estimates of the frequency-domain data for the frame, and generates an estimated index subset that identifies locations of the tonal components selected from the magnitude estimates of the frequency-domain data for the frame;

and

wherein the sign estimator estimates signs for the tonal components from the subset of signs for the frame based on the estimated index subset for the frame.

48. The decoder of claim 40,

wherein the FLC module includes a component selection module that selects tonal components from magnitudes of frequency-domain data for a neighboring frame of the frame, and generates an index subset that identifies locations of the tonal components selected from the magnitudes of the frequency-domain data for the neighboring frame;

and

wherein the sign estimator estimates signs for the tonal components from the subset of signs for the frame based on the index subset for the neighboring frame.

49. An apparatus for concealing a frame of an audio signal comprising:

means for receiving the frame which includes frequency-domain data of the audio signal;

means for detecting one or more errors in the frame and discarding the frequency-domain data as a result of detecting the errors;

means for estimating magnitudes of replacement frequency-domain data for the frame based on frequency-domain data included in neighboring frames of the frame;

means for estimating signs of the replacement frequency-domain data for the frame based on a subset of signs for the frame transmitted from an encoder as side-information of a neighboring frame of the frame; and

means for combining the magnitude estimates and the sign estimates to estimate the replacement frequency-domain data for the frame.