Enhanced error concealment for spatial audio

• US 20040039464A1
• Filed: 06/13/2003
• Published: 02/26/2004
• Est. Priority Date: 06/14/2002
• Status: Abandoned Application

• Current Assignee: Nokia Solutions and Networks US LLC (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Lakaniemi, Ari, Virolainen, Jussi
• Application Number:
  US10/465,909
• Publication Number:
  US 20040039464A1
• Time in Patent Office: Days
• US Class Current: 700/94
• CPC Class Codes: G10L 19/005 Correction of errors induced by the transmission channel, if related to the coding algorithm
• CPC Class Codes: G10L 19/008 Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• CPC Class Codes: G11B 20/00992 Circuits for stereophonic or quadraphonic recording or reproducing
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 20/18 Error detection or correction; Testing , e.g. of drop-outs

Abstract

An error concealment method for multi-channel digital audio involves receiving an audio signal having audio data forming a first audio channel and a second audio channel included therein, wherein the first and second audio channels are correlated with each other in a manner so that a spatial sensation is typically perceived when listened to by a user. Erroneous first-channel data is detected in the first audio channel, and second-channel data is obtained from the second audio channel. The erroneous first-channel data of the first audio channel is corrected by using the second-channel data. Upon detection of the erroneous first-channel data, a spatially perceivable inter-channel relation between the first and second audio channels is determined, and the determined inter-channel relation is used when correcting the erroneous first-channel data of the first audio channel so as to preserve the spatial sensation perceived by the user.

Claims

1. An error concealment method for multi-channel digital audio, the method comprising the steps of receiving an audio signal having audio data forming a first audio channel and a second audio channel included therein, said first and second audio channels being correlated with each other in a manner so that a spatial sensation is typically perceived when listened to by a user;

detecting erroneous first-channel data in the first audio channel;

obtaining second-channel data from the second audio channel;

determining, upon detection of the erroneous first-channel data, a spatially perceivable inter-channel relation between the first and second audio channels; and

correcting the erroneous first-channel data of the first audio channel by using the second-channel data and the determined inter-channel relation so as to preserve the spatial sensation perceived by the user.

2. A method as in claim 1, wherein the erroneous first-channel data of the first audio channel is corrected by manipulating the second-channel data in accordance with the determined inter-channel relation and then replacing the erroneous first-channel data with manipulated second-channel data.

3. A method as in claim 2, wherein the determined inter-channel relation is a phase difference between the first and second audio channels.

4. A method as in claim 3, wherein the manipulation of the second-channel data comprises selecting said second-channel data from the second audio channel with a time shift with respect to the first audio channel, said time shift corresponding to the determined phase difference.

5. A method as in claim 3, wherein the phase difference is determined by analyzing the first and second channels of the received audio signal with respect to each other.

6. A method as in claim 5, wherein the analysis of the first and second channels of the received audio signal involves:

low-pass filtering of each of the first and second channels; and

detecting the phases of the first and second channels after low-pass filtering by matching peaks or zero-crossings, or both in voiced phonemes.

7. A method as in claim 1, wherein the spatially perceivable inter-channel relation is determined from metadata received together with the audio signal.

8. A method as in claim 1, comprising the additional step of decoding the received audio signal prior to said step of detecting erroneous first-channel data in the first audio channel.

9. A method as in claim 1, wherein the first and second audio channels each comprises a plurality of audio frames and wherein the detection and correction of erroneous first-channel data concern at least one entire audio frame.

10. A method as in claim 1, wherein the first and second audio channels each comprises a plurality of audio frames and wherein the detection and correction of erroneous first-channel data concern part or parts of an audio frame.

11. A method as in claim 10, wherein said part or parts of an audio frame relates to one or more spectral sub-bands.

12. A method as in claim 9, wherein the detection and correction of erroneous first-channel data is performed on a plurality of time domain audio samples contained in the audio frame.

13. A method as in claim 1, wherein the first and second audio channels are left and right stereo channels, or vice versa.

14. A method as in claim 1, comprising the additional steps, after said step of detecting erroneous first-channel data in the first audio channel, of:

detecting erroneous second-channel data in the second audio channel, essentially concurrent with the erroneous first-channel data detected in the first audio channel;

selecting either the first audio channel or the second audio channel as source channel for audio reconstruction;

reconstructing the erroneous data of the selected source channel from preceding data in the selected source channel; and

reconstructing the erroneous data of the other of the first and second audio channels, which was not selected as source channel, from the reconstructed data of the source channel in accordance with the remaining steps of claim 1.

15. A method as in claim 14, wherein the one of the first audio channel or the second audio channel which has the highest signal energy or power level is selected as source channel.

16. A method as in claim 14, wherein the one of the first audio channel or the second audio channel which is leading in terms of phase is selected as source channel.

17. A method as in claim 16, wherein said step of reconstructing the erroneous data of the selected source channel from preceding data in the selected source channel is performed by attenuated extrapolation or copying of said preceding data.

18. A method as in claim 1, wherein the audio signal is received from a teleconference bridge.

19. A method as in claim 1, wherein the audio signal is received from a stereo music server.

20. A method as in claim 1, wherein the audio signal is received over a radio network, a fixed telecommunications network, a mobile telecommunications network, a short-range optical link or a short-range radio link.

21. A method as in claim 14, comprising the additional steps of attenuating the reconstructed data of said source channel and said other channel;

maintaining the first and second audio channels attenuated for as long as there are consecutive errors on the first and second audio channels; and

upon detecting that there are no more consecutive errors on the first and second audio channels, amplifying the first and second audio channels to cancel the attenuation thereof.

22. A method as in claim 1, wherein the step of correcting the erroneous first-channel data of the first audio channel involves using the second-channel data of the second audio channel as well as preceding non-erroneous first-channel data of the first audio channel.

23. A method as in claim 22 and comprising the additional step of decoding the received audio signal prior to said step of detecting erroneous first-channel data in the first audio channel, wherein the received audio signal is decoded by at least one codec, such as an MPEG4 or MPEG-2 AAC codec, an ISO/MPEG Audio Layer-3 (MP3) codec, or two mono codecs like GSM EFR/FR/HR speech codec, AMR, Wideband AMR, G. 711, G. 722, G. 722.1, G. 723, G. 728, or an MPEG1/2/4 CELP+AAC codec.

24. A method as in claim 1, wherein the detection and correction of erroneous first-channel data concern a audio component or components, such as a principal audio component or components, which is or are detected or indicated to be present in the audio signal.

25. A computer program product directly loadable into a memory of a processor, where the computer program product comprises program code for performing the method according to claim 1 when executed by said processor.

26. An integrated circuit, which is adapted to perform the method according to claim 1.

28. A user terminal for a communications network, the user terminal comprising an integrated circuit according to claim 26.

30. A user terminal as in claim 28, wherein the communications network includes a mobile telecommunications network and the user terminal is a mobile terminal.

31. A user terminal as in claim 28, adapted to receive the audio signal from a teleconference bridge over the communications network.

34. A teleconference system comprising a communications network, a plurality of user terminals according to claim 28 and a teleconference bridge, wherein the user terminals are connected to the teleconference bridge over the communications network.

27. A receiver of multi-channel digital audio, comprising means for receiving an audio signal having audio data forming a first audio channel and a second audio channel included therein, said first and second audio channels being correlated with each other in a manner so that a spatial sensation is typically perceived when listened to by a user;

means for detecting erroneous first-channel data in the first audio channel;

means for obtaining second-channel data from the second audio channel;

means for correcting the erroneous first-channel data of the first audio channel by using the second-channel data; and

means for determining, upon detection of the erroneous first-channel data, a spatially perceivable inter-channel relation between the first and second audio channels, wherein

said means for correcting the erroneous first-channel data of the first audio channel is adapted to use the determined inter-channel relation when correcting the erroneous first-channel data so as to preserve the spatial sensation perceived by the user.

29. A user terminal for a communications network, the user terminal comprising a receiver according to claim 27.

32. A user terminal as in claim 29, wherein the communications network includes a mobile telecommunications network and the user terminal is a mobile terminal.

33. A user terminal as in claim 29, adapted to receive the audio signal from a teleconference bridge over the communications network.

35. A teleconference system comprising a communications network, a plurality of user terminals according to claim 29 and a teleconference bridge, wherein the user terminals are connected to the teleconference bridge over the communications network.