System and method for error concealment in digital audio transmission

• US 7,447,639 B2
• Filed: 12/14/2001
• Issued: 11/04/2008
• Est. Priority Date: 01/24/2001
• Status: Expired due to Fees

• Current Assignee: Nokia Solutions and Networks US LLC (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Wang, Ye
• Primary Examiner(s): Opsasnick; Michael N
• Application Number:
  US10/020,579
• Publication Number:
  US 20020138795A1
• Time in Patent Office: 2,517 Days
• Field of Search: 704/201, 704/219, 704/230, 704500-504
• US Class Current: 704/503
• CPC Class Codes: G10H 1/0058 Transmission between separate instruments or between individual components of a musical system G10H1/0083 takes precedence
• CPC Class Codes: G10H 2240/061 MP3, i.e. MPEG-1 or MPEG-2 Audio Layer III, lossy audio compression
• CPC Class Codes: G10H 2240/185 Error prevention, detection or correction in files or streams for electrophonic musical instruments
• CPC Class Codes: G10H 2240/245 ISDN [Integrated Services Digital Network]
• CPC Class Codes: G10H 2240/251 Mobile telephone transmission, i.e. transmitting, accessing or controlling music data wirelessly via a wireless or mobile telephone receiver, analogue or digital, e.g. DECT, GSM, UMTS
• CPC Class Codes: G10H 2240/295 Packet switched network, e.g. token ring
• CPC Class Codes: G10H 2240/305 Internet or TCP/IP protocol use for any electrophonic musical instrument data or musical parameter transmission purposes
• CPC Class Codes: G10L 19/005 Correction of errors induced by the transmission channel, if related to the coding algorithm
• CPC Class Codes: G10L 19/0212 using orthogonal transformation

Abstract

A beat-pattern based error concealment system and method which detects drum-like beat patterns of music signals on the encoder side of the system and embeds the beat information as data ancillary to a preceding audio data interval in the transmitted compressed bitstream. The embedded information is then used to perform an error concealment task on the decoder side of the system. The beat detector functions as part of an error concealment system in an audio decoding section used in audio information transfer and audio download-streaming system terminal devices such as mobile phones. The disclosed sender-based method improves error concealment performance while reducing decoder complexity.

Claims

1. A method comprising:

(a) formatting a stream of audio data provided by a audio source into a sequence of audio data intervals;

(b) transform encoding the sequence of audio data intervals to form a sequence of encoded audio data intervals, each of the encoded audio data intervals having a plurality of transform coefficients;

(c) analyzing the transform coefficients of the sequence of encoded audio data intervals in the sequence so as to identify encoded transient audio data intervals, each of the encoded transient audio data intervals including a short transient signal having first transient signal characteristics; and

(d) embedding ancillary data into encoded audio data intervals preceding the encoded transient audio data intervals, the ancillary data providing notification that the encoded transient audio data intervals include the short transient signals.

2. A method as in claim 1 wherein the audio data intervals are formatted as pulse code modulation data.

3. A method as in claim 1 wherein step (b) comprises applying a modified discrete cosine transform to the sequence of audio data intervals.

4. A method as in claim 1 wherein step (b) comprises applying a shifted discrete Fourier transform to the sequence of audio data intervals.

5. A method as in claim 1 further comprising the steps of:

sending the encoded audio data intervals having the ancillary information to a receiver; and

subsequently sending the encoded transient audio data intervals to the receiver.

6. A method as in claim 1 wherein the short transient signal comprises a drumbeat.

7. A method as in claim 1 further comprising the steps of:

(e) analyzing the sequence of encoded audio data intervals to identify encoded transient audio data intervals which include a short transient signal having second transient signal characteristics; and

(f) embedding a second type of ancillary data into encoded audio data intervals of the sequence that precede the encoded transient audio data intervals including the short transient signal having second transient signal characteristics, the second type of ancillary data providing notification of the encoded transient audio data intervals including the short transient signal having second transient signal characteristics.

8. A method as in claim 1 wherein

each of the encoded audio data intervals has a plurality of frequency domain transform coefficients, and

step (c) comprises analyzing the frequency domain transform coefficients of the sequence of encoded audio data intervals to identify encoded transient audio data intervals.

9. A method as in claim 1 wherein the ancillary data in each of the preceding encoded audio data intervals is distinct from data identifying a sampling window applicable to the encoded audio data transient interval for which that ancillary data provides notification.

10. A method as in claim 1 wherein step (c) comprises, as to each of the encoded audio data intervals in the sequence, performing a frequency analysis on the transform coefficients to detect the short transient signal.

11. A method as in claim 10 wherein performing a frequency analysis comprises applying a shifted discrete Fourier transform.

12. A method as in claim 10 wherein performing a frequency analysis comprises extracting a feature vector from the transform coefficients.

13. A method as in claim 12 wherein the feature vector comprises a member of a group consisting of a primitive band energy value, an element-to-mean ratio of band energy, and a differential band energy value.

14. A method comprising:

(a) receiving transform-encoded audio data intervals of a sequence of transform-encoded audio data intervals, each of the intervals having a plurality of transform coefficients, wherein less than all of the intervals are transient intervals, and wherein each of the transient intervals corresponds to an audio segment that includes a beat;

(b) receiving ancillary data identifying the transient intervals;

(c) identifying transient intervals of the sequence that are defective; and

(d) replacing transform coefficients of the defective transient intervals with transform coefficients from received transient intervals not identified as defective.

15. A method as in claim 14 wherein step (c) comprises determining whether a transient interval of the sequence is corrupted or missing.

16. A method as in claim 14 wherein step (d) comprises, as to each of the defective transient intervals, replacing the transform coefficients of the defective transient interval with transform coefficients from a previously-received transient interval.

17. A method as in claim 14 wherein step (d) comprises, as to each of the defective transient intervals,

(d1) matching a window type of the defective transient interval with a window type of a received transient interval not identified as defective, and

(d2) replacing transform coefficients of the defective transient interval with transform coefficients from the matching received non-defective transient interval.

18. A method as in claim 14, further comprising the step of:

(e) identifying each of multiple transient intervals received in step (a) by a type of beat in the audio segment to which that transient interval corresponds, and wherein step (d) comprises, as to each of the defective transient intervals,

(d1) matching the beat type of the defective transient interval with the beat type of a non-defective received transient interval, and

(d2) replacing transform coefficients of the defective transient interval with transform coefficients from the matching non-defective received transient interval.

19. A method as in claim 14, further comprising the step of:

(e) identifying each of multiple transient intervals received in step (a) by a type of beat in the audio segment to which that transient interval corresponds, and wherein step (d) comprises, as to each of the defective transient intervals,

(d1) matching a window type and the beat type of the defective transient interval with a window type and the beat type of a non-defective received transient interval,

(d2) replacing transform coefficients of the defective transient interval with transform coefficients from the matching non-defective received transient interval.

20. A method as in claim 14 further comprising the step of:

(e) converting received intervals not identified as defective and the intervals having replacement coefficients to formatted audio samples.

21. A method as in claim 20 wherein the formatted audio samples are pulse code modulation formatted.

22. A method as in claim 14 wherein step (d) comprises, as to each of the defective transient intervals,

(d1) replacing transform coefficients for a low-frequency band and for a high-frequency band with transform coefficients from a received transient interval not identified as defective, and

(d2) replacing transform coefficients for a mid-frequency band with transform coefficients from a received interval other than the interval supplying the replacement coefficients in step (d1).

23. A method as in claim 22, further comprising the steps of:

(e) as to each of the defective transient intervals,

(e1) inversely transforming the mid-frequency band replaced coefficients to a time domain component,

(e2) inversely transforming the low-frequency and high-frequency band replaced coefficients to a time domain component, and

(e3) constructing a replacement signal in the time domain corresponding to the defective transient interval by weighting and combining the time domain components of steps (e1) and (e2).

24. A method as in claim 23, wherein step (e3) comprises weighting the time domain components of steps (e1) and (e2) using triangle functions.

25. A device comprising:

a decoder configured to perform steps that include

(a) receiving transform-encoded audio data intervals of a sequence of transform-encoded audio data intervals, each of the intervals having a plurality of transform coefficients, wherein less than all of the intervals are transient intervals, and wherein each of the transient intervals corresponds to an audio segment that includes a beat,

(b) retrieving ancillary data identifying the transient intervals, and

(c) identifying transient intervals of the sequence that are defective; and

an error concealment unit configured to perform a step that includes

(d) providing replacement transform coefficients for defective transient intervals, wherein the replacement transform coefficients are obtained from received transient intervals not identified as defective, and

wherein the decoder is further configured to perform steps that include

(e) as to each of the defective transient intervals,

(e1) matching a window type of the defective transient interval with a window type of a received transient interval not identified as defective, and

(e2) replacing transform coefficients of the defective transient interval with transform coefficients from the matching non-defective received transient interval.

26. A device as in claim 25 further comprising a buffer for storing at least one received transient interval not identified as defective.

27. A device as in claim 26 wherein step (d) comprises, as to each of the defective transient intervals, providing replacement transform coefficients obtained from a previously-received transient interval.

28. A system comprising:

an audio source for providing audio streaming information, the audio source including an encoder for converting the audio streaming information into a sequence of coded audio data intervals, each of the coded audio data intervals having a plurality of frequency domain transform coefficients, and a transient detector for classifying, by analysis of frequency domain transform coefficients, coded audio data intervals of the sequence that have a short transient signal as transient coded audio data intervals; and

a receiving terminal for converting the sequence of coded audio data intervals into the audio sample, the receiving terminal including an error concealment unit for replacing frequency domain transform coefficients of a defective transient audio data interval with frequency domain transform coefficients from a received transient audio data interval found to be error-free.

29. An error concealment system as in claim 28 wherein the receiving terminal further comprises a decoder for decoding the sequence of coded audio data intervals.

30. An error concealment system as in claim 28 wherein the audio source comprises a member of the group consisting of a server unit, a microphone, a personal digital assistant, and a mobile phone.

31. An error concealment system as in claim 28 wherein the receiving terminal comprises a member of the group consisting of a mobile phone, a personal digital assistant, and a computer.

32. An error concealment system as in claim 28 further comprising a telecommunications network connecting the receiving terminal with the audio source.

33. An error concealment system as in claim 32 wherein the telecommunications network comprises a wired network suitable for access by a telephone.

34. An error concealment system as in claim 32 wherein the telecommunications network comprises a member of the group consisting of a Global System for Mobile Communications (GSM), a General Packet Radio Service (GPRS), a Wideband CDMA (WCDMA), a DECT, a wireless LAN (WLAN), and a Universal Mobile Telecommunications System (UMIS).

35. A device comprising:

a decoder configured to perform steps that include

(a) receiving transform-encoded audio data intervals of a sequence of transform-encoded audio data intervals, each of the intervals having a plurality of transform coefficients, wherein less than all of the intervals are transient intervals, and wherein each of the transient intervals corresponds to an audio segment that includes a beat,

(b) retrieving ancillary data identifying the transient intervals, and

(c) identifying transient intervals of the sequence that are defective; and

an error concealment unit configured to perform a step that includes

(d) providing replacement transform coefficients for defective transient intervals, wherein the replacement transform coefficients are obtained from received transient intervals not identified as defective, and

wherein the decoder is further configured to perform steps that include

(e) identifying each of multiple transient intervals received in step (a) by a type of beat in the audio segment to which that transient interval corresponds, and

(f) as to each of the defective transient intervals,

(f1) matching the beat type of the defective transient interval with the beat type of a non-defective received transient interval, and

(f3) replacing transform coefficients of the defective transient interval with transform coefficients from the matching non-defective received transient interval.

36. A device comprising:

a decoder configured to perform steps that include

(a) receiving transform-encoded audio data intervals of a sequence of transform-encoded audio data intervals, each of the intervals having a plurality of transform coefficients, wherein less than all of the intervals are transient intervals, and wherein each of the transient intervals corresponds to an audio segment that includes a beat,

(b) retrieving ancillary data identifying the transient intervals, and

(c) identifying transient intervals of the sequence that are defective; and

an error concealment unit configured to perform a step that includes

(d) providing replacement transform coefficients for defective transient intervals, wherein the replacement transform coefficients are obtained from received transient intervals not identified as defective, and

wherein the decoder is further configured to perform steps that include

(e) identifying each of multiple transient intervals received in step (a) by a type of beat in the audio segment to which that transient interval corresponds, and

(f) as to each of the defective transient intervals,

(f1) matching a window type and the beat type of the defective transient interval with a window type and the beat type of a non-defective received transient interval,

(f2) replacing transform coefficients of the defective transient interval with transform coefficients from the matching non-defective received transient interval.

37. A device comprising:

a decoder configured to perform steps that include

(a) receiving transform-encoded audio data intervals of a sequence of transform-encoded audio data intervals, each of the intervals having a plurality of transform coefficients, wherein less than all of the intervals are transient intervals, and wherein each of the transient intervals corresponds to an audio segment that includes a beat,

(b) retrieving ancillary data identifying the transient intervals, and

(c) identifying transient intervals of the sequence that are defective; and

an error concealment unit configured to perform a step that includes

(d) providing replacement transform coefficients for defective transient intervals, wherein the replacement transform coefficients are obtained from received transient intervals not identified as defective, and

wherein the decoder is further configured to perform steps that include

(e) as to each of the defective transient intervals,

(e1) replacing transform coefficients for a low-frequency band and for a high-frequency band with transform coefficients from a received transient interval not identified as defective, and

(e2) replacing transform coefficients for a mid-frequency band with transform coefficients from a received interval other than the interval supplying the replacement coefficients in step (e1).

38. A device as in claim 37 wherein the decoder is further configured to perform steps that include

(e) as to each of the defective transient intervals,

(e3) inversely transforming the mid-frequency band replaced coefficients to a time domain component,

(e4) inversely transforming the low-frequency and high-frequency band replaced coefficients to a time domain component, and

(e5) constructing a replacement signal in the time domain corresponding to the defective transient interval by weighting and combining the time domain components of steps (e3) and (e4).

39. A device as in claim 38, wherein step (e5) comprises weighting the time domain components of steps (e3) and (e4) using triangle functions.