Method and Device for Sound Activity Detection and Sound Signal Classification

• US 20110035213A1
• Filed: 06/20/2008
• Published: 02/10/2011
• Est. Priority Date: 06/22/2007
• Status: Active Grant

• Current Assignee: VoiceAge EVS LLC (FIG LLC (d/b/a Fortress Investment Group LLC))
• Original Assignee: VoiceAge Corporation
• Inventors: Malenovsky, Vladimir, Salami, Redwan, Jelinek, Milan, Vaillancourt, Tommmy
• Granted Patent: US 8,990,073 B2
• Time in Patent Office: Days
• US Class Current: 704/208
• CPC Class Codes: G10L 19/22 Mode decision, i.e. based on audio signal content versus external parameters
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

A device and method for estimating a tonality of a sound signal comprise: calculating a current residual spectrum of the sound signal; detecting peaks in the current residual spectrum; calculating a correlation map between the current residual spectrum and a previous residual spectrum for each detected peak; and calculating a long-term correlation map based on the calculated correlation map, the long-term correlation map being indicative of a tonality in the sound signal.

Claims

1. A method for estimating a tonality of a sound signal the method comprising:

calculating a current residual spectrum of the sound signal;

detecting peaks in the current residual spectrum;

calculating a correlation map between the current residual spectrum and a previous residual spectrum for each detected peak; and

calculating a long-term correlation map based on the calculated correlation map, the long-term correlation map being indicative of a tonality in the sound signal.

2. A method as defined in claim 1, wherein calculating the current residual spectrum comprises:

searching for minima in the spectrum of the sound signal in a current frame;

estimating a spectral floor by connecting the minima with each other; and

subtracting the estimated spectral floor from the spectrum of the sound signal in the current frame so as to produce the current residual spectrum.

3. A method as defined in claim 1, wherein detecting the peaks in the current residual spectrum comprises locating a maximum between each pair of two consecutive minima.

4. A method as defined in claim 1, wherein calculating the correlation map comprises:

for each detected peak in the current residual spectrum, calculating a normalized correlation value with the previous residual spectrum, over frequency bins between two consecutive minima in the current residual spectrum that delimit the peak; and

assigning a score to each detected peak, the score corresponding to the normalized correlation value; and

for each detected peak, assigning the normalized correlation value of the peak over the frequency bins between the two consecutive minima that delimit the peak so as to form the correlation map.

5. A method as defined in claim 1, wherein calculating the long-term correlation map comprises:

filtering the correlation map through an one-pole filter on a frequency bin by frequency bin basis; and

summing the filtered correlation map over the frequency bins so as to produce a summed long-term correlation map.

6. A method as defined in claim 1, further comprising detecting strong tones in the sound signal.

7. A method as defined in claim 6, wherein detecting the strong tones in the sound signal comprises searching in the correlation map for frequency bins having a magnitude that exceeds a given fixed threshold.

8. A method as defined in claim 6, wherein detecting the strong tones in the sound signal comprises comparing the summed long-term correlation map with an adaptive threshold indicative of sound activity in the sound signal.

9. A method as defined in claim 1, further comprising verification of a presence of strong tones.

10. A method for detecting sound activity in a sound signal, wherein the sound signal is classified as one of an inactive sound signal and an active sound signal according to the detected sound activity in the sound signal, the method comprising:

estimating a parameter related to a tonality of the sound signal used for distinguishing a music signal from a background noise signal;

wherein the tonality estimation is performed according to claim 1.

11. A method as defined in claim 10, further comprising preventing update of noise energy estimates when a tonal sound signal is detected.

12. A method as defined in claim 10, wherein detecting the sound activity in the sound signal further comprises using a signal-to-noise ratio (SNR)-based sound activity detection.

13. A method as defined in claim 12, wherein using the signal-to-noise ratio (SNR)-based sound activity detection comprises detecting the sound signal based on a frequency dependent signal-to-noise ratio (SNR).

14. A method as defined in claim 12, wherein using the signal-to-noise ratio (SNR)-based sound activity detection comprises comparing an average signal-to noise-ratio (SNR_av) to a threshold calculated as a function of a long-term signal-to noise-ratio (SNR_LT).

15. A method as defined in claim 14, wherein using the signal-to-noise ratio (SNR)-based sound activity detection in the sound signal further comprises using noise energy estimates calculated in a previous frame in a SNR calculation.

16. A method as defined in claim 15, wherein using the signal-to-noise ratio (SNR)-based sound activity detection further comprises updating the noise estimates for a next frame.

17. A method as defined in claim 16, wherein updating the noise energy estimates for a next frame comprises calculating an update decision based on at least one of a pitch stability, a voicing, a non-stationarity parameter of the sound signal and a ratio between a second order and a sixteenth order of linear prediction residual error energies.

18. A method as defined in claim 14, comprising classifying the sound signal as one of an inactive sound signal and active sound signal, which comprises determining an inactive sound signal when the average signal-to-noise ratio (SNR_av) is inferior to the calculated threshold.

19. A method as defined in claim 14, comprising classifying the sound signal as one of an inactive sound signal and active sound signal, which comprises determining an active sound signal when the average signal-to-noise ratio (SNR_av) is larger than the calculated threshold.

20. A method as defined in claim 10, wherein estimating the parameter related to the tonality of the sound signal prevents updating of noise energy estimates when a music signal is detected.

21. A method as defined in claim 10, further comprising calculating a complementary non-stationarity parameter and a noise character parameter in order to distinguish a music signal from a background noise signal and prevent update of noise energy estimates on the music signal.

22. A method as defined in claim 21, wherein calculating the complementary non-stationarity parameter comprises calculating a parameter similar to a conventional non-stationarity with resetting a long-term energy when a spectral attack is detected.

23. A method as defined in claim 22, wherein resetting the long-term energy comprises setting the long-term energy to a current frame energy.

24. A method as defined in claim 22, wherein detecting the spectral attack and resetting the long-term energy comprises calculating a spectral diversity parameter.

25. A method as defined in claim 24, wherein calculating the spectral diversity parameter comprises:

calculating a ratio between an energy of the sound signal in a current frame and an energy of the sound signal in a previous frame, for frequency bands higher than a given number; and

calculating the spectral diversity as a weighted sum of the computed ratio over all the frequency bands higher than the given number.

26. A method as defined in claim 22, wherein calculating the complementary non-stationarity parameter further comprises calculating an activity prediction parameter indicative of an activity of the sound signal.

27. A method as defined in claim 26, wherein calculating the activity prediction parameter comprises:

calculating a long-term value of a binary decision obtained from estimating the parameter related to the tonality of the sound signal and the conventional non-stationarity parameter.’

28. A method as defined in claim 21, wherein the update of the noise energy estimates is prevented in response to having simultaneously the activity prediction parameter larger than a first given fixed threshold and the complementary non-stationarity parameter larger than a second given fixed threshold

29. A method as defined in claim 21, wherein calculating the noise character parameter comprises:

dividing a plurality of frequency bands into a first group of a certain number of first frequency bands and a second group of a rest of the frequency bands;

calculating a first energy value for the first group of frequency bands and a second energy value of the second group of frequency bands;

calculating a ratio between the first and second energy values so as to produce the noise character parameter; and

calculating a long-term value of the noise character parameter based on the calculated noise character parameter.

30. A method as defined in claim 29, wherein the update of the noise energy estimates is prevented in response to having the noise character parameter inferior than a given fixed threshold.

31. A method for classifying a sound signal in order to optimize encoding of the sound signal using the classification of the sound signal, the method comprising:

detecting a sound activity in the sound signal;

classifying the sound signal as one of an inactive sound signal and an active sound signal according to the detected sound activity in the sound signal; and

in response to the classification of the sound signal as an active sound signal, further classifying the active sound signal as one of an unvoiced speech signal and a non-unvoiced speech signal;

wherein classifying the active sound signal as an unvoiced speech signal comprises estimating a tonality of the sound signal in order to prevent classifying music signals as unvoiced speech signals, wherein the tonality estimation is performed according to claim 1.

32. A method as defined in claim 31, further comprising encoding the sound signal according to the classification of the sound signal.

33. A method as defined in claim 32, wherein encoding the sound signal according to the classification of the sound signal comprises encoding the inactive sound signal using comfort noise generation.

34. A method as defined in claim 31, wherein classifying the active sound signal as an unvoiced speech signal comprises calculating a decision rule based on at least one of a voicing measure, an average spectral tilt measure, a maximum short-time energy increase at low level, a tonal stability and a relative frame energy.

35. A method as defined in claim 31, further comprising classifying the non-unvoiced speech signal as one of a stable voiced speech signal and another type of signal different from the stable voiced speech signal.

36. A method as defined in claim 35, wherein classifying the non-unvoiced speech signal as the stable voiced speech signal comprises calculating a decision rule based on at least one of a normalized correlation, an average spectral tilt and an open-loop pitch estimates of the sound signal.

37. A method for encoding a higher band of a sound signal using a classification of the sound signal, the method comprising:

classifying the sound signal as one of a tonal sound signal and a non-tonal sound signal;

wherein classifying the sound signal as a tonal signal comprises estimating a tonality of the sound signal according to claim 1.

38. A method as defined in claim 37, wherein estimating the tonality of the sound signal comprises using an alternative method for calculating a spectral floor.

39. A method as defined in claim 38, wherein using the alternative method for calculating the spectral floor comprises filtering a log-energy spectrum of the sound signal in a current frame using a moving-average filter.

40. A method as defined in claim 37, wherein estimating the tonality of the sound signal comprises smoothing the residual spectrum by means of a short-time moving-average filter.

41. A method as defined in claim 37, further comprising encoding the higher band of the sound signal according to the classification of said sound signal.

42. A method as defined in claim 41, wherein encoding the higher band of the sound signal according to the classification of said sound signal comprises encoding the tonal sound signals using a model optimized for such signals.

43. A method as defined in claim 37, wherein the higher band of the sound signal comprises a frequency range above 7 kHz.

44. A device for estimating a tonality of a sound signal, the device comprising:

means for calculating a current residual spectrum of the sound signal;

means for detecting peaks in the current residual spectrum;

means for calculating a correlation map between the current residual spectrum and a previous residual spectrum for each detected peak; and

means for calculating a long-term correlation map based on the calculated correlation map, the long-term correlation map being indicative of a tonality in the sound signal.

49. A device for detecting sound activity in a sound signal, wherein the sound signal is classified as one of an inactive sound signal and an active sound signal according to the detected sound activity in the sound signal, the device comprising:

means for estimating a parameter related to a tonality of the sound signal used for distinguishing a music signal from a background noise signal;

wherein the tonality parameter estimation means comprises a device according to claim 44.

45. A device for estimating a tonality of a sound signal, the device comprising:

a calculator of a current residual spectrum of the sound signal;

a detector of peaks in the current residual spectrum;

a calculator of a correlation map between the current residual spectrum and a previous residual spectrum for each detected peak; and

a calculator of a long-term correlation map based on the calculated correlation map, the long-term correlation map being indicative of a tonality in the sound signal.

46. A device as defined in claim 45, wherein the calculator of the current residual spectrum comprises:

a locator of minima in the spectrum of the sound signal in a current frame;

an estimator of a spectral floor which connects the minima with each other; and

a subtractor of the estimated spectral floor from the spectrum so as to produce the current residual spectrum.

47. A device as defined in claim 45, wherein the calculator of the long-term correlation map comprises:

a filter for filtering the correlation map on a frequency bin by frequency bin basis; and

an adder for summing the filtered correlation map over the frequency bins so as to produce a summed long-term correlation map.

48. A device as defined in claim 45, further comprising a detector of strong tones in the sound signal.

50. A device for detecting sound activity in a sound signal, wherein the sound signal is classified as one of an inactive sound signal and an active sound signal according to the detected sound activity in the sound signal, the device comprising:

a tonality estimator of the sound signal, used for distinguishing a music signal from a background noise signal;

wherein the tonality estimator comprises a device according to claim 45.

51. A device as defined in claim 50, further comprising a signal-to-noise ratio (SNR)-based sound activity detector.

52. A device as defined in claim 51, wherein the (SNR)-based sound activity detector comprises a comparator of an average signal to noise ratio (SNR_av) with a threshold which is a function of a long-term signal to noise ratio (SNR_LT).

53. A device as defined in claim 50, further comprising a noise estimator for updating noise energy estimates in a calculation of a signal-to-noise ratio (SNR) in the SNR-based sound activity detector.

54. A device as defined in claim 50, further comprising a calculator of a complementary non-stationarity parameter and a calculator of a noise character of the sound signal for distinguishing a music signal from a background noise signal and preventing update of noise energy estimates.

55. A device as defined in claim 50, further comprising a calculator of a spectral parameter used for detecting spectral changes and spectral attacks in the sound signal.

56. A device for classifying a sound signal in order to optimize encoding of the sound signal using the classification of the sound signal, the device comprising:

means for detecting a sound activity in the sound signal;

means for classifying the sound signal as one of an inactive sound signal and active sound signal according to the detected sound activity in the sound signal; and

in response to the classification of the sound signal as an active sound signal, means for further classifying the active sound signal as one of an unvoiced speech signal and a non-unvoiced speech signal;

wherein the means for further classifying the sound signal as an unvoiced speech signal comprises means for estimating a parameter related to a tonality of the sound signal in order to prevent classifying music signals as unvoiced speech signals wherein the means for estimating the tonality related parameter comprises a device according to claim 45.

57. A device for classifying a sound signal in order to optimize encoding of the sound signal using the classification of the sound signal, the device comprising:

a detector of sound activity in the sound signal;

a first sound signal classifier for classifying the sound signal as one of an inactive sound signal and an active sound signal according to the detected sound activity in the sound signal; and

a second sound signal classifier in connection with the first sound signal classifier for classifying the active sound signal as one of an unvoiced speech signal and a non-unvoiced speech signal;

wherein the sound activity detector comprises a tonality estimator for estimating a tonality of the sound signal in order to prevent classifying music signals as unvoiced speech signals, wherein the tonality estimator comprises a device according to claim 45.

58. A device as defined in claim 57, further comprising a sound encoder for encoding the sound signal according to the classification of the sound signal.

59. A device as defined in claim 58, wherein the sound encoder comprises a noise encoder for encoding inactive sound signals.

60. A device as defined in claim 58, wherein the sound encoder comprises an unvoiced speech optimized coder.

61. A device as defined in claim 58, wherein the sound encoder comprises a voiced speech optimized coder for coding stable voiced signals.

62. A device as defined in claim 58, wherein the sound encoder comprises a generic sound signal coder for coding fast evolving voiced signals.

63. A device for encoding a higher band of a sound signal using a classification of the sound signal, the device comprising:

means for classifying the sound signal as one of a tonal sound signal and a non-tonal sound signal; and

means for encoding the higher band of the classified sound signal;

wherein the means for classifying the sound signal as a tonal signal comprises a device for estimating a tonality of the sound signal according to claim 45.

64. A device for encoding a higher band of a sound signal using a classification of the sound signal, the device comprising:

a sound signal classifier to classify the sound signal as one of a tonal sound signal and a non-tonal sound signal; and

a sound encoder for encoding the higher band of the classified sound signal;

wherein the sound signal classifier comprises device for estimating a tonality of the sound signal according to claim 45.

65. A device as defined in claim 64, further comprising a moving-average filter for calculating a spectral floor derived from the sound signal, wherein the spectral floor is used in estimating the tonality of the sound signal.

66. A device as defined in claim 64, further comprising a, short-time moving-average filter for smoothing a residual spectrum of the sound signal, wherein the residual spectrum is used in estimating