Systems, methods, apparatus, and computer-readable media for coherence detection

• US 8,724,829 B2
• Filed: 10/23/2009
• Issued: 05/13/2014
• Est. Priority Date: 10/24/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Visser, Erik, Liu, Ian Ernan
• Primary Examiner(s): Lao, Lun-See
• Application Number:
  US12/605,158
• Publication Number:
  US 20110038489A1
• Time in Patent Office: 1,663 Days
• Field of Search: 381 97- 99, 381102-104, 381/91, 381/92, 381 941- 943, 381/122, 381/66, 381/17, 381/56, 700/94, 702/190
• US Class Current: 381/97
• CPC Class Codes: G01S 3/8006 Multi-channel systems specially adapted for direction-finding, i.e. having a single aerial system capable of giving simultaneous indications of the directions of different signals
• CPC Class Codes: G01S 3/86 with means for eliminating undesired waves, e.g. disturbing noises
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise

Abstract

Based on phase differences between corresponding frequency components of different channels of a multichannel signal, a measure of directional coherency is calculated. Application of such a measure to voice activity detection and noise reduction are also disclosed.

Claims

1. A method of processing a multichannel signal, said method comprising:

for each of a plurality of different frequency components of the multichannel signal, calculating a difference between a phase of the frequency component in a first channel of the multichannel signal and a phase of the frequency component in a second channel of the multichannel signal;

based on information from the plurality of calculated phase differences, calculating (A) a value of a first coherency measure that indicates a degree of coherence among directions of arrival of at least the plurality of different frequency components in a first spatial sector and (B) a value of a second coherency measure that indicates a degree of coherence among the directions of arrival of at least the plurality of different frequency components in a second spatial sector that is different than the first spatial sector;

calculating a contrast of the first coherency measure by evaluating a relation between the calculated value of the first coherency measure and an average value of the first coherency measure over time;

calculating a contrast of the second coherency measure by evaluating a relation between the calculated value of the second coherency measure and an average value of the second coherency measure over time; and

indicating one among the first and second spatial sectors, based on which among the first and second coherency measures has the greatest contrast,

wherein each of said first and second channels of the multichannel signal is based on a signal produced by a corresponding one of a pair of microphones, and

wherein at least one among said calculating the difference, said calculating the value of the first coherency measure, said calculating the value of the second coherency measure, said calculating the contrast of the first coherency measure, and said calculating the contrast of the second coherency measure is performed by at least one processor.

2. A method according to claim 1, wherein the plurality of different frequency components includes at least one frequency component having a frequency not greater than one kilohertz and at least one frequency component having a frequency not less than fifteen hundred Hertz.

3. A method according to claim 1, wherein the plurality of different frequency components includes at least one frequency component having a frequency not greater than eight hundred Hertz and at least one frequency component having a frequency not less than eighteen hundred Hertz.

4. A method according to claim 1, wherein said method includes selecting the plurality of different frequency components based on an estimated pitch frequency of the multichannel signal.

5. A method according to claim 1, wherein said method includes:

at a time subsequent to said indicating, detecting that a value of the coherency measure that corresponds to the indicated sector is not greater than a corresponding threshold value; and

in response to said detecting, varying an amplitude of at least one frequency component of at least one channel of the multichannel signal according to a specified noise spectral profile.

6. A method according to claim 1, wherein said method includes, based on said indicating one among the first and second spatial sectors and on information from at least one frequency component of the multichannel signal whose direction of arrival is not within the indicated spatial sector, updating an estimate of a noise component of at least one channel of the multichannel signal.

7. A method according to claim 1, wherein said method comprises calculating, for each of a plurality of different frequency components of a second multichannel signal, a difference between a phase of the frequency component in a first channel of the second multichannel signal and a phase of the frequency component in a second channel of the second multichannel signal to obtain a second plurality of calculated phase differences, and

wherein each of said first and second channels of the second multichannel signal is based on a signal produced by a corresponding one of a second pair of microphones that is different from said pair of microphones, and

wherein at least one of said calculating the value of the first coherency measure and said calculating the value of the second coherency measure is based on information from the second plurality of calculated phase differences, a direction specification device to specify, when the external mode is set, a direction of an emission source of the external sound with respect to the user, based at least in part on the stereo external signal acquired by the external signal acquisition device;

a fade control device to control an operation to generate the stereo output signal by the output signal generation device in accordance with the operation mode set by the mode setting device and the direction of the emission source of the external sound specified by the direction specification device, wherein when the external mode is set and the direction of the emission source of the external sound is specified to correspond with the first direction, the fade control device causes the output signal generation device to generate the first output signal from at least the stereo external signal, and also to generate the second output signal from at least the stereo audio signal, and

when the external mode is set, the fade control device causes the output signal generation device to generate the first output signal from the stereo external signal only, and to generate the second output signal by converting the stereo audio signal to a monaural audio signal.

8. A method according to claim 1, wherein said calculating the value of the first coherency measure comprises calculating, for each of the plurality of calculated phase differences, a corresponding direction indicator, and

wherein each of the plurality of direction indicators indicates at least one of (A) a direction of arrival of the corresponding frequency component, (B) a time delay of arrival of the corresponding frequency component, and (C) a ratio between the calculated phase difference and the frequency of the corresponding frequency component, and

wherein said value of the first coherency measure is based on the plurality of direction indicators.

9. A method according to claim 8, wherein said calculating the value of the first coherency measure comprises, for each of the plurality of direction indicators, rating the direction indicator according to a directional masking function to obtain a corresponding mask score, and

wherein said value of the first coherency measure is based on the plurality of mask scores.

10. A method according to claim 9, wherein said method includes varying an amplitude of at least one of the plurality of frequency components of at least one channel of the multichannel signal, based on the corresponding mask score, to produce a masked signal.

11. A method according to claim 9, wherein said method includes:

based on the mask scores of at least some of the plurality of direction indicators that correspond to frequency components in a specified subband of the multichannel signal, calculating a subband mask score; and

varying amplitudes of frequency components of at least the specified subband of at least one channel of the multichannel signal, based on the subband mask score, to produce a masked signal.

12. A method according to claim 10, wherein said method includes attenuating the masked signal when a ratio of a level of the masked signal and a level of the at least one channel is small.

13. A method according to claim 1, wherein said indicating one among the first and second spatial sectors is based on a state of a relation between (A) said calculated value of said coherency measure that has the greatest contrast and (B) a threshold value that corresponds to said coherency measure;

and

wherein the threshold value that corresponds to the first coherency measure differs from the threshold value that corresponds to the second coherency measure.

14. A non-transitory computer-readable medium comprising instructions which when executed by at least one processor cause the at least one processor to perform a method according to claim 1.

15. A method according to claim 13, wherein said first spatial sector includes an axis of the pair of microphones and said second spatial sector includes a direction orthogonal to said axis, and

wherein the threshold value that corresponds to the first coherency measure is less than the threshold value that corresponds to the second coherency measure.

16. A method according to claim 13, wherein said method comprises, at a time subsequent to said indicating one among the first and second spatial sectors, indicating the other among the first and second spatial sectors in response to detecting that said other coherency measure has the greatest contrast, and

wherein said indicating the other spatial sector occurs only if a value of said other coherency measure at said subsequent time is not less than said threshold value that corresponds to said other coherency measure.

17. A method according to claim 1, wherein said method comprises:

applying a null beam to at least two channels of the multichannel signal, wherein a direction of said beam is based on said indicated one among the first and second spatial sectors; and

based on a result of said applying a null beam, updating an estimate of a noise component of the multichannel signal.

18. An apparatus for processing a multichannel signal, said apparatus comprising:

means for calculating, for each of a plurality of different frequency components of the multichannel signal, a difference between a phase of the frequency component in a first channel of the multichannel signal and a phase of the frequency component in a second channel of the multichannel signal;

means for calculating, based on information from the plurality of calculated phase differences, (A) a value of a first coherency measure that indicates a degree of coherence among directions of arrival of at least the plurality of different frequency components in a first spatial sector and (B) a value of a second coherency measure that indicates a degree of coherence among the directions of arrival of at least the plurality of different frequency components in a second spatial sector that is different than the first spatial sector;

means for calculating a contrast of the first coherency measure by evaluating a relation between the calculated value of the first coherency measure and an average value of the first coherency measure over time and for calculating a contrast of the second coherency measure by evaluating a relation between the calculated value of the second coherency measure and an average value of the second coherency measure over time; and

means for indicating one among the first and second spatial sectors, based on which among the first and second coherency measures has the greatest contrast,

wherein each of said first and second channels of the multichannel signal is based on a signal produced by a corresponding one of a pair of microphones, and

wherein at least one among said means for calculating the difference, said means for calculating the value of the first coherency measure and the value of the second coherency measure, and said means for calculating the contrast of the first coherency measure and the contrast of the second coherency measure is implemented by at least one processor.

19. An apparatus according to claim 18, wherein the plurality of different frequency components includes at least one frequency component having a frequency not greater than one kilohertz and at least one frequency component having a frequency not less than fifteen hundred Hertz.

20. An apparatus according to claim 18, wherein the plurality of different frequency components includes at least one frequency component having a frequency not greater than eight hundred Hertz and at least one frequency component having a frequency not less than eighteen hundred Hertz.

21. An apparatus according to claim 18, wherein said apparatus includes means for selecting the plurality of different frequency components based on an estimated pitch frequency of the multichannel signal.

22. An apparatus according to claim 18, wherein said apparatus includes:

means for detecting, at a time subsequent to said indicating, that a value of the coherency measure that corresponds to the indicated sector is not greater than a corresponding threshold value; and

means for varying, in response to said detecting, an amplitude of at least one frequency component of at least one channel of the multichannel signal according to a specified noise spectral profile.

23. An apparatus according to claim 18, wherein said apparatus includes means for updating, based on said indicating one among the first and second spatial sectors and on information from at least one frequency component of the multichannel signal whose direction of arrival is not within the indicated spatial sector, an estimate of a noise component of at least one channel of the multichannel signal.

24. An apparatus according to claim 18, wherein said apparatus comprises means for calculating, for each of a plurality of different frequency components of a second multichannel signal, a difference between a phase of the frequency component in a first channel of the second multichannel signal and a phase of the frequency component in a second channel of the second multichannel signal to obtain a second plurality of calculated phase differences, and

wherein each of said first and second channels of the second multichannel signal is based on a signal produced by a corresponding one of a second pair of microphones that is different from said pair of microphones, and

wherein said means for calculating the value of the first coherency measure and the value of the second coherency measure is configured to calculate the value of at least one of the first coherency measure and the second coherency measure based on information from the second plurality of calculated phase differences.

25. An apparatus according to claim 18, wherein said means for calculating the value of the first coherency measure is configured to calculate, for each of the plurality of calculated phase differences, a corresponding direction indicator, and

wherein each of the plurality of direction indicators indicates at least one of (A) a direction of arrival of the corresponding frequency component, (B) a time delay of arrival of the corresponding frequency component, and (C) a ratio between the calculated phase difference and the frequency of the corresponding frequency component, and

wherein said value of the first coherency measure is based on the plurality of direction indicators.

26. An apparatus according to claim 25, wherein said means for calculating the value of the first coherency measure comprises means for rating, for each of the plurality of direction indicators, the direction indicator according to a directional masking function to obtain a corresponding mask score, and

wherein said value of the first coherency measure is based on the plurality of mask scores.

27. An apparatus according to claim 26, wherein said apparatus includes means for varying an amplitude of at least one of the plurality of frequency components of at least one channel of the multichannel signal, based on the corresponding mask score, to produce a masked signal.

28. An apparatus according to claim 26, wherein said apparatus includes:

means for calculating, based on the mask scores of at least some of the plurality of direction indicators that correspond to frequency components in a specified subband of the multichannel signal, a subband mask score; and

means for varying amplitudes of frequency components of at least the specified subband of at least one channel of the multichannel signal, based on the subband mask score, to produce a masked signal.

29. An apparatus according to claim 27, wherein said apparatus includes means for attenuating the masked signal when a ratio of a level of the masked signal and a level of the at least one channel is small.

30. An apparatus according to claim 18, wherein said indicating one among the first and second spatial sectors is based on a state of a relation between (A) said calculated value of said coherency measure that has the greatest contrast and (B) a threshold value that corresponds to said coherency measure;

and

wherein the threshold value that corresponds to the first coherency measure differs from the threshold value that corresponds to the second coherency measure.

31. An apparatus according to claim 30, wherein said first spatial sector includes an axis of the pair of microphones and said second spatial sector includes a direction orthogonal to said axis, and wherein the threshold value that corresponds to the first coherency measure is less than the threshold value that corresponds to the second coherency measure.

32. An apparatus according to claim 30, wherein said means for indicating is configured to indicate, at a time subsequent to said indicating one among the first and second spatial sectors, the other among the first and second spatial sectors in response to detecting that said other coherency measure has the greatest contrast, and

wherein said means for indicating is configured to indicate the other spatial sector only if a value of said other coherency measure at said subsequent time is not less than said threshold value that corresponds to said other coherency measure.

33. An apparatus according to claim 18, wherein said apparatus comprises:

means for applying a null beam to at least two channels of the multichannel signal, wherein a direction of said beam is based on said indicated one among the first and second spatial sectors; and

means for updating an estimate of a noise component of the multichannel signal, based on an output of said means for applying a null beam.

34. An apparatus for processing a multichannel signal, said apparatus comprising:

a phase difference calculator configured to calculate, for each of a plurality of different frequency components of the multichannel signal, a difference between a phase of the frequency component in a first channel of the multichannel signal and a phase of the frequency component in a second channel of the multichannel signal; and

a coherency measure calculator configured to calculate, based on information from the plurality of calculated phase differences, (A) a value of a first coherency measure that indicates a degree of coherence among directions of arrival of at least the plurality of different frequency components in a first spatial sector and (B) a value of a second coherency measure that indicates a degree of coherence among the directions of arrival of at least the plurality of different frequency components in a second spatial sector that is different than the first spatial sector,

wherein said coherency measure calculator is configured to calculate a contrast of the first coherency measure by evaluating a relation between the calculated value of the first coherency measure and an average value of the first coherency measure over time and to calculate a contrast of the second coherency measure by evaluating a relation between the calculated value of the second coherency measure and an average value of the second coherency measure over time, and

wherein said coherency measure calculator is configured to indicate one among the first and second spatial sectors, based on which among the first and second coherency measures has the greatest contrast, and

wherein each of said first and second channels of the multichannel signal is based on a signal produced by a corresponding one of a pair of microphones.

35. An apparatus according to claim 34, wherein the plurality of different frequency components includes at least one frequency component having a frequency not greater than one kilohertz and at least one frequency component having a frequency not less than fifteen hundred Hertz.

36. An apparatus according to claim 34, wherein the plurality of different frequency components includes at least one frequency component having a frequency not greater than eight hundred Hertz and at least one frequency component having a frequency not less than eighteen hundred Hertz.

37. An apparatus according to claim 34, wherein at least one among said phase difference calculator and said coherency measure calculator is configured to select the plurality of different frequency components based on an estimated pitch frequency of the multichannel signal.

38. An apparatus according to claim 34, wherein said coherency measure calculator is configured to detect, at a time subsequent to said indicating, that a value of the coherency measure that corresponds to the indicated sector is not greater than a corresponding threshold value;

and

wherein said apparatus includes a spectrum modifier configured to vary, in response to said detecting, an amplitude of at least one frequency component of at least one channel of the multichannel signal according to a specified noise spectral profile.

39. An apparatus according to claim 34, wherein said apparatus includes a noise estimator configured to update, based on said indicating one among the first and second spatial sectors and on information from at least one frequency component of the multichannel signal whose direction of arrival is not within the indicated spatial sector, an estimate of a noise component of at least one channel of the multichannel signal.

40. An apparatus according to claim 34, wherein said apparatus comprises a second phase difference calculator configured to calculate, for each of a plurality of different frequency components of a second multichannel signal, a difference between a phase of the frequency component in a first channel of the second multichannel signal and a phase of the frequency component in a second channel of the second multichannel signal to obtain a second plurality of calculated phase differences, and

wherein each of said first and second channels of the second multichannel signal is based on a signal produced by a corresponding one of a second pair of microphones that is different from said pair of microphones, and

wherein said coherency measure calculator is configured to calculate the value of at least one of the first coherency measure and the second coherency measure based on information from the second plurality of calculated phase differences.

41. An apparatus according to claim 34, wherein said coherency measure calculator is configured to calculate, for each of the plurality of calculated phase differences, a corresponding direction indicator, and

wherein each of the plurality of direction indicators indicates at least one of (A) a direction of arrival of the corresponding frequency component, (B) a time delay of arrival of the corresponding frequency component, and (C) a ratio between the calculated phase difference and the frequency of the corresponding frequency component, and

wherein said value of the first coherency measure is based on the plurality of direction indicators.

42. An apparatus according to claim 41, wherein said coherency measure calculator is configured to rate, for each of the plurality of direction indicators, the direction indicator according to a directional masking function to obtain a corresponding mask score, and

wherein said value of the first coherency measure is based on the plurality of mask scores.

43. An apparatus according to claim 42, wherein said apparatus includes a masked signal generator configured to vary an amplitude of at least one of the plurality of frequency components of at least one channel of the multichannel signal, based on the corresponding mask score, to produce a masked signal.

44. An apparatus according to claim 42, wherein said apparatus includes a masked signal generator configured to calculate, based on the mask scores of at least some of the plurality of direction indicators that correspond to frequency components in a specified subband of the multichannel signal, a subband mask score, and to vary amplitudes of frequency components of at least the specified subband of at least one channel of the multichannel signal, based on the subband mask score, to produce a masked signal.

45. An apparatus according to claim 43, wherein said masked signal generator is configured to attenuate the masked signal when a ratio of a level of the masked signal and a level of the at least one channel is small.

46. An apparatus according to claim 34, wherein said coherency measure calculator is configured to indicate said one among the first and second spatial sectors based on a state of a relation between (A) said calculated value of said coherency measure that has the greatest contrast and (B) a threshold value that corresponds to said coherency measure,

wherein the threshold value that corresponds to the first coherency measure differs from the threshold value that corresponds to the second coherency measure.

47. An apparatus according to claim 46, wherein said first spatial sector includes an axis of the pair of microphones and said second spatial sector includes a direction orthogonal to said axis, and

wherein the threshold value that corresponds to the first coherency measure is less than the threshold value that corresponds to the second coherency measure.

48. An apparatus according to claim 46, wherein said coherency measure calculator is configured to indicate, at a time subsequent to said indicating one among the first and second spatial sectors, the other among the first and second spatial sectors in response to detecting that said other coherency measure has the greatest contrast, and

wherein said coherency measure calculator is configured to indicate the other spatial sector only if a value of said other coherency measure at said subsequent time is not less than said threshold value that corresponds to said other coherency measure.

49. An apparatus according to claim 34, wherein said apparatus comprises:

a null beamformer configured to apply a null beam to at least two channels of the multichannel signal, wherein a direction of said beam is based on said indicated one among the first and second spatial sectors; and

a noise estimator configured to update an estimate of a noise component of the multichannel signal, based on an output of said null beamformer.