Multi-channel speech enhancement

• US 10,339,949 B1
• Filed: 12/19/2017
• Issued: 07/02/2019
• Est. Priority Date: 12/19/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Dusan, Sorin V.
• Primary Examiner(s): Woo, Stella L.
• Application Number:
  US15/847,786
• Publication Number:
  US 20190189144A1
• Time in Patent Office: 560 Days
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 21/0364 for improving intelligibility
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H04R 1/1041 Mechanical or electronic switches, or control elements switches in general H01H
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2410/05 Noise reduction with a separate noise microphone
• CPC Class Codes: H04R 2430/25 Array processing for suppression of unwanted side-lobes in directivity characteristics, e.g. a blocking matrix
• CPC Class Codes: H04R 2460/13 Hearing devices using bone conduction transducers
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Speech enhancers suppress impairments in an acoustic signal. An audio appliance has a first microphone and a second microphone. The first microphone provides a first signal, and the second microphone provides a second signal. A voice-activity detector can determine a presence of user speech responsive to a combination of voice-activity cues, including a first level difference between the first signal and the second signal within a first frequency band, and a second level difference between the first signal and the second signal within a second frequency band. A noise suppressor suppresses impairments originating from a direction of, e.g., up to about 75-degrees from an axis extending from the second microphone to the first microphone. An output device can output a noise-suppressed output-signal corresponding to a determined presence or absence of speech by the voice-activity detector. The impairments can be suppressed by, e.g., between about 3 dB and about 20 dB.

Claims

1. An audio appliance, comprising:

a first microphone transducer to provide a first acoustic signal;

a second microphone transducer to provide a second acoustic signal, wherein the first microphone transducer and the second microphone transducer are spaced apart from each other and define a longitudinal axis;

a voice-activity detector configured to determine a presence or an absence of user speech responsive to a combination of voice-activity cues comprising, a first level difference between the first acoustic signal and the second acoustic signal within a first frequency band, and a second level difference between the first acoustic signal and the second acoustic signal within a second frequency band; and

a noise suppressor configured, responsive to a determined presence of speech by the voice-activity detector, to suppress in a noise-supressed output-signal impairments originating from a direction of up to about 75-degrees from the longitudinal axis by between about 3 dB and about 20 dB; and

an output device to output the noise-suppressed output-signal.

2. An audio appliance according to claim 1, wherein the combination of voice-activity cues further comprises a comparison of the first level difference to a first threshold and a comparison of the second level difference to a second threshold, wherein the voice-activity detector is to determine a presence of user speech responsive to the first level difference exceeding the first threshold and the second level difference exceeding the second threshold.

3. An audio appliance according to claim 2, wherein the comparison of the first level difference to the first threshold and the comparison of the second level difference to the second threshold comprises a first voice-activity cue, wherein the combination of voice-activity cues further comprises a second voice-activity cue comprising a comparison of the second level difference to a third threshold and a comparison of the first level difference to a fourth threshold.

4. An audio appliance according to claim 3, wherein the voice-activity detector determines a presence of user speech responsive to the second level difference exceeding the third threshold and the first level difference exceeding the fourth threshold.

5. An audio appliance according to claim 3, wherein the combination of voice-activity cues further comprises a level difference between a first beam-formed combination of the first acoustic signal with the second acoustic signal and a second beam-formed combination of the first acoustic signal with the second acoustic signal.

6. An audio appliance according to claim 5, wherein the voice-activity detector determines a presence of user speech responsive to the second level difference exceeding the third threshold, the first level difference exceeding the fourth threshold, and the level difference between the first beam-formed combination and the second beam-formed combination exceeding a fifth corresponding threshold level difference.

7. An audio appliance according to claim 2, wherein the comparison of the first level difference to the first threshold and the comparison of the second level difference to the second threshold comprises a first voice-activity cue, wherein the combination of voice-activity cues further comprises a second voice-activity cue comprising a comparison of an output from an accelerometer within a third frequency band to a third threshold, the second voice-activity cue further comprising a comparison of a fourth threshold to a level difference between a first beam-formed combination of the first acoustic signal with the second acoustic signal and a second beam-formed combination of the first acoustic signal with the second acoustic signal.

8. An audio appliance according to claim 1, wherein the combination of voice-activity cues further comprises a level of a first beam-formed combination of the first acoustic signal and the second acoustic signal within a third frequency band, a level of a second beam-formed combination of the first acoustic signal and the second acoustic signal within the third frequency band, a level difference between the first beam-formed combination and the second beam-formed combination within the third frequency band, or a combination thereof.

9. An audio appliance according to claim 1, further comprising an accelerometer, wherein the combination of voice-activity cues comprises an output from the accelerometer within a third frequency band.

10. An audio appliance according to claim 1, wherein the noise-suppressed output-signal comprises a synthesis of a plurality of gain-adjusted frequency bins of the first acoustic signal, the second acoustic signal, or a selected beam-formed combination of the first acoustic signal with the second acoustic signal, wherein each gain-adjusted frequency bin corresponds to a respective gain determined responsive to an estimate of noise-power in the respective frequency bin, and wherein the estimate of noise-power in the respective bin corresponds to the determined presence or absence of speech by the voice-activity detector.

11. An audio appliance comprising a first microphone transducer, a second microphone transducer, wherein the first microphone transducer and the second microphone transducer are spaced apart from each other to define a longitudinal axis, an output device, a processor, and a memory, wherein the memory contains instructions that, when executed by the processor, cause the audio appliance

to receive a first audio signal from the first microphone transducer and to receive a second audio signal from the second microphone transducer;

to determine a presence or an absence of speech responsive to a combination of voice-activity cues comprising a first level difference between the first acoustic signal and the second acoustic signal within a first frequency band and a second level difference between the first acoustic signal and the second acoustic signal within a second frequency band;

responsive to the determined presence of speech, to generate a noise-suppressed output-signal having between about 3 dB and about 20 dB attenuation to impairments originating from a direction of up to about 75-degrees from the longitudinal axis of the audio appliance; and

with the output device, to output flail the noise-suppressed output-signal.

12. An audio appliance according to claim 11, wherein the combination of voice-activity cues further comprises a comparison of the first level difference to a first threshold and a comparison of the second level difference to a second threshold, wherein, when executed by the processor, the instructions further cause the audio appliance to determine a presence of speech responsive to the first level difference exceeding the first threshold and the second level difference exceeding the second threshold.

13. An audio appliance according to claim 11, wherein the combination of voice-activity cues further comprises a level of a first beam-formed combination of the first acoustic signal and the second acoustic signal within a third frequency band, a level of a second beam-formed combination of the first acoustic signal and the second acoustic signal within the third frequency band, a level difference between the first beam-formed combination and the second beam-formed combination, or a combination thereof.

14. An audio appliance according to claim 11, further comprising an accelerometer, wherein the combination of voice-activity cues comprises an output from the accelerometer within a third frequency band.

15. An audio appliance comprising a first microphone transducer, a second microphone transducer, an output device, a processor, and a memory, wherein the first transducer and the second transducer are spaced apart from each other to define a longitudinal axis, wherein the memory contains instructions that, when executed by the processor, cause the audio appliance

to determine a presence of voice activity in an observed acoustic signal responsive to a combination of a first voice-activity cue corresponding to a first frequency band with a second voice-activity cue corresponding to a second frequency band;

responsive to a determined presence of voice activity in the observed acoustic signal, to suppress impairments originating from a direction of up to about 75-degrees from the longitudinal axis in the observed acoustic signal by between about 3 dB and about 20 dB compared to a level of the impairments in the observed acoustic signal; and

to synthesize and to output a noise-suppressed output signal corresponding to the observed acoustic signal and the suppressed impairments.

16. An audio appliance according to claim 15, wherein the first voice-activity cue comprises a measure of a first voice-activity statistic corresponding to an output from the first microphone transducer and an output from the second microphone transducer.

17. An audio appliance according to claim 16, wherein the second voice-activity cue comprises a measure of a second voice-activity statistic corresponding to an output from the first microphone transducer and an output from the second microphone transducer.

18. An audio appliance according to claim 15, further comprising an accelerometer and/or a beam former to generate one or more beam-formed combinations of an output from the first microphone transducer with an output from the second microphone transducer.

19. An audio appliance according to claim 18, wherein the first voice-activity cue comprises a measure of an acoustic signal from the first microphone, a measure of an acoustic signal from the second microphone, a measure of an output from the accelerometer, and/or a measure of an output from the beam former.