Microphone signal fusion

• US 9,961,443 B2
• Filed: 07/18/2016
• Issued: 05/01/2018
• Est. Priority Date: 09/14/2015
• Status: Active Grant

• Current Assignee: Knowles Electronics Llc (Knowles Corporation)
• Original Assignee: Knowles Electronics Llc (Knowles Corporation)
• Inventors: Yen, Kuan-Chieh, Miller, Thomas E., Syed, Mushtaq
• Primary Examiner(s): Zhang, Leshui
• Application Number:
  US15/213,203
• Publication Number:
  US 20170078790A1
• Time in Patent Office: 652 Days
• Field of Search: 381312, 381316, 381317, 381318, 381320, 381321, 381 711, 381 716, 381 718, 381 7111, 381 7112, 381 7113, 381 7114, 381 72, 381 731, 381 74, 381 92, 381 93, 381 941, 381 942, 381 743, 381 944, 381 945, 381 747, 381 949, 381 95, 381 97, 381 98, 381 99, 381100, 381101, 381102, 381103, 381119, 381122, 381111, 381112, 381113, 381114, 381115, 381123, 381 58, 381314, 381110, 700 94, 4555752, 4555691, 4555692, 455570
• CPC Class Codes: G10L 2021/02165 Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 21/0308 characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
• CPC Class Codes: H04R 1/1016 Earpieces of the intra-aural type
• CPC Class Codes: H04R 1/1041 Mechanical or electronic switches, or control elements switches in general H01H
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/107 Monophonic and stereophonic headphones with microphone for two-way hands free communication
• CPC Class Codes: H04R 2225/43 Signal processing in hearing aids to enhance the speech intelligibility
• CPC Class Codes: H04R 2410/05 Noise reduction with a separate noise microphone
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 2430/03 Synergistic effects of band splitting and sub-band processing
• CPC Class Codes: H04R 2460/13 Hearing devices using bone conduction transducers
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Provided are systems and methods for microphone signal fusion. An example method commences with receiving a first and second signal representing sounds captured, respectively, by external and internal microphones. The internal microphone is located inside an ear canal and sealed for isolation from outside acoustic signals. The external microphone is located outside the ear canal. The first signal comprises a voice component. The second signal comprises a voice component modified by at least human tissue. The first and second signals are processed to obtain noise estimates. The voice component of the second signal is aligned with the voice component of the first signal. The first signal and the aligned voice component of the second signal are blended, based on the noise estimates, to generate an enhanced voice signal. Prior to aligning, the voice component of the second signal may be processed to emphasize high frequency content, improving effective alignment bandwidth.

Claims

1. A method for fusion of microphone signals, the method comprising:

receiving a first signal including at least a voice component, the voice component of the first signal having a first frequency response, and a second signal including at least the voice component, the voice component of the second signal having a second frequency response that is modified from the first frequency response by at least transmission of the voice component through a human tissue;

processing the first signal to obtain first noise estimates;

aligning the voice component in the second signal spectrally with the voice component in the first signal by applying a filter to the second signal that causes the second frequency response to be altered toward the first frequency response within a bandwidth of the voice component of the second signal; and

blending, based at least on the first noise estimates, the first signal and the aligned voice component in the second signal to generate an enhanced voice signal.

2. The method of claim 1, wherein the second signal represents at least one sound captured by an internal microphone located inside an ear canal.

3. The method of claim 2, wherein the internal microphone is at least partially sealed for isolation from acoustic signals external to the ear canal.

4. The method of claim 2, wherein the first signal represents at least one sound captured by an external microphone located outside the ear canal.

5. The method of claim 2, wherein the voice component of the second signal, representing the at least one sound captured by the internal microphone, comprises low frequency content and high frequency content.

6. The method of claim 5, wherein, prior to the aligning, the voice component of the second signal representing the at least one sound captured by the internal microphone is processed to emphasize the high frequency content.

7. The method of claim 6, wherein the emphasizing the high frequency content comprises applying perceptual-based frequency weighting to the high frequency content.

8. The method of claim 1, wherein the filter includes an adaptive filter calculated based on cross-correlation of the first signal and the second signal and auto-correlation of the second signal.

9. The method of claim 1, wherein the filter is derived from empirical data.

10. A system for fusion of microphone signals, the system comprising:

a digital signal processor, configured to;

receive a first signal including at least a voice component, the voice component of the first signal having a first frequency response, and a second signal including at least the voice component, the voice component of the second signal having a second frequency response that is modified from the first frequency response by at least transmission of the voice component through a human tissue;

process the first signal to obtain first noise estimates;

align the voice component in the second signal spectrally with the voice component in the first signal by applying a filter to the second signal that causes the second frequency response to be altered toward the first frequency response within a bandwidth of the voice component of the second signal; and

blend, based at least on the first noise estimates, the first signal and the aligned voice component in the second signal to generate an enhanced voice signal.

11. The system of claim 10, wherein the second signal represents at least one sound captured by an internal microphone located inside an ear canal.

12. The system of claim 11, wherein the internal microphone is at least partially sealed for isolation from acoustic signals external to the ear canal.

13. The system of claim 11, wherein the first signal represents at least one sound captured by an external microphone located outside the ear canal.

14. The system of claim 11, wherein the voice component of the second signal, representing the at least one sound captured by the internal microphone, comprises low frequency content and high frequency content.

15. The system of claim 14, wherein, prior to the aligning, the voice component of the second signal representing the at least one sound captured by the internal microphone is processed to emphasize the high frequency content.

16. The system of claim 15, wherein the emphasizing the high frequency content comprises applying perceptual-based frequency weighting to the high frequency content.

17. The system of claim 10, wherein the filter includes an adaptive filter calculated based on cross-correlation of the first signal and the second signal and auto-correlation of the second signal.

18. The system of claim 10, wherein the filter is derived from empirical data.

19. A non-transitory computer-readable storage medium having embodied thereon instructions, which, when executed by at least one processor, perform steps of a method, the method comprising:

receiving a first signal including at least a voice component, the voice component of the first signal having a first frequency response, and a second signal including at least the voice component, the voice component of the second signal having a second frequency response that is modified from the first frequency response by at least transmission of the voice component through a human tissue, the first signal representing at least one sound captured by an external microphone located outside the ear canal, and the second signal representing at least one sound captured by an internal microphone located inside an ear canal;

processing the first signal to obtain first noise estimates;

aligning the voice component in the second signal spectrally with the voice component in the first signal by applying a filter to the second signal that causes the second frequency response to be altered toward the first frequency response within a bandwidth of the voice component of the second signal; and

blending, based at least on the first noise estimates, the first signal and the aligned voice component in the second signal to generate an enhanced voice signal;

the voice component of the second signal, representing the at least one sound captured by the internal microphone, comprising low frequency content and high frequency content and, prior to the aligning, processing the voice component of the second signal, representing the at least one sound captured by the internal microphone, to emphasize the high frequency content.