Microphone signal fusion

• US 9,401,158 B1
• Filed: 09/14/2015
• Issued: 07/26/2016
• Est. Priority Date: 09/14/2015
• Status: Active Grant

• Current Assignee: Knowles Electronics Llc (Knowles Corporation)
• Original Assignee: Knowles Electronics Llc (Knowles Corporation)
• Inventors: Yen, Kuan-Chieh, Syed, Mushtaq, Miller, Thomas
• Primary Examiner(s): Zhang, Leshui
• Application Number:
  US14/853,947
• Time in Patent Office: 316 Days
• Field of Search: 381/312, 381/316, 381/317, 381/318, 381/320, 381/321, 381/71.1, 381/71.6, 381/71.8, 381/71.11, 381/71.12, 381/71.13, 381/71.14, 381/72, 381/73.1, 381/74, 381/92, 381/93, 381/94.1, 381/94.2, 381/74.3, 381/94.4, 381/94.5, 381/74.7, 381/94.9, 381/95, 381/97, 381/98, 381/99, 381/100, 381/101, 381/102, 381/103, 381/119, 381/122, 381/111, 381/112, 381/113, 381/114, 381/115, 381/123, 381/110, 700/94, 455/575.2, 455/569.1, 455/569.2, 455/570
• US Class Current: 1/1
• CPC Class Codes: G10L 2021/02165 Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 21/0308 characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
• CPC Class Codes: H04R 1/1016 Earpieces of the intra-aural type
• CPC Class Codes: H04R 1/1041 Mechanical or electronic switches, or control elements switches in general H01H
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/107 Monophonic and stereophonic headphones with microphone for two-way hands free communication
• CPC Class Codes: H04R 2225/43 Signal processing in hearing aids to enhance the speech intelligibility
• CPC Class Codes: H04R 2410/05 Noise reduction with a separate noise microphone
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 2430/03 Synergistic effects of band splitting and sub-band processing
• CPC Class Codes: H04R 2460/13 Hearing devices using bone conduction transducers
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Provided are systems and methods for microphone signal fusion. An example method commences with receiving a first and second signal representing sounds captured, respectively, by internal and external microphones. The second signal includes at least a voice component. The first signal and the voice component are modified by at least human tissue. The first and second signals are processed to obtain noise estimates. The first signal is aligned with the second signal. The second signal and the aligned first signal are blended based on the noise estimates to generate an enhanced voice signal. The internal microphone is located inside an ear canal and sealed for isolation from acoustic signals outside the ear canal. The external microphone is located outside the ear canal. All of parts of the processing, blending and aligning of the systems and method may be performed on a subband basis in the frequency domain.

Claims

1. A method for fusion of microphone signals, the method comprising:

receiving a first signal including at least a voice component and a second signal including at least the voice component modified by at least a human tissue;

processing the first signal to obtain first noise estimates;

aligning the voice component in the second signal spectrally with the voice component in the first signal; and

blending, based at least on the first noise estimates, the first signal and the aligned voice component in the second signal to generate an enhanced voice signal, the blending including;

assigning, based at least on the first noise estimates, a first weight to the first signal and a second weight to the second signal; and

mixing the first signal and the second signal according to the first weight and the second weight.

2. The method of claim 1, wherein the second signal represents at least one sound captured by an internal microphone located inside an ear canal.

3. The method of claim 2, wherein the internal microphone is at least partially sealed for isolation from acoustic signals external to the ear canal.

4. The method of claim 1, wherein the first signal represents at least one sound captured by an external microphone located outside an ear canal.

5. The method of claim 1, further comprising processing the second signal to obtain second noise estimates.

6. The method of claim 5, wherein the assigning, of the first weight to the first signal and the second weight to the second signal, is based at least on the first noise estimates and the second noise estimates.

7. The method of claim 6, wherein the first weight receives a larger value than the second weight when a signal-to-noise ratio (SNR) of the first signal is larger than a SNR of the second signal, and wherein the second weight receives a larger value than the first weight when the SNR of the first signal is smaller than the SNR of the second signal, the difference between the first weight and the second weight corresponding to the difference between the SNR of the first signal and the SNR of the second signal.

8. The method of claim 5, further comprising:

prior to the aligning, performing, based on the first noise estimates, noise reduction of the first signal; and

prior to the aligning, performing, based on the second noise estimates, noise reduction of the second signal.

9. The method of claim 5, further comprising:

after the aligning, performing noise reduction of the first signal based on the first noise estimates; and

after the aligning, performing noise reduction of the second signal based on the second noise estimates.

10. The method of claim 1, wherein at least one of the aligning and blending is performed for subbands in the frequency domain.

11. The method of claim 1, wherein the processing, aligning, and blending are performed for subbands in the frequency domain.

12. The method of claim 1, further comprising performing noise reduction of the first signal.

13. The method of claim 1, further comprising performing noise reduction of the second signal.

14. The method of claim 1, wherein the aligning includes applying a spectral alignment filter to the second signal.

15. The method of claim 14, wherein the spectral alignment filter includes an empirically derived filter.

16. The method of claim 14, wherein the spectral alignment filter includes an adaptive filter calculated based on cross-correlation of the first signal and the second signal and auto-correlation of the second signal.

17. A system for fusion of microphone signals, the system comprising:

a digital signal processor, configured to;

receive a first signal including at least a voice component and a second signal including at least the voice component modified by at least a human tissue;

process the first signal to obtain first noise estimates;

align the voice component in the second signal spectrally with the voice component in the first signal; and

blend, based at least on the first noise estimates, the first signal and the aligned voice component in the second signal to generate an enhanced voice signal, including;

assigning, based at least on the first noise estimates, a first weight to the first signal and a second weight to the second signal; and

mixing the first signal and the second signal according to the first weight and the second weight.

18. The system of claim 17, further comprising:

an internal microphone located inside an ear canal and sealed to be isolated from acoustic signals external to the ear canal, the second signal representing at least one sound captured by the internal microphone; and

an external microphone located outside the ear canal, the first signal representing at least one sound captured by the external microphone.

19. The system of claim 17, wherein the digital signal processor is further configured to process the second signal to obtain second noise estimates.

20. The system of claim 19, wherein the assigning, of the first weight to the first signal and the second weight to the second signal, is based at least on the first noise estimates and the second noise estimates.

21. The system of claim 20, wherein the first weight receives a larger value than the second weight when a signal-to-noise ratio (SNR) of the first signal is larger than a SNR of the second signal, and wherein the second weight receives a larger value than the first weight when the SNR of the first signal is smaller than the SNR of the second signal, the difference between the first weight and second weight corresponding to the difference between the SNR of the first signal and the SNR of the second signal.

22. The system of claim 19, wherein the digital signal processor is further configured to:

perform, prior to the aligning and based on the first noise estimates, noise reduction of the first signal; and

perform, prior to the aligning and based on the second noise estimates, noise reduction of the second signal.

23. The system of claim 19, wherein the digital signal processor is further configured to:

perform, after the aligning and based on the first noise estimates, noise reduction of the first signal; and

perform, after the aligning and based on the second noise estimates, noise reduction of the second signal.

24. The system of claim 17, wherein the processing, aligning, and blending are performed for subbands in the frequency domain.

25. The system of claim 17, wherein the digital signal processor is further configured to perform noise reduction of the first signal and the second signal.

26. The system of claim 17, wherein the aligning includes applying a spectral alignment filter to the second signal.

27. The system of claim 26, wherein the spectral alignment filter includes one of an empirically derived filter and an adaptive filter, the adaptive filter being calculated based on cross-correlation of the first signal and the second signal and auto-correlation of the second signal.

28. A non-transitory computer-readable storage medium having embodied thereon instructions, which when executed by at least one processor, perform steps of a method, the method comprising:

receiving a first signal including at least a voice component and a second signal including at least the voice component modified by at least a human tissue;

processing the first signal to obtain first noise estimates;

aligning the voice component in the second signal spectrally to the voice component in the first signal; and

blending, based at least on the first noise estimates, the first signal and the aligned voice component in the second signal to generate an enhanced voice signal, the blending including;

assigning, based at least on the first noise estimates, a first weight to the first signal and a second weight to the second signal; and

mixing the first signal and the second signal according to the first weight and the second weight.