Adaptively filtering a microphone signal responsive to vibration sensed in a user's face while speaking

• US 9,767,817 B2
• Filed: 05/14/2008
• Issued: 09/19/2017
• Est. Priority Date: 05/14/2008
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Mobile Communications Incorporated (Sony Group Corp.)
• Inventors: Glebe, Dag
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US12/120,589
• Publication Number:
  US 20090287485A1
• Time in Patent Office: 3,415 Days
• Field of Search: 704200-210, 704220-229, 704233-2568, 704271, 704276, 704E19001-E19049, 704E21001-E2102, 704E11001-E11007, 398 39, 398118-131, 381 56- 60, 381 711- 731, 381 95-110, 381150-433
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 25/00 Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 muting semiconductor-based amplifiers when some special characteristics of a signal are sensed by a speech detector, e.g. sensing when no signal is present, H03G3/34
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band
• CPC Class Codes: H04R 1/02 Casings; Cabinets ; Supports therefor; Mountings therein H04R1/28 takes precedence ; attachments for microphones H04R1/08; mounting of transducers in earpieces H04R1/1075
• CPC Class Codes: H04R 1/08 Mouthpieces; Microphones; Attachments therefor
• CPC Class Codes: H04R 29/00 Monitoring arrangements; Testing arrangements for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308

Abstract

Electronic devices and methods are disclosed that adaptively filter a microphone signal responsive to vibration that is sensed in the face of a user speaking into a microphone of the device. An electronic device can include a microphone, a vibration sensor, a vibration characterization unit, and an adaptive sound filter. The microphone generates a microphone signal that can include a user speech component and a background noise component. The vibration sensor senses vibration of the face while a user speaks into the microphone, and generates a vibration signal containing frequency components that are indicative of the sensed vibration. The vibration characterization unit generates speech characterization data that characterize at least one of the frequency components of the vibration signal that is associated with the speech component of the microphone signal. The adaptive sound filter filters the microphone signal using filter coefficients that are tuned in response to the speech characterization data to generate a filtered speech signal with an attenuated background noise component relative to the user speech component from the microphone signal.

Claims

1. An electronic device comprising:

a microphone that is configured to generate a microphone signal responsive to incident sound, the microphone signal including a user speech component and a background noise component;

a housing enclosing the microphone, wherein the housing is positioned adjacent to and extends along a cheek of a user'"'"'s face while the user speaks into the microphone to generate the speech component of the microphone signal and while positioning another portion of the housing adjacent to the user'"'"'s ear;

a vibration sensor enclosed by the housing that is configured to sense vibration of an external portion of the user'"'"'s face while the user speaks into the microphone and to generate a vibration signal containing frequency components that are indicative of the sensed vibration;

a vibration characterization unit enclosed by the housing that is configured to generate speech characterization data that characterize at least one of the frequency components of the vibration signal that is associated with the speech component of the microphone signal; and

an adaptive sound filter enclosed by the housing that filters the microphone signal using filter coefficients that are tuned in response to the speech characterization data to generate a filtered speech signal with an attenuated background noise component relative to the user speech component of the microphone signal.

2. The electronic device of claim 1, wherein:

the adaptive sound filter comprises a digital finite impulse response (FIR) filter that is configured to filter the microphone signal to generate the filtered speech signal; and

the vibration characterization unit tunes FIR coefficients applied to each of a plurality of time-delay taps of the FIR filter in response to the at least one of the frequency components of the vibration signal.

3. The electronic device of claim 1, wherein the vibration characterization unit comprises:

a plurality of band pass filters, each configured to pass-through a different narrow frequency bandwidth component of the vibration signal; and

a frequency characterization unit that is configured to characterize the frequency components of the vibration signal that are associated with the speech component of the microphone signal in response to the magnitude of each of the narrow frequency bandwidth components passed through the respective band pass filters.

4. The electronic device of claim 3, wherein the adaptive sound filter tunes its filter coefficients in response to the magnitude of each of the narrow frequency bandwidth components passed through the respective band pass filters.

5. The electronic device of claim 3, wherein the frequency characterization unit is further configured to identify different harmonic content of formants spoken by the user in response to the magnitude of each of the narrow frequency bandwidth components passed through the respective band pass filters, and to tune the filter coefficients of the adaptive sound filter in response to the identified harmonic content of formants spoken by the user.

6. The electronic device of claim 5, further comprising a voice codec that is configured to encode the filtered speech signal, wherein the frequency characterization unit is further configured to tune coefficients used by the voice codec to encode the filtered speech signal in response to the identified harmonic content of formants spoken by the user.

7. The electronic device of claim 1, wherein the adaptive sound filter comprises a plurality of variable band pass filters, each tuned to pass-through a different narrow frequency bandwidth component of the microphone signal in response to the magnitude of each of the narrow frequency bandwidth components passed through the corresponding band pass filters of the vibration characterization unit, wherein the filtered speech signal is generated based on combining the passed-through components of the microphone signal output by the variable band pass filters of the adaptive sound filter.

8. The electronic device of claim 1, wherein the vibration characterization unit is further configured to correlate the frequency components of the vibration signal to corresponding frequency components of the microphone signal, and to generate the speech characterization data to be indicative of a result of the correlation.

9. The electronic device of claim 1, wherein the vibration sensor comprises an accelerometer that is attached to an inner surface of the housing and configured to sense vibration conducted from the face of the user through the housing to the vibration sensor.

10. The electronic device of claim 9, wherein:

the housing comprises a raised edge portion that projects away from a major portion of a front surface of the housing to urge contact with the face of the user speaking into the microphone; and

the vibration sensor is attached to an inner surface of the raised edge portion of the housing to sense vibration from contacting the face while the user is speaking.

11. The electronic device of claim 10, wherein at least a major portion of the raised edge portion of the housing comprises a more rigid material than the major portion of the front surface of the housing to cause increased conduction of higher frequency components of vibration from the face of the user through the raised edge portion of the housing to the vibration sensor.

12. The electronic device of claim 1, further comprising a speaker enclosed by the housing, wherein the speaker is positioned adjacent to the user'"'"'s ear while the user speaks into the microphone to generate the speech component of the microphone signal.

13. A method comprising:

operating a circuit to sense vibration of an external portion of a user'"'"'s face while the user speaks into a microphone and to generate a vibration signal containing frequency components that are indicative of the sensed vibration, wherein the microphone is enclosed by a housing that extends along a cheek of the user'"'"'s face while the user speaks into the microphone and while positioning another portion of the housing adjacent to the user'"'"'s ear;

generating speech characterization data that characterize at least one of the frequency components of the vibration signal that is associated with a speech component of a microphone signal generated by the microphone responsive to incident sound; and

adaptively filtering the microphone signal using filter coefficients that are tuned in response to the speech characterization data to generate a filtered speech signal with an attenuated background noise component relative to the speech component of the microphone signal.

14. The method of claim 13, wherein the adaptive filtering comprises:

filtering the microphone signal to generate the filtered speech signal using a digital finite impulse response (FIR) filter; and

tuning FIR coefficients applied to each of a plurality of time-delay taps of the FIR filter in response to the at least one of the frequency components of the vibration signal.

15. The method of claim 13, wherein the generation of the speech characterization data comprises:

filtering the vibration signal using a plurality of band pass filters that are each configured to pass-through a different narrow frequency bandwidth component of the vibration signal; and

generating the speech characterization data in response to the magnitude of each of the narrow frequency bandwidth components passed through the respective band pass filters.

16. The method of claim 15, wherein the adaptive filtering comprises tuning the filter coefficients in response to the magnitude of each of the narrow frequency bandwidth components passed through the respective band pass filters.

17. An electronic device comprising:

a microphone that is configured to generate a microphone signal responsive to incident sound, the microphone signal including a user speech component and a background noise component;

a housing enclosing the microphone, the housing positioned adjacent to and extending along a cheek of a user'"'"'s face who is speaking into the microphone while positioning another portion of the housing adjacent to the user'"'"'s ear, the housing having a raised edge portion that projects away from a major portion of a front surface of the housing to urge contact with an external portion of the user'"'"'s face who is speaking into the microphone;

a vibration sensor attached to an inner surface of the raised edge portion of the housing and configured to sense vibration conducted from the external portion of the face of the user through the housing to the vibration sensor and to generate a vibration signal containing frequency components that are indicative of the sensed vibration;

a vibration characterization unit that includes a plurality of band pass filters, each configured to pass-through a different narrow frequency bandwidth component of the vibration signal; and

an adaptive sound filter that filters the microphone signal using filter coefficients that are tuned in response to the magnitude of each of the narrow frequency bandwidth components passed through the respective band pass filters to generate a filtered speech signal with an attenuated background noise component relative to the user speech component of the microphone signal.

18. The electronic device of claim 17, further comprising a voice codec that is configured to encode the filtered speech signal, wherein the vibration characterization unit is further configured to identify different harmonic content of formants spoken by the user in response to the magnitude of each of the narrow frequency bandwidth components passed through the respective band pass filters, and to tune the filter coefficients used by the voice codec to encode the filtered speech signal in response to the identified harmonic content of formants spoken by the user.

19. An electronic device comprising:

a microphone that is configured to generate a microphone signal responsive to incident sound, the microphone signal including a user speech component and a background noise component;

a housing enclosing the microphone, wherein the housing is positioned adjacent to and extends along a cheek of a user'"'"'s face while the user speaks into the microphone to generate the speech component of the microphone signal and while positioning another portion of the housing adjacent to the user'"'"'s ear;

a vibration sensor that is configured to sense vibration of an external portion of the user'"'"'s face while the user speaks into the microphone and to generate a vibration signal containing frequency components that are indicative of the sensed vibration, wherein the vibration sensor comprises;

a light source that is configured to generate coherent light that is modulated by a known signal waveform to illuminate adjacent external facial skin of the user speaking into the microphone;

a light sensor that is configured to generate a sensed light signal in response to sensing reflection of the light from the adjacent external facial skin, the sensed light signal being modulated by vibration of the illuminated external facial skin; and

a comparison circuit that is configured to compare the known signal waveform to the sensed light signal to generate the vibration signal containing frequency components that are indicative of the vibration of the illuminated external facial skin of the user;

a vibration characterization unit that is configured to generate speech characterization data that characterize at least one of the frequency components of the vibration signal that is associated with the speech component of the microphone signal; and

an adaptive sound filter that filters the microphone signal using filter coefficients that are tuned in response to the speech characterization data to generate a filtered speech signal with an attenuated background noise component relative to the user speech component of the microphone signal.

20. The electronic device of claim 19, wherein the comparison circuit is configured to modulate the vibration signal to track over time phase differences between the known signal waveform, which modulates the light output by the light source, and the sensed light signal.

21. The electronic device of claim 19, wherein the light source comprises a laser diode.