CAPTURE AND EXTRACTION OF OWN VOICE SIGNAL

• US 20190043518A1
• Filed: 02/24/2017
• Published: 02/07/2019
• Est. Priority Date: 02/25/2016
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: LI, Chunjian
• Granted Patent: US 10,586,552 B2
• Time in Patent Office: Days
• CPC Class Codes: G10L 2021/02165 Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
• CPC Class Codes: G10L 21/02 Speech enhancement, e.g. noise reduction or echo cancellation reducing echo effects in line transmission systems H04B3/20; echo suppression in hands-free telephones H04M9/08
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 25/03 characterised by the type of extracted parameters
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H04R 2201/107 Monophonic and stereophonic headphones with microphone for two-way hands free communication
• CPC Class Codes: H04R 2410/05 Noise reduction with a separate noise microphone
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Methods and systems employing an internal microphone and an external microphone of a headset to capture own voice content in the presence of noise, extract the own voice content from background noise (by performing noise reduction on the microphone outputs to generate a noise reduced signal indicative of the own voice content), and optionally also perform voice activity detection to identify segments of own voice presence or absence. In some embodiments, the external microphone is employed to capture the own voice content, the internal microphone signal is employed to infer the noise captured by the external microphone, and the inferred noise is subtracted from the external microphone signal to generate the noise reduced signal. Aspects include methods performed by any embodiment of the system, and a system or device configured (e.g., programmed) to perform any embodiment of the method.

Claims

1. A method for capturing sound using a headset having at least one earpiece, wherein a user'"'"'s ear canal is closed by the earpiece, the earpiece including an external microphone and an internal microphone, wherein the internal microphone is positioned in or on an inside portion of the earpiece and the external microphone is positioned in or on an outside portion of the earpiece, and wherein the internal microphone is located in a chamber formed by the earpiece and an ear of the user, said method including steps of:

(a) in the presence of sound including own voice content and noise, generating an external microphone signal indicative of the sound as captured by the external microphone, and generating an internal microphone signal indicative of the sound as captured by the internal microphone, where the own voice content is indicative of at least one vocal utterance of the user of the headset; and

(b) performing noise reduction on the external microphone signal, including by filtering the internal microphone signal to generate a filtered signal indicative of at least some of the noise as captured by the external microphone, and generating a noise reduced signal indicative of the own voice content by subtracting the filtered signal from the external microphone signal,

wherein the step of filtering the internal microphone signal to generate the filtered signal corresponds to application of a transfer function, InvP(z), to the internal microphone signal, wherein the transfer function, InvP(z), is equal to or at least substantially equal to an inverse of a transfer function, P(z), that represents filtering during transit through the earpiece to the internal microphone.

2. The method of claim 1, wherein the step of filtering the internal microphone signal to generate the filtered signal corresponds to application of the transfer function, InvP(z), to the internal microphone signal, so that said filtered signal is the signal, InvP(z)M, where

M is the internal microphone signal,

InvP(z) is the inverse of the transfer function, P(z),

Se is ambient sound, which is noise originating from one or more sources external to the user of the headset, as sensed and captured by the external microphone, whereby said ambient sound, Se, is distinct from and does not include the own voice content, and

P(z)Se is a signal at least substantially equal to the ambient sound, Se, as sensed and captured by the internal microphone, whereby the signal P(z)Se corresponds to the ambient sound, Se, after undergoing filtering by the transfer function P(z) during transit through the earpiece to the internal microphone.

3. The method of claim 2, wherein step (b) includes a step of performing equalization on the noise reduced signal to reduce distortion of the own voice content indicated by the noise reduced signal, thereby generating an equalized noise reduced signal, wherein the step of performing equalization on the noise reduced signal corresponds to application of a transfer function, E(z), to the noise reduced signal, so that said equalized noise reduced signal is the signal, E(z)X, where

X is the noise reduced signal,

E(z) is at least substantially equal to P(z)InvT(z),

InvT(z) is the inverse of a transfer function, T(z), and

the transfer function, T(z), characterizes filtering of the own voice content due to transmission through a portion of the user'"'"'s body to the internal microphone.

4. The method of claim 3, wherein the transfer function, E(z), is a stable approximation to P(z)InvT(z).

5. The method of claim 1, wherein step (b) includes a step of performing equalization on the noise reduced signal to reduce distortion of the own voice content indicated by the noise reduced signal, thereby generating an equalized noise reduced signal.

6. The method of claim 1, wherein step (b) includes performing residual noise reduction on the equalized noise reduced signal.

7. The method of claim 6, wherein the noise includes coherent noise and incoherent noise, subtraction of the filtered signal from the external microphone signal in step (b) removes most of the coherent noise from the external microphone signal, the noise reduced signal and the equalized noise reduced signal are indicative of at least some of the incoherent noise, and the residual noise reduction is performed so as to remove at least some of the incoherent noise from the equalized noise reduced signal.

8. The method of claim 6, also including a step of:

performing own voice detection on at least one of the noise reduced signal, the equalized noise reduced signal, the external microphone signal, or the internal microphone signal to determine time segments of own voice activity, and wherein the step of performing residual noise reduction on the equalized noise reduced signal uses a noise estimate determined from at least one of the noise reduced signal, the equalized noise reduced signal, the external microphone signal, or the internal microphone signal at times between the time segments of own voice activity.

9. The method of claim 8, wherein the step of performing own voice detection includes steps of:

comparing power of the noise reduced signal or the equalized noise reduced signal, and power of the external microphone signal, on a frame by frame basis;

identifying each frame, of the noise reduced signal or the equalized noise reduced signal, whose power is much smaller than the power of a corresponding frame of the external microphone signal as an own-voice absent frame corresponding to a time segment other than a time segment of own voice activity; and

identifying each frame, of the noise reduced signal or the equalized noise reduced signal, whose power is not much smaller than the power of the corresponding frame of the external microphone signal as an own-voice frame corresponding to a time segment of own voice activity.

10. The method of claim 8, wherein the step of performing own voice detection includes steps of:

comparing levels of frequency components of time segments of the internal microphone signal and levels of frequency components of corresponding time segments of the external microphone signal in a low frequency range;

determining that each time segment of the internal microphone signal and the external microphone signal in which the levels of the frequency components of the internal microphone signal are higher than the levels of the frequency components of the external microphone signal, in the low frequency range, is indicative of own voice activity; and

determining that each time segment of the internal microphone signal and the external microphone signal in which the levels of the frequency components of the internal microphone signal are not higher than the levels of the frequency components of the external microphone signal, in the low frequency range, is not indicative of own voice activity.

11. The method of claim 10, wherein the low frequency range is a range from a frequency at least substantially equal to 100 Hz to a frequency at least substantially equal to 500 Hz.

12. A headset, including:

at least one earpiece including an external microphone positioned in or on an outside portion of the earpiece and an internal microphone positioned in or on an inside portion of the earpiece, wherein a user'"'"'s ear canal is closed by the earpiece and the internal microphone is located in a chamber formed by the earpiece and an ear of the user, configured to operate in the presence of sound including own voice content and noise, to generate an external microphone signal indicative of the sound as captured by the external microphone, and to generate an internal microphone signal indicative of the sound as captured by the internal microphone, where the own voice content is indicative of at least one vocal utterance of the user of the headset; and

an audio processing system coupled to receive the external microphone signal and the internal microphone signal, and configured to perform noise reduction on the external microphone signal and the internal microphone signal to generate a noise reduced signal indicative of the own voice content, including by;

filtering the internal microphone signal to generate a filtered signal indicative of at least some of the noise as captured by the external microphone, and generating the noise reduced signal by subtracting the filtered signal from the external microphone signal,

wherein the audio processing system is configured to filter the internal microphone signal to generate the filtered signal in a manner corresponding to application of a transfer function, InvP(z), to the internal microphone signal, wherein the transfer function, InvP(z), is equal to or at least substantially equal to an inverse of a transfer function, P(z), that represents filtering during transit through the earpiece to the internal microphone.

13. The headset of claim 12, wherein the audio processing system is configured to filter the internal microphone signal to generate the filtered signal in a manner corresponding to application of the transfer function, InvP(z), to said internal microphone signal, so that said filtered signal is the signal, InvP(z)M, where

M is the internal microphone signal,

InvP(z) is the inverse of the transfer function, P(z),

Se is ambient sound, which is noise originating from one or more sources external to the user of the headset, as sensed and captured by the external microphone, whereby said ambient sound, Se, is distinct from and does not include the own voice content, and

P(z)Se is a signal at least substantially equal to the ambient sound, Se, as sensed and captured by the internal microphone, whereby the signal P(z)Se corresponds to the ambient sound, Se, after undergoing filtering by the transfer function P(z) during transit through the earpiece to the internal microphone.

16. The headset of claim 12, wherein the audio processing system includes an equalization subsystem coupled to receive the noise reduced signal and configured to perform equalization on said noise reduced signal to reduce distortion of the own voice content indicated by said noise reduced signal, thereby generating an equalized noise reduced signal.

17. The headset of claim 16, wherein the audio processing system also includes a noise reduction subsystem coupled and configured to perform residual noise reduction on the equalized noise reduced signal.

14-15. -15. (canceled)

18-22. -22. (canceled)

23. An audio processing system for extracting own voice content captured by a microphone set of an earpiece of a headset, where the own voice content is indicative of at least one vocal utterance of a user of the headset and the microphone set includes an external microphone positioned in or on an outside portion of the earpiece and an internal microphone positioned in or on an inside portion of the earpiece, wherein the user'"'"'s ear canal is closed by the earpiece and the internal microphone is located in a chamber formed by the earpiece and an ear of the user, said audio processing system including:

at least one input coupled to receive an external microphone signal indicative of output of the external microphone and an internal microphone signal indicative of output of the internal microphone, where the external microphone signal and the internal microphone signal have been generated with the external microphone and the internal microphone in the presence of sound including noise and the own voice content, the external microphone signal is indicative of the sound as captured by the external microphone, and the internal microphone signal is indicative of the sound as captured by the internal microphone; and

a noise cancellation subsystem coupled and configured to perform noise reduction on the external microphone signal and the internal microphone signal to generate a noise reduced signal indicative of the own voice content, including by;

filtering the internal microphone signal to generate a filtered signal indicative of at least some of the noise as captured by the external microphone, and generating the noise reduced signal by subtracting the filtered signal from the external microphone signal,

wherein the noise cancellation subsystem is configured to filter the internal microphone signal to generate the filtered signal in a manner corresponding to application of a transfer function, InvP(z), to the internal microphone signal, wherein the transfer function, InvP(z), is equal to or at least substantially equal to an inverse of a transfer function, P(z), that represents filtering during transit through the earpiece to the internal microphone.

24. The system of claim 23, wherein the noise cancellation subsystem is configured to filter the internal microphone signal to generate the filtered signal in a manner corresponding to application of the transfer function, InvP(z), to said internal microphone signal, so that said filtered signal is the signal, InvP(z)M, where

M is the internal microphone signal,

InvP(z) is the inverse of the transfer function, P(z),

Se is ambient sound, which is noise originating from one or more sources external to the user of the headset, as sensed and captured by the external microphone, whereby said ambient sound, Se, is distinct from and does not include the own voice content, and

P(z)Se is a signal at least substantially equal to the ambient sound, Se, as sensed and captured by the internal microphone, whereby the signal P(z)Se corresponds to the ambient sound, Se, after undergoing filtering by the transfer function P(z) during transit through the earpiece to the internal microphone.

27. The system of claim 23, also including:

an equalization subsystem coupled to receive the noise reduced signal and configured to perform equalization on said noise reduced signal to reduce distortion of the own voice content indicated by said noise reduced signal, thereby generating an equalized noise reduced signal.

28. The system of claim 27, also including:

a noise reduction subsystem coupled and configured to perform residual noise reduction on the equalized noise reduced signal.

25-26. -26. (canceled)

29-33. -33. (canceled)

34. A tangible, computer readable medium which stores, in a non-transitory manner, code for programming an audio processing system to perform processing on an external microphone signal indicative of output of an external microphone of an earpiece of a headset and an internal microphone signal indicative of output of an internal microphone of the earpiece, wherein the internal microphone is positioned in or on an inside portion of the earpiece and the external microphone is positioned in or on an outside portion of the earpiece, wherein a user'"'"'s ear canal is closed by the earpiece and the internal microphone is located in a chamber formed by the earpiece and an ear of the user, and where the external microphone signal and the internal microphone signal have been generated with the external microphone and the internal microphone in the presence of sound including noise and own voice content, the external microphone signal is indicative of the sound as captured by the external microphone, the internal microphone signal is indicative of the sound as captured by the internal microphone, and the own voice content is indicative of at least one vocal utterance of the user of the headset, said processing including a step of:

performing noise reduction on the external microphone signal, including by filtering the internal microphone signal to generate a filtered signal indicative of at least some of the noise as captured by the external microphone, and generating a noise reduced signal indicative of the own voice content by subtracting the filtered signal from the external microphone signal,

wherein the step of filtering the internal microphone signal to generate the filtered signal corresponds to application of a transfer function, InvP(z), to the internal microphone signal, wherein the transfer function, InvP(z), is equal to or at least substantially equal to an inverse of a transfer function, P(z), that represents filtering during transit through the earpiece to the internal microphone.

35-42. -42. (canceled)