System and method of improving voice quality in a wireless headset with untethered earbuds of a mobile device

• US 9,913,022 B2
• Filed: 11/16/2016
• Issued: 03/06/2018
• Est. Priority Date: 02/21/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Dusan, Sorin V., Paquier, Baptiste P., Lindahl, Aram M.
• Primary Examiner(s): Edun, Muhammad N
• Application Number:
  US15/353,308
• Publication Number:
  US 20170127172A1
• Time in Patent Office: 475 Days
• Field of Search: None
• CPC Class Codes: G10K 11/178 by electro-acoustically regenerating the original acoustic waves in anti-phase
• CPC Class Codes: G10L 2021/02161 Number of inputs available containing the signal or the noise to be suppressed
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 25/06 the extracted parameters being correlation coefficients
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G10L 25/90 Pitch determination of speech signals
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 2460/13 Hearing devices using bone conduction transducers
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/033 Headphones for stereophonic communication details thereof, e.g. relating to batteries, cables or control elements H04R1/10

Abstract

Method of improving voice quality using a wireless headset with untethered earbuds starts by receiving first acoustic signal from first microphone included in first untethered earbud and receiving second acoustic signal from second microphone included in second untethered earbud. First inertial sensor output is received from first inertial sensor included in first earbud and second inertial sensor output is received from second inertial sensor included in second earbud. First earbud processes first noise/wind level captured by first microphone, first acoustic signal and first inertial sensor output and second earbud processes second noise/wind level captured by second microphone, second acoustic signal, and second inertial sensor output. First and second noise/wind levels and first and second inertial sensor outputs are communicated between the earbuds. First earbud transmits first acoustic signal and first inertial sensor output when first noise and wind level is lower than second noise/wind level. Other embodiments are described.

Claims

1. A method of improving voice quality of a mobile device using a wireless headset with untethered earbuds comprising:

receiving a first group of acoustic signals from a first front microphone, a first rear microphone and a first end microphone, respectively, included in a first untethered earbud;

receiving a second group of acoustic signals from a second front microphone, a second rear microphone, and a second end microphone, respectively, included in a second untethered earbud;

determining whether the first earbud is in-ear or whether it is out-ear based on a power ratio of a pair of the first group of acoustic signals,

determining whether the second earbud is in-ear or whether it is out-ear based on a power ratio of a pair of the second group of acoustic signals;

receiving a first inertial sensor output from a first inertial sensor included in the first earbud and receiving a second inertial sensor output from a second inertial sensor included in the second earbud;

transmitting by the first earbud the first group of acoustic signals and the first inertial sensor output when the first earbud is determined to be in-ear, and not when the first earbud is determined to be out-ear; and

transmitting by the second earbud the second group of acoustic signals and the second inertial sensor output when the second earbud is determined to be in-ear, and not when the second earbud is determined to be out-ear.

2. The method of claim 1 further comprising:

monitoring a first battery level of the first earbud and a second battery level of the second earbud; and

wherein if the battery level of one of the first and second earbuds that is transmitting is smaller than the battery level of the other one that is non-transmitting, by a predetermined threshold, then the non-transmitting earbud becomes a transmitting earbud and starts to transmit its group of acoustic signals and its inertial sensor output.

3. The method of claim 1 wherein determining whether the first earbud and the second earbud are in-ear or whether they are out-ear is based on the first inertial sensor output and the second inertial sensor output, respectively.

4. The method of claim 3,

wherein the first inertial sensor output includes first x, y, and z signals and the second inertial sensor output includes second x, y and z signals,

wherein determining whether the first earbud and the second earbud are in-ear or whether they are out-ear is based on classifying a combination of the first x, y, and z signals and the second x, y, and z signals.

5. The method of claim 1 wherein the first and second groups of acoustic signals comprise acoustic signals generated by the user'"'"'s speech or acoustic signals outputted from an earbud speaker during playback.

6. The method of claim 1, when the first earbud transmits the first group of acoustic signals and the first inertial sensor output, further comprising:

generating by a voice activity detector (VAD) a VAD output based on (i) one or more of the first group of acoustic signals and (ii) the first inertial sensor output.

7. The method of claim 6, wherein generating the VAD output comprises:

computing a power envelope of at least one of x, y, z signals generated by the first inertial sensor; and

setting the VAD output to 1 to indicate that the user'"'"'s voiced speech is detected if the power envelope is greater than a threshold and setting the VAD output to 0 to indicate that the user'"'"'s voiced speech is not detected if the power envelope is less than the threshold.

8. The method of claim 6, wherein generating the VAD output comprises:

computing the normalized cross-correlation between any pair of x, y, z direction signals generated by the first inertial sensor;

setting the VAD output to 1 to indicate that the user'"'"'s voiced speech is detected if normalized cross-correlation is greater than a threshold within a short delay range, and setting the VAD output to 0 to indicate that the user'"'"'s voiced speech is not detected if the normalized cross-correlation is less than the threshold.

9. The method of claim 6, wherein generating the VAD output comprises:

detecting voiced speech included in one or more of the first group of acoustic signals;

detecting the vibration of the user'"'"'s vocal chords from the first inertial sensor output;

computing a coincidence of the detected speech in one or more of the first group of acoustic signals and the vibration of the user'"'"'s vocal chords; and

setting the VAD output to indicate that the user'"'"'s voiced speech is detected if the coincidence is detected and setting the VAD output to indicate that the user'"'"'s voiced speech is not detected if the coincidence is not detected.

10. The method of claim 9, wherein generating the VAD output comprises:

detecting unvoiced speech in the first group of acoustic signals by;

analyzing one or more of the first group of acoustic signals;

if an energy envelope in a high frequency band of said one or more of the first group of acoustic signals is greater than a threshold, a VAD output for unvoiced speech (VADu) is set to indicate that unvoiced speech is detected; and

setting a global VAD output to indicate that the user'"'"'s speech is detected if the voiced speech is detected or if the VADu is set to indicate that unvoiced speech is detected.

11. The method of claim 10, further comprising:

generating a pitch estimate by a pitch detector based on autocorrelation and using the first inertial sensor output, wherein the pitch estimate is obtained by (i) using an X, Y, or Z signal generated by the first inertial sensor that has a highest power level or (ii) using a combination of the X, Y, and Z signals generated by the first inertial sensor.

12. The method of claim 1, wherein the first inertial sensor and the second inertial sensor are accelerometers.

13. A system for improving voice quality of a mobile device comprising:

a wireless headset including a first untethered earbud and a second untethered earbud,

wherein the first earbud includes a first front microphone, a first rear microphone and a first end microphone to transmit a first group of acoustic signals, respectively, a first inertial sensor to generate a first inertial sensor output, a first earbud processor to determine whether the first earbud is in-ear or whether it is out-ear based on a power ratio of a pair of the first group of acoustic signals, and a first communication interface, and

wherein the second earbud includes a second front microphone, a second rear microphone and a second end microphone to transmit a second group of acoustic signals, respectively, a second inertial sensor to generate a second inertial sensor output, a second earbud processor to determine whether the second earbud is in-ear or whether it is out-ear based on a power ratio of a pair of the second group of acoustic signals, and a second communication interface,

wherein the first communication interface is to transmit the first group of acoustic signals and the first inertial sensor output when the first earbud processor has determined that the first earbud is in-ear, and not when the first earbud is determined to be out-ear, and

wherein the second communication interface is to transmit the second group of acoustic signals and the second inertial sensor output when the second earbud processor has determined that the second earbud is in-ear, and not when the second earbud is determined to be out-ear.

14. The system of claim 13, wherein

the first earbud processor monitors a first battery level of the first earbud and the second earbud processor monitors a second battery level of the second earbud;

and

wherein if the battery level of one of the first and second earbuds, whose communication interface is transmitting its group of acoustic signals and its inertial sensor output, is smaller than the battery level of the other one of the first and second earbuds, whose communication interface is not transmitting its group of acoustic signals and its inertial sensor output, by a predetermined threshold, then the non-transmitting earbud becomes a transmitting earbud wherein its communication interface starts to transmit its group of acoustic signals and its inertial sensor output.

15. The system of claim 13 wherein the first and second earbud processors are to determine whether the first earbud and the second earbud are in-ear or out-ear based on the first inertial sensor output and the second inertial sensor output, respectively.

16. The system of claim 15,

wherein the first inertial sensor output includes first x, y, and z signals and the second inertial sensor output includes second x, y and z signals,

wherein the first earbud processor and the second earbud processor determine whether the first earbud and the second earbud are in-ear or out-ear based on classifying a combination of the first x, y, and z signals and the second x, y, and z signals.

17. The system of claim 13

wherein the first and second groups of acoustic signals comprise acoustic signals generated by the user'"'"'s speech or acoustic signals outputted from a an earbud speaker during playback.

18. The system of claim 13, when the first communication interface transmits the first group of acoustic signals and the first inertial sensor output, the system further comprising:

a voice activity detector (VAD) to generate a VAD output based on (i) one or more of the first group of acoustic signals and (ii) the first inertial sensor output.

19. The system of claim 18, wherein the VAD generating the VAD output comprises:

the VAD computing a power envelope of at least one of x, y, z signals generated by the first inertial sensor; and

the VAD setting the VAD output to 1 to indicate that the user'"'"'s voiced speech is detected if the power envelope is greater than a threshold and setting the VAD output to 0 to indicate that the user'"'"'s voiced speech is not detected if the power envelope is less than the threshold.

20. The system of claim 18, wherein the VAD generating the VAD output comprises:

the VAD computing the normalized cross-correlation between any pair of x, y, z direction signals generated by the first inertial sensor;

the VAD setting the VAD output to 1 to indicate that the user'"'"'s voiced speech is detected if normalized cross-correlation is greater than a threshold within a short delay range, and setting the VAD output to 0 to indicate that the user'"'"'s voiced speech is not detected if the normalized cross-correlation is less than the threshold.

21. The system of claim 18, wherein the VAD generating the VAD output comprises the VAD:

detecting voiced speech included in one or more of the first group of acoustic signals;

detecting the vibration of the user'"'"'s vocal chords from the first inertial sensor output;

computing a coincidence of the detected speech in one or more of the first group acoustic signals and the vibration of the user'"'"'s vocal chords; and

setting the VAD output to indicate that the user'"'"'s voiced speech is detected if the coincidence is detected and setting the VAD output to indicate that the user'"'"'s voiced speech is not detected if the coincidence is not detected.

22. The system of claim 21, wherein the VAD generating the VAD output comprises the VAD:

detecting unvoiced speech in the first group of acoustic signals by;

analyzing one or more of the first group of acoustic signals;

if an energy envelope in a high frequency band of said one or more of the first group of acoustics signal is greater than a threshold, a VAD output for unvoiced speech (VADu) is set to indicate that unvoiced speech is detected; and

setting a global VAD output to indicate that the user'"'"'s speech is detected if the voiced speech is detected or if the VADu is set to indicate that unvoiced speech is detected.

23. The system of claim 22, further comprising:

a pitch detector to generate a pitch estimate based on autocorrelation and using the first inertial sensor output, wherein the pitch estimate is obtained by (i) using an X, Y, or Z signal generated by the first inertial sensor that has a highest power level or (ii) using a combination of the X, Y, and Z signals generated by the first inertial sensor.

24. The system of claim 13, wherein the first inertial sensor and the second inertial sensor are accelerometers.