Detecting the positions of earbuds and use of these positions for selecting the optimum microphones in a headset

• US 9,516,442 B1
• Filed: 01/21/2014
• Issued: 12/06/2016
• Est. Priority Date: 09/28/2012
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Dusan, Sorin V., Kanaris, Alexander, Lindahl, Aram M.
• Primary Examiner(s): Tran, Quoc D
• Assistant Examiner(s): Zhu, Qin
• Application Number:
  US14/160,427
• Time in Patent Office: 1,050 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04R 1/1016 Earpieces of the intra-aural type
• CPC Class Codes: H04R 1/222 for microphones H04R1/24, H04R1/26 take precedence
• CPC Class Codes: H04R 2201/403 Linear arrays of transducers
• CPC Class Codes: H04R 2201/405 Non-uniform arrays of transducers or a plurality of uniform arrays with different transducer spacing
• CPC Class Codes: H04R 29/001 for loudspeakers H04R29/007 takes precedence

Abstract

Embodiments of the invention determine whether speaker earbuds of a headset are positioned in a user'"'"'s ears. The headset may be a “Y” shaped headset with two earbuds having speakers and a plug for insertion into a jack of the audio device. Multiple microphones are located on wired lengths to the earbuds and a common wire between the lengths and the plug, to receive speech from the user'"'"'s mouth. Each earbud may have a front and rear microphone, and an accelerometer. Embodiments can detect user speech vibrations at one or more of the microphones, and in the accelerometers in the earbuds. Based on these detections, it can be determined whether one or both of the earbuds are in user'"'"'s ears. To provide more accurate beamforming, when only one of the earbuds is in the user'"'"'s ears, only the microphones leading to that earbud are selected for beamforming input.

Claims

1. A method for operating a headset, the method comprising:

detecting an audio signal through at least one of a plurality of microphones of the headset having a plurality of earbuds including a first earbud and a second earbud;

detecting the audio signal through at least one of a plurality of accelerometers that are not the same items as the microphones and that are located in the earbuds by (1) high pass filtering at least one output of the at least one of the plurality of accelerometers to pass frequencies of audible sound and (2) generating a binary signal that indicates detection of frequencies of audible sound in the at least one output;

based on the detections of the audio signal through the at least one of the microphones and through the at least one of the accelerometers, determining whether one or two of the first and second earbuds are in an ear of a user, wherein determining includes using the binary signal; and

if it is determined that only one earbud is in an ear of the user, selecting one or more microphones of the only one earbud, for user beamforming data input.

2. The method of claim 1, wherein the one or more microphones selected for user beamforming data input excludes microphones of the one earbud determined not to be in the ears of the user.

3. The method of claim 1, further comprising, if it is determined that two earbuds are in ears of the user, selecting the microphones of the two earbuds for the user beamforming data input.

4. The method of claim 1, wherein detecting the audio signal through at least one of a plurality of accelerometers comprises performing accelerometer voice activity detection comprising:

one of;

converting a plurality of direction vibration signals of the high pass filtered at least one output of the at least one of a plurality of accelerometers into a power signal to determine an amount of vibration of the at least one of a plurality of accelerometers in each dimension;

performing a normalized cross-correlation between a pair of orthogonal accelerometer output signals of the high pass filtered at least one output of the at least one of a plurality of accelerometers to determine an amount of vibration of the at least one of a plurality of accelerometers in two dimensions;

or

computing a normalized cross-correlation between three pairs of orthogonal accelerometer output signals of the high pass filtered at least one output of the at least one of a plurality of accelerometers to determine an amount of vibration of the at least one of a plurality of accelerometers in three dimensions, and selecting a pair of orthogonal accelerometer output signals with the strongest cross correlation.

5. The method of claim 4, wherein detecting the audio signal through at least one of a plurality of microphones comprises:

performing microphone voice activity detection using the audio signal detected at the at least one of a plurality of microphones.

6. The method of claim 4, wherein determining comprises combining the accelerometer voice activity detection with microphone voice activity detection from any one or more of the microphones.

7. The method of claim 4, wherein detecting the audio signal through at least one of a plurality of microphones includes filtering to pass only frequencies of sound for speech;

and wherein detecting the audio signal through at least one accelerometer includes detecting user speech vibrations in the accelerometer.

8. The method of claim 1, wherein detecting the audio signal through at least one accelerometer includes:

cross correlating two orthogonal signals of the high pass filtered at least one output of the at least one of a plurality of accelerometers to produce a normalized cross correlated output signal; and

detecting the audio signal while the normalized cross correlated output signal within a short delay interval exceeds a threshold.

9. The method of claim 8, wherein detecting the audio signal through at least one accelerometer includes:

removing cross talk in the accelerometer signals resulting from output of an earbud speaker; and

wherein detecting the audio signal while the normalized cross correlated output signal exceeds a threshold includes;

computing a maximum of the normalized cross correlated output signal during a predetermined short delay interval of time.

20. The method of claim 1, wherein determining whether one or two of the first and second earbuds are in an ear of the user includes determining whether the outputs of the front and rear microphones display significant corresponding or correlated energy.

21. The method of claim 1, determining whether one or two of the first and second earbuds are in an ear of the user includes:

identifying speech from the user using the output of one of the plurality of microphones;

identifying speech from the user using the output of one of the plurality of accelerometers;

and when speech from the user is identified in the output of the microphone and in the output of the accelerometer, determine that an earbud is in the ear of the user.

24. The method of claim 1, further comprising:

detecting the audio signal through at least another of the plurality of accelerometers that are located in another of the earbuds by (1) high pass filtering at least another output of the at least another of the plurality of accelerometers to pass frequencies of audible sound and (2) generating another binary signal that indicates detection of frequencies of audible sound in the at least another output;

based on the detections of the audio signal through the at least one of the microphones, through the at least one of the accelerometers, and through the at least another of the accelerometers, determining whether one or two of the first and second earbuds are in ears of the user, wherein determining includes using the binary signal and using the another binary signal; and

if it is determined that only one earbud is in an ear of the user, selecting one or more microphones of the only one earbud, for user beamforming data input.

10. An apparatus to detect whether at least one of a plurality of earbuds of a headset including a first earbud and a second earbud is in an ear of a user, the apparatus comprising:

microphone voice detection circuitry to detect an audio signal through at least one of a plurality of microphones in the at least one of the earbuds;

accelerometer voice detection circuitry to detect the audio signal through at least one of a plurality of accelerometers that are not the same items as the microphones and that are located in the earbuds by (1) high pass filtering at least one output of the at least one of the plurality of accelerometers to pass frequencies of sound and (2) generating a binary signal that indicates detection of frequencies of audible sound in the at least one output; and

earbud position detection circuitry to, based on both of the detections of the audio signal through the at last one of the microphones and through the at least one of the accelerometers, determine whether one or two of the first and second earbuds are in an ear of the user, wherein determining includes using the binary signal.

11. The apparatus of claim 10, wherein the accelerometers are voice vibration detection accelerometers;

and wherein the apparatus is an electronic audio computing device comprising;

communication circuitry that communicates with the headset, wherein the communication circuitry has corresponding channels to receive signals from the microphones and accelerometers, and for sending signals to speakers in the earbuds.

12. The apparatus of claim 10 further comprising beamforming circuitry to, if it is determined that only one earbud of the at least one of the earbuds is in an ear of the user, select one or more microphones of the one earbud in the ears of the user, for user beamforming data input.

13. The apparatus of claim 10, the accelerometer voice detection circuitry further comprising accelerometer voice activity detection circuitry to:

one of;

convert a plurality of direction vibration signals of the high pass filtered at least one output of the at least one of a plurality of accelerometers into a power signal to determine an amount of vibration of the at least one of a plurality of accelerometers in each dimension;

perform a normalized cross-correlation between a pair of orthogonal accelerometer output signals of the high pass filtered at least one output of the at least one of a plurality of accelerometers to determine an amount of vibration of the at least one of a plurality of accelerometers in two dimensions;

or

compute a normalized cross-correlation between three pairs of orthogonal accelerometer output signals of the high pass filtered at least one output of the at least one of a plurality of accelerometer to determine an amount of vibration of the at least one of a plurality of accelerometer in three dimensions, and selecting a pair of orthogonal accelerometer output signals with the strongest cross correlation.

14. The apparatus of claim 13, the microphone voice detection circuitry further comprising microphone voice activity detection circuitry to perform microphone voice activity detection using the audio signal detected at the at least one of a plurality of microphones of the headset.

15. The apparatus of claim 13, the earbud position detection circuitry further comprising combining circuitry to combine the accelerometer voice activity detection with microphone voice activity detection from any one or more of the microphones.

16. The apparatus of claim 10, the accelerometer voice detection circuitry further comprising circuitry to:

compute normalized cross correlation of two orthogonal signals of the high pass filtered at least one output of the at least one of a plurality of accelerometers; and

detect the audio signal while a maximum on a short delay interval of the normalized cross correlated output signal exceeds a threshold.

25. The apparatus of claim 10, further comprising:

the accelerometer voice detection circuitry detecting the audio signal through at least another of the plurality of accelerometers that are located in another of the earbuds by (1) high pass filtering at least another output of the at least another of the plurality of accelerometers to pass frequencies of audible sound and (2) generating another binary signal that indicates detection of frequencies of audible sound in the at least another output;

based on the detections of the audio signal through the at least one of the microphones, through the at least one of the accelerometers, and through the at least another of the accelerometers, the earbud position detection circuitry determining whether one or two of the first and second earbuds are in ears of the user, wherein determining includes using the binary signal and using the another binary signal; and

if it is determined that only one earbud is in an ear of the user, selecting one or more microphones of the only one earbud, for user beamforming data input.

17. A non-transitory computer-readable medium storing data and instructions to cause a programmable processor to perform operations for operating a headset, the operations comprising:

detecting an audio signal through at least one of a plurality of microphones of the headset having a plurality of earbuds including a first earbud and a second earbud;

detecting the audio signal through at least one of a plurality of accelerometers that are not the same items as the microphones and that are located in the earbuds by (1) high pass filtering at least one output of the at least one of the plurality of accelerometers to pass frequencies of audible sound and (2) generating a binary signal that indicates detection of frequencies of audible sound in the at least one output;

based on the detections of the same audio signal through the at last one of the microphones and through the at least one of the accelerometers, determining whether one or two of the first and second earbuds are in an ear of the user;

based on the detections of the same audio signal through the at least one of the microphones and through the at least one of the accelerometers, determining whether one or two of the first and second earbuds are in an ear of the user, wherein determining includes using the binary signal; and

if it is determined that only one earbud is in an ear of the user, selecting one or more microphones of the only one earbud, for user beamforming data input; and

if it is determined that both earbuds are in the ears of the user, selecting one or more microphones for user beamforming data input.

18. The medium of claim 17, wherein detecting the audio signal through at least one of the microphones comprises:

performing microphone voice activity detection using the audio signal detected at the at least one of a plurality of microphones;

and wherein detecting the audio signal through at least one of the accelerometers comprises performing accelerometer voice activity detection comprising;

one of;

converting a plurality of direction vibration signals of the high pass filtered least one output of the at least one of a plurality of accelerometers into a positive power signal to determine an amount of vibration of the at least one of a plurality of accelerometer in each dimension;

performing a normalized cross-correlation between a pair of orthogonal accelerometer output signals of the high pass filtered at least one output of the at least one of a plurality of accelerometers to determine an amount of vibration of the at least one of a plurality of accelerometers in two dimensions;

or

combining the accelerometer voice activity detection with microphone voice activity detection from any one or more of the microphones.

19. The medium of claim 17, wherein detecting the audio signal through at least one accelerometer includes:

computing cross correlation of two orthogonal signals of the high pass filtered at least one output of the at least one of a plurality of accelerometers to produce a normalized cross correlation output signal; and

detecting the audio signal while the normalized cross correlated output signal exceeds a threshold.

26. The medium of claim 17, the operations further comprising:

detecting the audio signal through at least another of the plurality of accelerometers that are located in another of earbuds by (1) high pass filtering at least another output of the at least another of the plurality of accelerometers to pass frequencies of audible sound and (2) generating another binary signal that indicates detection of frequencies of audible sound in the at least another output;

based on the detections of the audio signal through the at least one of the microphones, through the at least one of the accelerometers, and through the at least another of the accelerometers, determining whether one or two of the first and second earbuds are in ears of the user, wherein determining includes using the binary signal and using the another binary signal; and

if it is determined that only one earbud is in an ear of the user, selecting one or more microphones of the only one earbud, for user beamforming data input.

22. A method for operating a headset, the method comprising:

detecting an audio signal through at least one of a plurality of microphones of the headset having a plurality of earbuds including a first earbud and a second earbud;

detecting the audio signal through at least one of a plurality of accelerometers that are not the same items as the microphones and that are located in the earbuds by (1) high pass filtering at least one output of the at least one of the plurality of accelerometers to pass frequencies of audible sound and (2) generating a binary signal that indicates detection of frequencies of audible sound in the at least one output;

based on the detections of the audio signal through the at last one of the microphones and through the at least one of the accelerometers, determining whether one or two of the first and second earbuds are in an ear of the user, wherein determining includes using the binary signal, and wherein detecting the audio signal through at least one of the accelerometers comprises performing accelerometer voice activity detection comprising;

filtering out a direct current (DC) power level output of the at least one of a plurality of accelerometers;

converting a plurality of direction vibration signals of the at least one of a plurality of accelerometers into a power signal to determine an amount of vibration of each of the at least one of a plurality of accelerometers; and

if it is determined that only one earbud is in an ear of the user, selecting one or more microphones of the only one earbud, for user beamforming data input.

23. The method of claim 22, wherein detecting the audio signal through the at least one of a plurality of accelerometers includes filtering to pass only frequencies of sound for speech.