System and method to detect close voice sources and automatically enhance situation awareness

• US 9,270,244 B2
• Filed: 03/13/2014
• Issued: 02/23/2016
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Staton Techiya, LLC
• Original Assignee: Personics Holdings, Inc.
• Inventors: Usher, John, Goldstein, Steven W.
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US14/210,430
• Publication Number:
  US 20140270200A1
• Time in Patent Office: 712 Days
• Field of Search: 381/56, 381/57, 381/60, 381/312, 381/317, 381/318, 381/328, 381/17, 381/74, 381/107, 381/309, 704/229, 340/944, 345/7, 379/93.05, 379/207.02, 434/178, 455/570
• US Class Current: 1/1
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H03G 3/32 the control being dependent upon ambient noise level or sound level
• CPC Class Codes: H04R 1/1016 Earpieces of the intra-aural type
• CPC Class Codes: H04R 1/1041 Mechanical or electronic switches, or control elements switches in general H01H
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/107 Monophonic and stereophonic headphones with microphone for two-way hands free communication
• CPC Class Codes: H04R 2420/01 Input selection or mixing for amplifiers or loudspeakers for hearing aids H04R25/43
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A system and method for enhancing two-way conversation between a user wearing at least one communication system or earphone and a second individual in proximity to the user is provided. The method includes detecting a spoken voice of the user from sounds captured at an ambient sound microphone, automatically activating a voice timer and setting a voice activity threshold for the voice timer responsive to detecting spoken voice, and adjusting a mixing gain of an audio content signal delivered to the earphone or communication system by way of the internal speaker with an ambient sound pass-through during activation of the voice timer. Separate mixing gains can be adjusted independently in accordance with spoken voice activity and reproduced audio content characteristics. Other embodiments are disclosed.

Claims

1. A method for close proximity detection and automatic audio mixing performed by a processor suitable for use with an earphone, the method comprising the steps of:

monitoring sound from an ambient sound microphone communicatively coupled to the processor;

automatically activating a voice timer responsive to detecting voice activity or a cessation of voice activity in the sound as part of the close proximity detection;

adjusting a mixing gain of an audio content signal delivered to the earphone with the ambient sound pass-through during a voice timer pending voice activity; and

wherein the audio content is one of a voice signal, music content, or audible sound delivered to the internal speaker for audible reproduction.

2. The method of claim 1, further comprising adjusting an ambient sound pass-through of the ambient sound microphone during the voice timer pending voice activity, wherein the voice timer exceeds a time duration of the voice activity.

3. The method of claim 1, wherein the earphone is configured to provide partial sound isolation.

4. The method of claim 1, wherein the earphone is configured to provide full sound isolation.

5. The method of claim 1, further comprising the steps of:

detecting between a first voice of a user wearing the earphone and a second voice of another individual in proximity to the user;

determining a direction and proximity of the individual with respect to the user wearing the earphone; and

adjusting a gain of the ambient sound pass-through of the ambient sound microphone as a function of the direction.

6. The method of claim 1, further comprising detecting between a first voice of a user wearing the earphone and a second voice of another individual in proximity to the user by evaluating at least one or more of sound level, vocal pitch, resonance characteristics, sound-to-silence gap analysis, signal to noise ratios, and previously learned speaker characteristics or sound patterns.

7. The method of claim 5, further comprising the steps of:

determining the proximity of the individual with respect to the user wearing the earphone; and

adjusting the gain of the ambient sound pass-through of the ambient sound microphone as a function of the direction and proximity.

8. The method of claim 5, further comprising the steps of:

determining the proximity of the individual with respect to the user wearing the earphone; and

adjusting the beamforming of a plurality of microphones including the ambient sound microphone based on direction and proximity.

9. The method of claim 5, further comprising the steps of:

determining the proximity of the individual with respect to the user wearing the earphone; and

adjusting a signal to noise ratio of an ambient sound with respect to background noise and the audio content signal.

10. The method of claim 5, wherein the step of adjusting the ambient sound pass- through comprises:

increasing a gain of the ambient sound delivered by the ambient sound microphone when voice activity is detected above a threshold; and

decreasing the gain of the ambient sound delivered by the ambient sound microphone when voice activity is detected below a threshold.

11. The method of claim 10, wherein the threshold comprises sound pressure level and one or more voice characteristics of spectra, phase, timing, or onsets.

12. The method of claim 10, wherein the step of adjusting the mixing gain of an audio content signal comprises:

decreasing a volume gain of the audio content signal delivered to the earphone when voice activity is detected above a threshold; and

increasing the volume gain of the audio content signal delivered to the earphone when voice activity is detected below a threshold.

13. The method of claim 1, wherein the audio content is one of a voice signal, music content, or audible sound delivered from a mobile device.

14. The method of claim 1, wherein the adjusting an ambient sound pass-through gain of the ambient sound microphone increases the signal to noise ratio of the ambient sound with respect to background noise and the audio content.

15. The method of claim 5, further comprising detecting an orientation of the ambient microphone location with respect to a leaning of the user in a direction of the individual in proximity to the user, and adjusting the gain of the ambient sound pass-through of the ambient sound microphone as a function of the leaning.

16. The method of claim 5, further comprising enabling a side voice activity detection when the individual is in close proximity within 2 meters and to a left or right side of the user greater than 45 degrees from a front of the user.

17. An communication system for close proximity detection and audio mixing, comprising:

a first ambient sound microphone;

an internal speaker for generating audible sounds configured to be directed towards an ear canal; and

a processor operatively coupled to the communication system, the first ambient sound microphone and the internal speaker, wherein the processor

monitors sound from the first ambient sound microphone;

automatically activates a voice timer responsive to detecting voice activity or a cessation of voice activity in the sound as part of the close proximity detection;

adjusts a mixing gain of an audio content signal delivered to the communciation system by way of the internal speaker with an ambient sound pass-through during the voice timer; and

wherein the audio content is one of a voice signal, music content, or audible sound delivered to the internal speaker for audible reproduction.

18. The communication system of claim 17, wherein the processor adjusts an ambient sound pass-through of the ambient sound microphone reproduced by the internal speaker during the voice timer and wherein the voice timer is pending for voice activity and exceeds a time length of the voice activity.

19. The communication system of claim 17, wherein the communication system comprises one or more of a headset, a headphone, an earpiece, a mobile phone, a smart phone, a watch, a personal music player.

20. The communication system of claim 17, wherein the processor:

distinguishes between a first voice of a user wearing the earphone and a second voice of another individual in proximity to the user;

determines a direction and proximity of the individual with respect to the user wearing the earphone; and

adjusts a gain of the ambient sound pass-through of the ambient sound microphone as a function of the direction.

21. The communication system of claim 20, further comprising a second ambient microphone to capture ambient sounds, and where the processor localizes the direction and the proximity of an incoming acoustic sound from the individual.

22. The communication system of claim 20, wherein the processor:

determines the proximity of the individual with respect to the user wearing the earphone; and

adjusts the gain of the ambient sound pass-through of the ambient sound microphone reproduced by the internal speaker as a function of the direction and proximity.

23. The communication system of claim 20, wherein the step of adjusting the ambient sound pass-through comprises:

increases a gain of the ambient sound from the ambient sound microphone delivered by the internal speaker when voice activity is detected above a threshold; and

decreases the gain of the ambient sound from the ambient sound microphone delivered by the internal speaker when voice activity is detected below a threshold.

24. The communication system of claim 23, wherein the step of adjusting the mixing gain of an audio content signal comprises:

decreases a volume of the audio content signal delivered to the internal speaker when voice activity is detected above a threshold; and

increases the volume of the audio content signal delivered to the internal speaker when voice activity is detected below a threshold.

25. The communication system of claim 17, wherein the audio content is one of a voice signal, music content, or audible sound delivered from a mobile device to the internal speaker for audible reproduction.

26. The communication system of claim 17, wherein the adjusting an ambient sound pass-through of the ambient sound microphone increases the signal to noise ratio of the ambient sound with respect to background noise.

27. The communication system of claim 17, wherein the communication system provides sound isolation from ambient sounds external to an ear canal of the user in which the communication system is inserted and from acoustic sounds internal to the ear canal of the user.

28. A method for enhancing two-way conversation, the method implemented by a processor operatively coupled to an earphone comprising the steps of:

detecting a spoken voice from sounds captured at an ambient sound microphone communicatively coupled to the earphone;

automatically activating a voice timer and setting a voice activity threshold for the voice timer responsive to detecting spoken voice as part of a close proximity detection;

adjusting an ambient sound pass-through of the ambient sound microphone reproduced by an internal speaker of the earphone during activation of the voice timer while a level of the spoken voice is above the voice activity threshold; and

adjusting a mixing gain of an audio content signal delivered to the earphone by way of the internal speaker with the ambient sound pass-through during activation of the voice timer and the close proximity detection,

wherein the earphone is configured to provide sound isolation from ambient sounds external to an ear canal in which the earphone is inserted and from acoustic sounds internal to the ear canal,

wherein the audio content is one of a voice signal, music content, or audible sound delivered from a mobile device to the internal speaker for audible reproduction.

29. The method of claim 28, further comprising detecting frontal voice activity from another individual in proximity to the earphone, and adjusting the mixing gain of the audio content and the ambient sound pass-through of the earphone based on a combined voice activity detection level with combined voice activity time expirations of the detected spoken voice and the detected frontal voice activity from another individual.

30. The method of claim 29, further comprising:

decreasing the audio content signal and increasing the ambient sound pass-through when voice activity is detected;

starting a voice activity timer when voice activity ceases;

decreasing the ambient sound pass-through and increasing the audio content signal when the frontal voice activity by the another individual is not detected

monitoring the voice activity timer for combined voice activity of the detected spoken voice and the frontal voice activity from the another individual; and

maintaining ambient sound pass-through level and audio content signal level during the combined voice activity,

wherein the voice activity timer bridges gaps between voice activity of the detected spoken voice and the frontal voice activity of the another individual to a time length that is a function of the combined voice activity.