Gain control for an electro-acoustic device with a facial movement detector

• US 9,225,306 B2
• Filed: 11/17/2014
• Issued: 12/29/2015
• Est. Priority Date: 08/30/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Sampat, Kuntal Dilipsinh, Park, Kee-Hyun
• Primary Examiner(s): NGUYEN, TUAN DUC
• Application Number:
  US14/542,747
• Publication Number:
  US 20150086052A1
• Time in Patent Office: 407 Days
• Field of Search: 381/23.1, 381/26, 381/92, 381/312, 381/313, 381/316, 381/319, 381/321
• US Class Current: 1/1
• CPC Class Codes: H03G 3/00 Gain control in amplifiers or frequency changers
• CPC Class Codes: H04R 25/00 Deaf-aid sets , i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception electrical stimulation of auditory nerves to promote the auditory function A61N1/36038; optical stimulation of auditory nerves to promote the auditory function A61N5/0622
• CPC Class Codes: H04R 25/48 using constructional means for obtaining a desired frequency response H04R25/652 takes precedence
• CPC Class Codes: H04S 7/304 For headphones

Abstract

Methods, systems and devices are provided for controlling a wearable device. Embodiments include receiving a facial movement indication from a facial movement detector measured contemporaneously with an input audio signal. Additionally, whether a first movement pattern associated with the wearer speaking is a movement match to the facial movement indication may be determined. In this way, a speaking gain profile may be applied to the input audio signal for generating an augmented audio segment in response to determining the first movement pattern is the movement match to the facial movement indication. Thus, the augmented audio segment may be output.

Claims

1. A method of controlling an output of a wearable device worn by a wearer, comprising:

receiving a facial movement indication from a facial movement detector measured contemporaneously with an input audio signal;

determining whether the facial movement indication matches a first movement pattern associated with the wearer speaking;

applying a speaking gain profile to the input audio signal for generating an augmented audio segment in response to determining that the facial movement indication matches the first movement pattern associated with the wearer speaking; and

outputting the augmented audio segment.

2. The method of claim 1, further comprising:

determining whether at least a portion of the input audio signal matches a first voice pattern associated with speech by the wearer, wherein applying the speaking gain profile is further in response to determining that the input audio signal matches the first voice pattern.

3. The method of claim 1, further comprising:

applying a non-speaking gain profile to the input audio signal for generating the augmented audio segment in response to determining that the input audio signal does not match a first voice pattern.

4. The method of claim 1, wherein the facial movement detector is configured on the wearable device to engage skin of the wearer.

5. The method of claim 1, wherein the wearable device includes a first microphone and a second microphone separate from the first microphone, wherein the speaking gain profile is applied in response to determining that a threshold portion of the input audio signal was received from the first microphone.

6. The method of claim 1, wherein the speaking gain profile applies a first set of gain values to multiple frequencies of the input audio signal, the first set of gain values based on a first frequency-dependent gain distribution of the wearer while the wearer is speaking.

7. The method of claim 1, further comprising:

applying a non-speaking gain profile to the input audio signal for generating the augmented audio segment in response to determining the facial movement indication does not match the first movement pattern associated with the wearer speaking, wherein the non-speaking gain profile applies a second set of gain values to multiple frequencies of the input audio signal, the second set of gain values based on a second frequency-dependent gain distribution of the wearer while the wearer is not speaking.

8. The method of claim 1, further comprising:

determining whether the facial movement indication matches other wearer activity associated with the wearer speaking that is not a pattern of movement, wherein the speaking gain profile is applied to the input audio signal for generating the augmented audio segment in response to determining that the facial movement indication includes the other wearer activity.

9. The method of claim 1, further comprising:

receiving a selection input indicating whether the speaking gain profile should be applied, wherein the speaking gain profile is applied to the input audio signal in response to the selection input indicating the speaking gain profile should be applied.

10. The method of claim 1, further comprising:

determining a first voice pattern associated with speech by the wearer in response to receiving a setup input.

11. The method of claim 10, further comprising:

receiving a start recording input from an input mechanism configured to initiate a voice recording of the wearer used for determining the first voice pattern.

12. A wearable device for adjusting sound heard by a wearer, comprising:

a facial movement detector configured to detect facial movements of the wearer;

a speaker for outputting an augmented audio segment;

a memory having stored thereon first movement pattern data associated with the wearer speaking; and

a processor coupled to the facial movement detector, the speaker and the memory, wherein the processor is configured with processor-executable instructions to perform operations comprising;

receiving a facial movement indication from the facial movement detector measured contemporaneously with an input audio signal;

determining whether the facial movement indication matches the first movement pattern;

applying a speaking gain profile to the input audio signal for generating the augmented audio segment in response to determining that the facial movement indication matches the first movement pattern; and

outputting the augmented audio segment to the speaker.

13. The wearable device of claim 12, wherein the processor is configured with the processor-executable instructions to perform operations further comprising:

determining whether at least a portion of the input audio signal matches a first voice pattern associated with speech by the wearer, wherein applying the speaking gain profile is further in response to determining that the input audio signal matches the first voice pattern.

14. The wearable device of claim 12, wherein the processor is configured with the processor-executable instructions to perform operations such that the facial movement detector is an electromyography sensor.

15. The wearable device of claim 12, wherein the processor is configured with the processor-executable instructions to perform operations further comprising:

applying a non-speaking gain profile to the input audio signal for generating the augmented audio segment in response to determining that the input audio signal does not match a first voice pattern.

16. The wearable device of claim 12, wherein the processor is configured with the processor-executable instructions to perform operations such that the wearable device includes a first microphone and a second microphone separate from the first microphone, wherein the speaking gain profile is applied in response to determining that a threshold portion of the input audio signal was received from the first microphone.

17. The wearable device of claim 12, wherein the processor is configured with the processor-executable instructions to perform operations such that the speaking gain profile applies a first set of gain values to multiple frequencies of the input audio signal, the first set of gain values based on a first frequency-dependent gain distribution of the wearer while the wearer is speaking.

18. The wearable device of claim 12, wherein the processor is configured with the processor-executable instructions to perform operations further comprising:

applying a non-speaking gain profile to the input audio signal for generating the augmented audio segment in response to determining the facial movement indication does not match the first movement pattern associated with the wearer speaking, such that the non-speaking gain profile applies a second set of gain values to multiple frequencies of the input audio signal, the second set of gain values based on a second frequency-dependent gain distribution of the wearer while the wearer is not speaking.

19. The wearable device of claim 12, wherein the processor is configured with the processor-executable instructions to perform operations further comprising:

determining whether the facial movement indication matches other wearer activity associated with the wearer speaking that is not a pattern of movement, wherein the speaking gain profile is applied to the input audio signal for generating the augmented audio segment in response to determining that the facial movement indication includes the other wearer activity.

20. The wearable device of claim 12, wherein the processor is configured with the processor-executable instructions to perform operations further comprising:

receiving a selection input indicating whether the speaking gain profile should be applied, wherein the speaking gain profile is applied to the input audio signal in response to the selection input indicating the speaking gain profile should be applied.

21. The wearable device of claim 12, wherein the processor is configured with the processor-executable instructions to perform operations further comprising:

determining a first voice pattern associated with speech by the wearer in response to receiving a setup input.

22. The wearable device of claim 21, further comprising:

an input mechanism configured to receive a start recording input for initiating a voice recording of the wearer used for determining the first voice pattern.

23. A wearable device for adjusting sound heard by a wearer, comprising:

means for detecting a facial movement contemporaneously with an input audio signal;

means for determining whether detected facial movement matches a first movement pattern associated with the wearer speaking;

means for applying a speaking gain profile to the input audio signal for generating an augmented audio segment in response to determining that the detected facial movement matches the first movement pattern associated with the wearer speaking; and

means for outputting the augmented audio segment.

24. The wearable device of claim 23, further comprising:

means for determining whether at least a portion of the input audio signal matches a first voice pattern associated with speech by the wearer, wherein applying the speaking gain profile is further in response to determining that the input audio signal matches the first voice pattern.

25. The wearable device of claim 23, further comprising:

means for applying a non-speaking gain profile to the input audio signal for generating the augmented audio segment in response to determining that the input audio signal does not match a first voice pattern.

26. The wearable device of claim 23, further comprising:

means for applying a non-speaking gain profile to the input audio signal for generating the augmented audio segment in response to determining the facial movement does not match the first movement pattern associated with the wearer speaking, wherein the non-speaking gain profile applies a second set of gain values to multiple frequencies of the input audio signal, the second set of gain values based on a second frequency-dependent gain distribution of the wearer while the wearer is not speaking.

27. The wearable device of claim 23, further comprising:

means for determining whether the facial movement matches other wearer activity associated with the wearer speaking that is not a pattern of movement, wherein the speaking gain profile is applied to the input audio signal for generating the augmented audio segment in response to determining that the facial movement includes the other wearer activity.

28. The wearable device of claim 23, further comprising:

means for receiving a selection input indicating whether the speaking gain profile should be applied, wherein the speaking gain profile is applied to the input audio signal in response to the selection input indicating the speaking gain profile should be applied.

29. The wearable device of claim 23, further comprising:

means for receiving a start recording input configured to initiate a voice recording of the wearer; and

means for determining a first voice pattern associated with speech by the wearer from the voice recording.

30. A non-transitory computer readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a wearable device worn by a wearer to perform operations for controlling an output of the wearable device, the operations comprising:

receiving a facial movement indication from a facial movement detector measured contemporaneously with an input audio signal;

determining whether the facial movement indication matches a first movement pattern associated with the wearer speaking;

applying a speaking gain profile to the input audio signal for generating an augmented audio segment in response to determining that the facial movement indication matches the first movement pattern associated with the wearer speaking; and

outputting the augmented audio segment.