Gain control for a hearing aid with a facial movement detector

• US 9,048,798 B2
• Filed: 08/30/2013
• Issued: 06/02/2015
• Est. Priority Date: 08/30/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Park, KeeHyun, Sampat, Kuntal Dilipsinh
• Primary Examiner(s): NGUYEN, TUAN DUC
• Application Number:
  US14/015,390
• Publication Number:
  US 20150063615A1
• Time in Patent Office: 641 Days
• Field of Search: 381/23.1, 381/26, 381/92, 381/312, 381/313, 381/316, 381/319, 381/321
• US Class Current: 1/1
• CPC Class Codes: H03G 3/00 Gain control in amplifiers or frequency changers
• CPC Class Codes: H04R 25/00 Deaf-aid sets , i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception electrical stimulation of auditory nerves to promote the auditory function A61N1/36038; optical stimulation of auditory nerves to promote the auditory function A61N5/0622
• CPC Class Codes: H04R 25/48 using constructional means for obtaining a desired frequency response H04R25/652 takes precedence
• CPC Class Codes: H04S 7/304 For headphones

Abstract

Methods, systems and devices are provided for controlling a hearing aid. Embodiments include receiving an input audio signal from a microphone. Also, a facial movement indication may be received from a facial movement detector measured contemporaneously with the input audio signal. Additionally, whether a first movement pattern associated with the wearer speaking is a movement match to the facial movement indication may be determined. In this way, a first gain profile may be applied to the input audio signal for generating an augmented audio segment in response to determining the first movement pattern is the movement match to the facial movement indication. Also, a second gain profile may be applied to the input audio signal for generating the augmented audio segment in response to determining the first movement pattern does not match the facial movement indication. Thus, the augmented audio segment may be output.

Claims

1. A method of controlling an output of a hearing aid worn by a wearer, comprising:

receiving an input audio signal from a microphone;

receiving a facial movement indication from a facial movement detector measured contemporaneously with the input audio signal;

determining whether the facial movement indication matches a first movement pattern associated with the wearer speaking;

applying a first gain profile to the input audio signal for generating an augmented audio segment in response to determining that the facial movement indication matches the first movement pattern associated with the wearer speaking;

applying a second gain profile to the input audio signal for generating the augmented audio segment in response to determining the facial movement indication does not match the first movement pattern associated with the wearer speaking; and

outputting the augmented audio segment.

2. The method of claim 1, further comprising:

determining whether at least a portion of the input audio signal matches a first voice pattern associated with speech by the wearer, wherein applying the first gain profile is further in response to determining that the input audio signal matches the first voice pattern; and

applying the second gain profile to the input audio signal for generating the augmented audio segment in response to determining that the input audio signal does not match the first voice pattern.

3. The method of claim 1, wherein the facial movement detector is an electromyography sensor.

4. The method of claim 1, wherein the facial movement detector is configured on the hearing aid to engage skin of the wearer.

5. The method of claim 1, wherein the microphone includes a first hearing aid microphone and a second hearing aid microphone separate from the first hearing aid microphone, wherein applying the first gain profile is applied in response to determining that a threshold portion of the input audio signal was received from the first hearing aid microphone.

6. The method of claim 1, wherein the first gain profile applies a first set of gain values to multiple frequencies of the input audio signal, the first set of gain values based on a first frequency-dependent gain distribution of the wearer while the wearer is speaking.

7. The method of claim 1, wherein the second gain profile applies a second set of gain values to multiple frequencies of the input audio signal, the second set of gain values based on a second frequency-dependent gain distribution of the wearer while the wearer is not speaking.

8. The method of claim 1, further comprising:

determining whether the facial movement indication matches other wearer activity associated with the wearer speaking that is not a pattern of movement, wherein the first gain profile is applied to the input audio signal for generating the augmented audio segment in response to determining that the facial movement indication includes the other wearer activity.

9. The method of claim 1, further comprising:

receiving a selection input indicating whether the first gain profile should be applied, wherein the first gain profile is applied to the input audio signal in response to the selection input indicating the first gain profile should be applied and the second gain profile is applied to the input audio signal in response to the selection input indicating the first gain profile should not be applied.

10. A hearing aid for adjusting sound heard by a wearer, comprising:

a facial movement detector configured to detect facial movements of the wearer;

a microphone configured to receive an input audio signal in response to detecting sound;

a speaker for outputting an augmented audio segment;

a memory having stored thereon first movement pattern data associated with the wearer speaking; and

a processor coupled to the facial movement detector, the microphone, the speaker and the memory, wherein the processor is configured with processor-executable instructions to perform operations comprising;

receiving the input audio signal from the microphone;

receiving a facial movement indication from the facial movement detector measured contemporaneously with the input audio signal;

determining whether the facial movement indication matches the first movement pattern;

applying a first gain profile to the input audio signal for generating the augmented audio segment in response to determining that the facial movement indication matches the first movement pattern;

applying a second gain profile to the input audio signal for generating the augmented audio segment in response to determining the facial movement indication does not match the first movement pattern; and

outputting the augmented audio segment to the speaker.

11. The hearing aid of claim 10, wherein the processor is configured with the processor-executable instructions to perform operations further comprising:

determining whether at least a portion of the input audio signal matches a first voice pattern associated with speech by the wearer, wherein applying the first gain profile is further in response to determining that the input audio signal matches the first voice pattern; and

applying the second gain profile to the input audio signal for generating the augmented audio segment in response to determining that the input audio signal does not match the first voice pattern.

12. The hearing aid of claim 10, wherein the processor is configured with the processor-executable instructions to perform operations such that the facial movement detector is an electromyography sensor.

13. The hearing aid of claim 10, wherein the processor is configured with the processor-executable instructions to perform operations such that the facial movement detector is configured on the hearing aid to engage skin of the wearer.

14. The hearing aid of claim 10, wherein the processor is configured with the processor-executable instructions to perform operations such that the microphone includes a first hearing aid microphone and a second hearing aid microphone separate from the first hearing aid microphone, wherein applying the first gain profile is applied in response to determining that a threshold portion of the input audio signal was received from the first hearing aid microphone.

15. The hearing aid of claim 10, wherein the processor is configured with the processor-executable instructions to perform operations such that the first gain profile applies a first set of gain values to multiple frequencies of the input audio signal, the first set of gain values based on a first frequency-dependent gain distribution of the wearer while the wearer is speaking.

16. The hearing aid of claim 10, wherein the processor is configured with the processor-executable instructions to perform operations such that the second gain profile applies a second set of gain values to multiple frequencies of the input audio signal, the second set of gain values based on a second frequency-dependent gain distribution of the wearer while the wearer is not speaking.

17. The hearing aid of claim 10, wherein the processor is configured with the processor-executable instructions to perform operations further comprising:

determining whether the facial movement indication matches other wearer activity associated with the wearer speaking that is not a pattern of movement, wherein the first gain profile is applied to the input audio signal for generating the augmented audio segment in response to determining that the facial movement indication includes the other wearer activity.

18. The hearing aid of claim 10, wherein the processor is configured with the processor-executable instructions to perform operations further comprising:

receiving a selection input indicating whether the first gain profile should be applied, wherein the first gain profile is applied to the input audio signal in response to the selection input indicating the first gain profile should be applied and the second gain profile is applied to the input audio signal in response to the selection input indicating the first gain profile should not be applied.

19. A hearing aid for adjusting sound heard by a wearer, comprising:

means for receiving an input audio signal from a microphone;

means for receiving a facial movement indication from a facial movement detector measured contemporaneously with the input audio signal;

means for determining whether the facial movement indication matches a first movement pattern associated with the wearer speaking;

means for applying a first gain profile to the input audio signal for generating an augmented audio segment in response to determining that the facial movement indication matches the first movement pattern associated with the wearer speaking;

means for applying a second gain profile to the input audio signal for generating the augmented audio segment in response to determining the facial movement indication does not match the first movement pattern associated with the wearer speaking; and

means for outputting the augmented audio segment.

20. The hearing aid of claim 19, further comprising:

means for determining whether at least a portion of the input audio signal matches a first voice pattern associated with speech by the wearer, wherein applying the first gain profile is further in response to determining that the input audio signal matches the first voice pattern; and

means for applying the second gain profile to the input audio signal for generating the augmented audio segment in response to determining that the input audio signal does not match the first voice pattern.

21. The hearing aid of claim 19, wherein the facial movement detector is an electromyography sensor.

22. The hearing aid of claim 19, wherein the facial movement detector is configured on the hearing aid to engage skin of the wearer.

23. The hearing aid of claim 19, wherein the microphone includes a first hearing aid microphone and a second hearing aid microphone separate from the first hearing aid microphone, wherein applying the first gain profile is applied in response to determining that a threshold portion of the input audio signal was received from the first hearing aid microphone.

24. The hearing aid of claim 19, wherein the first gain profile applies a first set of gain values to multiple frequencies of the input audio signal, the first set of gain values based on a first frequency-dependent gain distribution of the wearer while the wearer is speaking.

25. The hearing aid of claim 19, wherein the second gain profile applies a second set of gain values to multiple frequencies of the input audio signal, the second set of gain values based on a second frequency-dependent gain distribution of the wearer while the wearer is not speaking.

26. The hearing aid of claim 19, further comprising:

means for determining whether the facial movement indication matches other wearer activity associated with the wearer speaking that is not a pattern of movement, wherein the first gain profile is applied to the input audio signal for generating the augmented audio segment in response to determining that the facial movement indication includes the other wearer activity.

27. The hearing aid of claim 19, further comprising:

means for receiving a selection input indicating whether the first gain profile should be applied, wherein the first gain profile is applied to the input audio signal in response to the selection input indicating the first gain profile should be applied and the second gain profile is applied to the input audio signal in response to the selection input indicating the first gain profile should not be applied.

28. A non-transitory computer readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a hearing aid worn by a wearer to perform operations for controlling an output of the hearing aid, the operations comprising:

receiving an input audio signal from a microphone;

receiving a facial movement indication from a facial movement detector measured contemporaneously with the input audio signal;

determining whether the facial movement indication matches a first movement pattern associated with the wearer speaking;

applying a first gain profile to the input audio signal for generating an augmented audio segment in response to determining that the facial movement indication matches the first movement pattern associated with the wearer speaking;

applying a second gain profile to the input audio signal for generating the augmented audio segment in response to determining the facial movement indication does not match the first movement pattern associated with the wearer speaking; and

outputting the augmented audio segment.

29. The non-transitory computer readable storage medium of claim 28, wherein the processor-executable instructions are configured to cause the processor of the hearing aid to perform operations further comprising:

determining whether at least a portion of the input audio signal matches a first voice pattern associated with speech by the wearer, wherein applying the first gain profile is further in response to determining that the input audio signal matches the first voice pattern; and

applying the second gain profile to the input audio signal for generating the augmented audio segment in response to determining that the input audio signal does not match the first voice pattern.

30. The non-transitory computer readable storage medium of claim 28, wherein the processor-executable instructions are configured to cause the processor of the hearing aid to perform operations such that the facial movement detector is an electromyography sensor.

31. The non-transitory computer readable storage medium of claim 28, wherein the processor-executable instructions are configured to cause the processor of the hearing aid to perform operations such that the facial movement detector is configured on the hearing aid to engage skin of the wearer.

32. The non-transitory computer readable storage medium of claim 28, wherein the processor-executable instructions are configured to cause the processor of the hearing aid to perform operations such that the microphone includes a first hearing aid microphone and a second hearing aid microphone separate from the first hearing aid microphone, wherein applying the first gain profile is applied in response to determining that a threshold portion of the input audio signal was received from the first hearing aid microphone.

33. The non-transitory computer readable storage medium of claim 28, wherein the processor-executable instructions are configured to cause the processor of the hearing aid to perform operations such that the first gain profile applies a first set of gain values to multiple frequencies of the input audio signal, the first set of gain values based on a first frequency-dependent gain distribution of the wearer while the wearer is speaking.

34. The non-transitory computer readable storage medium of claim 28, wherein the processor-executable instructions are configured to cause the processor of the hearing aid to perform operations such that the second gain profile applies a second set of gain values to multiple frequencies of the input audio signal, the second set of gain values based on a second frequency-dependent gain distribution of the wearer while the wearer is not speaking.

35. The non-transitory computer readable storage medium of claim 28, wherein the processor-executable instructions are configured to cause the processor of the hearing aid to perform operations further comprising:

determining whether the facial movement indication matches other wearer activity associated with the wearer speaking that is not a pattern of movement, wherein the first gain profile is applied to the input audio signal for generating the augmented audio segment in response to determining that the facial movement indication includes the other wearer activity.

36. The non-transitory computer readable storage medium of claim 28, wherein the processor-executable instructions are configured to cause the processor of the hearing aid to perform operations further comprising:

receiving a selection input indicating whether the first gain profile should be applied, wherein the first gain profile is applied to the input audio signal in response to the selection input indicating the first gain profile should be applied and the second gain profile is applied to the input audio signal in response to the selection input indicating the first gain profile should not be applied.