Method and apparatus for detecting facial changes

• US 9,830,507 B2
• Filed: 03/28/2011
• Issued: 11/28/2017
• Est. Priority Date: 03/28/2011
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Vetek, Akos, Lekkala, Jukka, Vehkaoja, Antti, Verho, Jarmo, Rantanen, Ville, Surakka, Veikko, Vanhala, Toni, Spakov, Oleg
• Primary Examiner(s): Hollington, Jermele M
• Assistant Examiner(s): Pothen, Feba
• Application Number:
  US13/073,820
• Publication Number:
  US 20120254244A1
• Time in Patent Office: 2,437 Days
• Field of Search: 324658-690
• CPC Class Codes: G06V 40/176 Dynamic expression

Abstract

In accordance with an example embodiment, there is an apparatus that has a headset, a contactless proximity sensor arranged to be supported by the headset in the proximity of a face of a user, and a sensor circuitry configured to cause output of a signal indicative of temporal variations in the distance between the contactless proximity sensor and the face of the user.

Claims

1. An apparatus, comprising:

a headset;

a contactless proximity sensor arranged to be supported by a microphone arm of the headset in the proximity of a face of a user, the sensor comprising a capacitive electrode configured to detect a capacitance between the electrode and a corner of the user'"'"'s mouth indicative of facial movements of the user'"'"'s face while the electrode and the corner of the user'"'"'s mouth are at a non-zero distance from one another, wherein the microphone arm of the headset comprises an arm including a microphone, the arm configured to extend over a cheek of the user to the proximity of the mouth of the user; and

a sensor circuitry configured to cause output of a signal indicative of a non-binary range of temporal variations in the distance between the capacitive electrode of the contactless proximity sensor and the corner of the user'"'"'s mouth.

2. The apparatus of claim 1, further comprising a wireless transmitter for the output of the signal indicative of temporal variations in the distance between the contactless proximity sensor and the corner of the user'"'"'s mouth.

3. The apparatus of claim 1, further comprising one or more further contactless proximity sensors for outputting with the sensor circuitry further signals indicative of temporal variations in the distance between respective contactless proximity sensor and the face of the user.

4. The apparatus of claim 3, wherein the contactless proximity sensors are arranged to be supported by the headset in the proximity of different parts of the face of the user.

5. The apparatus of claim 1, further comprising a wireless transmitter configured to wirelessly output the signal indicative of temporal variations in the distance between the contactless proximity sensor and the corner of the user'"'"'s mouth.

11. The apparatus of claim 1, wherein the detected capacitance between the electrode and the corner of the user'"'"'s mouth is indicative of facial expressions of the user.

15. The apparatus of claim 1, wherein the temporal variations in the distance between the capacitive electrode of the contactless proximity sensor and the corner of the user'"'"'s mouth indicate distinctions between smiling, grimacing and talking.

6. A method, comprising:

associating facial detection criteria with facial movements;

receiving capacitance variation information based on operation of one or more capacitive sensors attached to a headset in the proximity of a face of a user, wherein the headset comprises a microphone arm including a microphone, the microphone arm configured to extend over a cheek of the user to the proximity of the mouth of the user, the one or more capacitive sensors each comprising a capacitive electrode configured to detect a capacitance between the electrode and a corner of the user'"'"'s mouth indicative of facial movements of the user'"'"'s face while the capacitive electrode and the corner of the user'"'"'s mouth are at a non-zero distance from one another, and wherein the capacitance variation information comprises a signal indicative of a non-binary range of temporal variations in distance between the electrode and the corner of the user'"'"'s mouth;

searching for the associated facial changes from the received capacitance variation information using the facial detection criteria; and

determining the facial change that is associated with the matched facial detection criteria in question on finding a sufficient correspondence between the facial detection criteria and the received capacitance variation information.

7. The method of claim 6 further comprising repeatedly providing determinations of facial changes to a data sink.

8. The method of claim 6 further comprising monitoring if either the determination of facial changes has become impossible or a termination signal is received and responsive to either event taking place, terminating the performing of the method.

12. The method of claim 6, wherein the detected capacitance between the electrode and the corner of the user'"'"'s mouth is indicative of facial expressions of the user.

16. The method of claim 6, wherein determining the facial change that is associated with the matched facial detection criteria in question comprises determining distinctions between smiling, grimacing and talking.

9. An apparatus, comprising:

at least one processor; and

at least one memory including computer program code;

the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to perform at least the following;

associating facial detection criteria with facial movements;

receiving capacitance variation information based on operation of one or more capacitive sensors attached to a headset in the proximity of a face of a user, wherein the headset comprises a microphone arm including a microphone, the microphone arm configured to extend over a cheek of the user to the proximity of the mouth of the user, the one or more capacitive sensors each comprising a capacitive electrode configured to detect a capacitance between the electrode and a corner of the user'"'"'s mouth indicative of facial movements of the user'"'"'s face while the capacitive electrode and the corner of the user'"'"'s mouth are at a non-zero distance from one another, and wherein the capacitance variation information comprises a signal indicative of a non-binary range of temporal variations in distance between the electrode and the corner of the user'"'"'s mouth;

searching for the associated facial changes from the received capacitance variation information using the facial detection criteria; and

determining the facial change that is associated with the matched facial detection criteria in question on finding a sufficient correspondence between the facial detection criteria and the received capacitance variation information.

13. The apparatus of claim 9, wherein the detected capacitance between the electrode and the corner of the user'"'"'s mouth is indicative of facial expressions of the user.

10. A computer program product comprising a non-transitory computer-readable medium having computer program code embodied therein for use with a computer, the computer program code comprising:

code for associating facial detection criteria with facial movements;

code for receiving capacitance variation information based on operation of one or more capacitive sensors attached to a headset in the proximity of a face of a user, wherein the headset comprises a microphone arm including a microphone, the microphone arm configured to extend over a cheek of the user to the proximity of the mouth of the user, the one or more capacitive sensors each comprising a capacitive electrode configured to detect a capacitance between the electrode and a corner of the user'"'"'s mouth indicative of facial movements of the user'"'"'s face while the capacitive electrode and the corner of the user'"'"'s mouth are at a non-zero distance from one another, and wherein the capacitance variation information comprises a signal indicative of a non-binary range of temporal variations in distance between the electrode and the corner of the user'"'"'s mouth;

code for searching for the associated facial changes from the received capacitance variation information using the facial detection criteria; and

code for determining the facial change that is associated with the matched facial detection criteria in question on finding a sufficient correspondence between the facial detection criteria and the received capacitance variation information.

14. The computer program product of claim 10, wherein the detected capacitance between the electrode and the corner of the user'"'"'s mouth is indicative of facial expressions of the user.

17. The computer program product of claim 10, wherein determining the facial change that is associated with the matched facial detection criteria in question comprises determining distinctions between smiling, grimacing and talking.