BACKGROUND ANALYSIS OF MENTAL STATE EXPRESSIONS

• US 20150206000A1
• Filed: 03/30/2015
• Published: 07/23/2015
• Est. Priority Date: 06/07/2010
• Status: Abandoned Application

• Current Assignee: Affectiva, Inc. (Smart Eye AB)
• Original Assignee: Affectiva, Inc. (Smart Eye AB)
• Inventors: el Kaliouby, Rana, Peacock, Timothy, Poulin, Gregory
• Application Number:
  US14/672,328
• Publication Number:
  US 20150206000A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 5/02055 Simultaneously evaluating both cardiovascular condition and temperature
• CPC Class Codes: A61B 5/02405 Determining heart rate variability
• CPC Class Codes: A61B 5/08 Detecting, measuring or recording devices for evaluating the respiratory organs A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/7475 User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment
• CPC Class Codes: H04L 51/02 using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
• CPC Class Codes: H04L 51/04 Real-time or near real-time messaging, e.g. instant messaging [IM]
• CPC Class Codes: H04L 51/52 for supporting social networking services

Abstract

Expression analysis is performed via a background process and provided to foreground applications that register for emotion services. The foreground services are provided notification when a particular, previously determined emotional state is detected. The emotional state can be identified using facial feature analysis and/or gesture analysis. Upon receiving the notification of the state from the background process, the foreground services perform an emotion response action. The emotion response action can include sending a reply message indicating that a desired emotional response has been detected, providing a reward, and/or generating an automatic like on a social media system.

Claims

1. A computer-implemented method for analysis comprising:

determining an expression to be detected on one or more devices;

monitoring, by background process on the one or more devices, for the expression;

identifying an occurrence of the expression; and

providing notification that the expression was identified.

2. The method of claim 1 further comprising enabling emotion services using the one or more devices.

3. The method of claim 2 wherein the emotion services include detecting smiles, expressions, frowns, heart rate, eyebrow raises, brow furrows, concentration, being expressive, and laughing.

4. The method of claim 2 wherein the emotion services are provided on a device.

5. The method of claim 2 wherein applications use the enabling.

6. The method of claim 1 wherein a foreground application receives the providing of the notification.

7. The method of claim 1 wherein a camera sensor is on continuously as part of the monitoring.

8. The method of claim 1 wherein the providing includes delivering a coupon or a reward.

9. The method of claim 8 wherein the providing is part of a gaming experience.

10. The method of claim 1 wherein the expression is a reflection of a mental state.

11. The method of claim 1 wherein the identifying is provided with a confidence interval.

12. The method of claim 1 wherein the monitoring is a background operation within the one or more devices.

13. The method of claim 1 wherein the monitoring is accomplished passively.

14. The method of claim 1 wherein the determining is accomplished by describing a mental state.

15. The method of claim 1 wherein the determining is accomplished by inputting an expression description.

16. The method of claim 1 wherein the determining is accomplished by providing an expression.

17. The method of claim 1 wherein the providing an expression is via an image.

18. The method of claim 1 wherein a foreground application receives the providing of the notification, and wherein the providing an expression is via an image, and further comprising:

enabling emotion services using the one or more devices; and

performing an emotion response action in response to the provided notification.

19. The method of claim 18 wherein the foreground application comprises a messaging application configured to send messages between a sender and a recipient, and wherein, upon the recipient accessing a first message, the background process identifies an occurrence of the identified expression, and wherein the emotion response action includes sending a second message to the sender which includes emotion scoring, wherein the second message is indicative of an emotional response of the recipient in response to the first message.

20. The method of claim 19 wherein the second message further includes the image.

21. The method of claim 19 wherein the second message further includes an emoji.

22. The method of claim 18 wherein the foreground application comprises a social media application configured to display a first message, and wherein, upon a recipient accessing the first message, the background process identifies an occurrence of the identified expression, and wherein the emotion response action includes posting a second message on a social media system which includes emotion scoring, wherein the second message is indicative of an emotional response of the recipient in response to the first message.

23. The method of claim 18 wherein the foreground application comprises a social media application configured to display a message, and wherein, upon a recipient accessing the message, the background process identifies an occurrence of the identified expression, and wherein the emotion response action includes generating an automatic like of the message.

24. The method of claim 18 wherein the emotion response action includes invoking an automated help application.

25. A computer program product embodied in a non-transitory computer readable medium for mental state analysis, the computer program product comprising:

code for determining an expression to be detected on one or more devices;

code for monitoring, by background process on the one or more devices, for the expression;

code for identifying an occurrence of the expression; and

code for providing notification that the expression was identified.

26. A computer system for mental state analysis comprising:

a memory which stores instructions;

one or more processors attached to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to;

determine an expression to be detected on one or more devices;

monitor, by background process on the one or more devices, for the expression;

identify an occurrence of the expression; and

provide notification that the expression was identified.