SYSTEMS AND METHODS USING OBSERVED EMOTIONAL DATA

• US 20130121591A1
• Filed: 11/14/2012
• Published: 05/16/2013
• Est. Priority Date: 11/14/2011
• Status: Active Grant

• Current Assignee: Sensory Logic Incorporated
• Original Assignee: Sensory Logic Incorporated
• Inventors: Hill, Daniel A.
• Granted Patent: US 8,903,176 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

Systems and techniques using observed emotional data are described herein. A sequence of visual observations of a subject can be received during execution of an application. An emotional state of the subject can be determined based on the sequence of visual observations. Execution of the application can be modified from a baseline execution using the emotional state.

Claims

1. A device comprising:

an image processing module configured to receive a sequence of visual observations of a subject during the execution of an application;

an emotion determination module configured to determine an emotional state of the subject based on the sequence of visual observations; and

a modification module configured to modify the execution of the application from a baseline execution using the emotional state.

2. The device of claim 1, wherein the emotion determination module is configured to identify a stimulus corresponding to the determined emotional state.

3. The device of claim 2, wherein to modify the execution of the application includes the modification module configured to store a correlation between the emotional state and the stimulus.

4. The device of claim 3, wherein the application is a social media application, wherein the stimulus is an article presented to the subject by the social media application, and wherein the correlation is an indication of at least one of like, dislike, or neutral indication of the article, the indication selected based on the emotional state.

5. The device of claim 3, wherein the application is a consumer marketing application, wherein the stimulus is presented in a consumer context, and wherein the correlation is a representation of the emotional state in a modified sequence of images correlated to the stimulus, the modified sequence of images including the sequence of images.

6. The device of claim 2, wherein the application is an interactive application, wherein the stimulus is a portion of the application for which help is available, wherein the emotional state crosses a threshold from a baseline emotional state, and wherein the modification module is configured to present the help to the subject.

7. The device of claim 2, wherein the application is an interactive application, wherein the stimulus is an element of the interactive application, wherein the element is one of a plurality of alternative items, and wherein the modification module is configured to select a next element from the plurality of alternative items based on the emotional state, the next element replacing the element in the interactive application.

8. The device of claim 1, wherein the application is a monitoring application, wherein the emotional state crosses a threshold from a baseline emotional state, and wherein the modification module is configured to intervene on behalf of the subject.

9. The device of claim 8, wherein to intervene, the modification module is configured to present an alarm to at least one of the subject or a user.

10. The device of claim 8, wherein to intervene, the modification module is configured to manipulate a physical aspect of an environment of the subject.

11. A massed machine-readable medium comprising instructions that, when executed by a machine, cause the machine to perform operations comprising:

receiving a sequence of visual observations of a subject during the execution of an application;

determining an emotional state of the subject based on the sequence of visual observations; and

modifying the execution of the application from a baseline execution using the emotional state.

12. The machine-readable medium of claim 11, wherein the operations comprise identifying a stimulus corresponding to the determined emotional state.

13. The machine-readable medium of claim 12, wherein modifying the execution of the application includes storing a correlation between the emotional state and the stimulus.

14. The machine-readable medium of claim 13, wherein the application is a social media application, wherein the stimulus is an article presented to the subject by the social media application, and wherein the correlation is an indication of at least one of like, dislike, or neutral indication of the article, the indication selected based on the emotional state.

15. The machine-readable medium of claim 13, wherein the application is a consumer marketing application, wherein the stimulus is presented in a consumer context, and wherein the correlation is a representation of the emotional state in a modified sequence of images correlated to the stimulus, the modified sequence of images including the sequence of images.

16. The machine-readable medium of claim 12, wherein the application is an interactive application, wherein the stimulus is a portion of the application for which help is available, wherein the emotional state crosses a threshold from a baseline emotional state, and wherein the operations comprise presenting the help to the subject.

17. The machine-readable medium of claim 12, wherein the application is an interactive application, wherein the stimulus is an element of the interactive application, wherein the element is one of a plurality of alternative items, and wherein the operations comprise selecting a next element from the plurality of alternative items based on the emotional state, the next element replacing the element in the interactive application.

18. The machine-readable medium of claim 11, wherein the application is a monitoring application, wherein the emotional state crosses a threshold from a baseline emotional state, and wherein the operations comprise intervening on behalf of the subject.

19. The machine-readable medium of claim 18, wherein the intervening includes presenting an alarm to at least one of the subject or a user.

20. The machine-readable medium of claim 18, wherein the intervening includes manipulating a physical aspect of an environment of the subject.

21. A method comprising:

receiving a sequence of visual observations of a subject during the execution of an application;

determining an emotional state of the subject based on the sequence of visual observations; and

modifying the execution of the application from a baseline execution using the emotional state.

22. The method of claim 21, comprising identifying a stimulus corresponding to the determined emotional state.

23. The method of claim 22, wherein modifying the execution of the application includes storing a correlation between the emotional state and the stimulus.

24. The method of claim 23, wherein the application is a social media application, wherein the stimulus is an article presented to the subject by the social media application, and wherein the correlation is an indication of at least one of like, dislike, or neutral indication of the article, the indication selected based on the emotional state.

25. The method of claim 23, wherein the application is a consumer marketing application, wherein the stimulus is presented in a consumer context, and wherein the correlation is a representation of the emotional state in a modified sequence of images correlated to the stimulus, the modified sequence of images including the sequence of images.

26. The method of claim 22 comprising presenting help to the subject, wherein the application is an interactive application, wherein the stimulus is a portion of the application for which the help is available, and wherein the emotional state crosses a threshold from a baseline emotional state.

27. The method of claim 22 comprising selecting a next element from a plurality of alternative items based on the emotional state, wherein the application is an interactive application, wherein the stimulus is an element of the interactive application, wherein the element is one of the plurality of alternative items, and wherein the next element replaces the element in the interactive application.

28. The method of claim 21 comprising intervening on behalf of the subject, wherein the application is a monitoring application, and wherein the emotional state crosses a threshold from a baseline emotional state.

29. The method of claim 28, wherein intervening includes presenting an alarm to at least one of the subject or a user.

30. The method of claim 28, wherein intervening includes manipulating a physical aspect of an environment containing the subject.