FACIAL CODING FOR EMOTIONAL INTERACTION ANALYSIS

• US 20130094722A1
• Filed: 12/03/2012
• Published: 04/18/2013
• Est. Priority Date: 08/13/2009
• Status: Active Grant

• Current Assignee: Sensory Logic Incorporated
• Original Assignee: Sensory Logic Incorporated
• Inventors: Hill, Daniel A.
• Granted Patent: US 8,929,616 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Various systems and techniques using facial coding for emotional interaction analysis are described herein. Machine-readable facial observations of a subject while the subject is exposed to a stimulus can be received. The machine readable observations can include a stimulus synchronization element. An emotional component of an emotional state of the subject can be determined based on the facial observations. The determination can include assigning a numerical weight to the emotional component. An emotional state to the stimulus synchronization event can be assigned based on the emotional component.

Claims

1. A machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:

receiving machine-readable facial observations of a subject while the subject is exposed to a stimulus, the machine readable observations including a stimulus synchronization element;

determining an emotional component of an emotional state of the subject based on the facial observations including assigning a numerical weight to the emotional component;

assigning an emotional state to the stimulus synchronization event based on the emotional component.

2. The medium of claim 1 comprising instructions that cause the machine to perform the operation of displaying a visual representation of the emotional state.

3. The medium of claim 2, wherein displaying the visual representation of the emotional state includes synchronizing the visual representation with a playback of the stimulus.

4. The medium of claim 3, wherein the visual representation includes a size based on the degree of the emotional state.

5. The medium of claim 4, wherein the visual representation includes a color based on the emotional state;

6. The medium of claim 2, wherein the emotional state is one of a plurality of emotional states for the subject while exposed to the stimulus, the emotional state corresponding to a sub-element of the stimulus, and each of the plurality of emotional states corresponding to respective other sub-elements of the stimulus.

7. The medium of claim 6, wherein the emotional state is identified as a peak among the plurality of emotional states.

8. The medium of claim 1, wherein the stimulus includes at least one of a visual component or

an audio component.

9. The medium of claim 1, wherein the emotional state includes at least one of an emotion;

a category of emotion, a valence of emotion, or an impact of emotion.

10. The medium of claim 1, wherein the synchronization element is at least one of time, a scene, a change in a visual aspect of the stimulus, a change in an audio aspect of the stimulus, or a change in subject matter of the stimulus.

11. A method comprising:

receiving machine-readable facial observations of a subject while the subject is exposed to a stimulus, the machine readable observations including a stimulus synchronization element;

determining an emotional component of an emotional state of the subject based on the facial observations including assigning a numerical weight to the emotional component;

assigning, using a hardware processor, an emotional state to the stimulus synchronization event based on the emotional component.

12. The method of claim 11 displaying, using the hardware processor, a visual representation of the emotional state.

13. The method of claim 12, wherein displaying the visual representation of the emotional state includes synchronizing the visual representation with a playback of the stimulus.

14. The method of claim 13, wherein the visual representation includes a size based on the degree of the emotional state.

15. The method of claim 14, wherein the visual representation includes a color based on the emotional state;

16. The method of claim 12, wherein the emotional state is one of a plurality of emotional states for the subject while exposed to the stimulus, the emotional state corresponding to a sub-element of the stimulus, and each of the plurality of emotional states corresponding to respective other sub-elements of the stimulus.

17. The method of claim 16, wherein the emotional state is identified as a peak among the plurality of emotional states.

18. The method of claim 11, wherein the stimulus includes at least one of a visual component or

an audio component.

19. The method of claim 11, wherein the emotional state includes at least one of an emotion;

a category of emotion, a valence of emotion, or an impact of emotion.

20. The method of claim 11, wherein the synchronization element is at least one of time, a scene, a change in a visual aspect of the stimulus, a change in an audio aspect of the stimulus, or a change in subject matter of the stimulus.