Facial coding for emotional interaction analysis

• US 8,929,616 B2
• Filed: 12/03/2012
• Issued: 01/06/2015
• Est. Priority Date: 08/13/2009
• Status: Expired due to Fees

• Current Assignee: Sensory Logic Incorporated
• Original Assignee: Sensory Logic Incorporated
• Inventors: Hill, Daniel A.
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/692,142
• Publication Number:
  US 20130094722A1
• Time in Patent Office: 764 Days
• Field of Search: 382/118, 382/190
• US Class Current: 382/118
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Various systems and techniques using facial coding for emotional interaction analysis are described herein. Machine-readable facial observations of a subject while the subject is exposed to a stimulus can be received. The machine readable observations can include a stimulus synchronization element. An emotional component of an emotional state of the subject can be determined based on the facial observations. The determination can include assigning a numerical weight to the emotional component. An emotional state to the stimulus synchronization event can be assigned based on the emotional component.

Claims

1. A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:

receiving machine-readable facial observations of a subject while the subject is exposed to a stimulus, the machine readable observations including a stimulus synchronization element;

determining an emotional component of an emotional state of the subject based on the facial observations including assigning a numerical weight to the emotional component;

assigning an emotional state to the stimulus synchronization event based on the emotional component; and

displaying a visual representation of the emotional state, wherein displaying the visual representation of the emotional state includes synchronizing the visual representation with a playback of the stimulus.

2. The medium of claim 1, wherein the visual representation includes a size based on the degree of the emotional state.

3. The medium of claim 2, wherein the visual representation includes a color based on the emotional state.

4. The medium of claim 1, wherein the emotional state is one of a plurality of emotional states for the subject while exposed to the stimulus, the emotional state corresponding to a sub-element of the stimulus, and each of the plurality of emotional states corresponding to respective other sub-elements of the stimulus.

5. The medium of claim 4, wherein the emotional state is identified as a peak among the plurality of emotional states.

6. The medium of claim 1, wherein the stimulus includes at least one of a visual component or an audio component.

7. The medium of claim 1, wherein the emotional state includes at least one of an emotion;

a category of emotion, a valence of emotion, or an impact of emotion.

8. The medium of claim 1, wherein the synchronization element is at least one of time, a scene, a change in a visual aspect of the stimulus, a change in an audio aspect of the stimulus, or a change in subject matter of the stimulus.

9. A method comprising:

receiving machine-readable facial observations of a subject while the subject is exposed to a stimulus, the machine readable observations including a stimulus synchronization element;

determining an emotional component of an emotional state of the subject based on the facial observations including assigning a numerical weight to the emotional component;

assigning, using a hardware processor, an emotional state to the stimulus synchronization event based on the emotional component; and

displaying, using the hardware processor, a visual representation of the emotional state, wherein displaying the visual representation of the emotional state includes synchronizing the visual representation with a playback of the stimulus.

10. The method of claim 9, wherein the visual representation includes a size based on the degree of the emotional state.

11. The method of claim 10, wherein the visual representation includes a color based on the emotional state.

12. The method of claim 9, wherein the emotional state is one of a plurality of emotional states for the subject while exposed to the stimulus, the emotional state corresponding to a sub-element of the stimulus, and each of the plurality of emotional states corresponding to respective other sub-elements of the stimulus.

13. The method of claim 12, wherein the emotional state is identified as a peak among the plurality of emotional states.

14. The method of claim 9, wherein the stimulus includes at least one of a visual component or an audio component.

15. The method of claim 9, wherein the emotional state includes at least one of an emotion;

a category of emotion, a valence of emotion, or an impact of emotion.

16. The method of claim 9, wherein the synchronization element is at least one of time, a scene, a change in a visual aspect of the stimulus, a change in an audio aspect of the stimulus, or a change in subject matter of the stimulus.