Laugh detector and system and method for tracking an emotional response to a media presentation

• US 7,889,073 B2
• Filed: 01/31/2008
• Issued: 02/15/2011
• Est. Priority Date: 01/31/2008
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Zalewski, Gary
• Primary Examiner(s): Hunnings; Travis R
• Application Number:
  US12/023,951
• Publication Number:
  US 20090195392A1
• Time in Patent Office: 1,111 Days
• Field of Search: 340/539.25
• US Class Current: 340/539.25
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Information in the form of emotional responses to a media presentation may be passively collected, for example by a microphone and/or a camera. This information may be tied to metadata at a time reference level in the media presentation and used to examine the content of the media presentation to assess a quality of, or user emotional response to, the content and/or to project the information onto a demographic. Passive collection of emotional responses may be used to add emotion as an element of speech or facial expression detection, to make use of such information, for example to judge the quality of content or to judge the nature of various individuals for future content that is to be provided to them or to those similarly situated demographically. Thus, the invention asks and answers such questions as: What makes people happy? What makes them laugh? What do they find interesting? Boring? Exciting?

Claims

1. An apparatus for using information in the form of passively collected emotional responses to a media presentation that is made to one or more individuals, comprising:

a media presentation perceivably exhibited to at least one individual having a profile;

a detector for passively capturing emotional responses of said at least one individual to said media presentation during exhibition of said media presentation;

a processor for producing a correlation that reflects a relationship between said at least one individual'"'"'s emotional responses and at least one portion of said media presentation which contained stimuli that elicited said emotional responses;

said processor identifying at least one group having a profile that is similar to the profile of said at least one individual;

wherein at least one or more of said portions of said media presentation is perceivably exhibited to said group;

wherein the at least one or more of said portions of said media that are perceivably exhibited to said group are selected based upon said correlation between said at least one individual'"'"'s emotional responses and at least one portion of said media presentation that contained stimuli that elicited said emotional response; and

wherein the at least one or more of said portions of said media that are perceivably exhibited to said group are presented to said group with some assurance of a positive reception by said group.

2. The apparatus of claim 1, further comprising:

said processor adapted to aggregate resulting emotional response data across a population of individuals to assess at least one aspect of said media presentation.

3. The apparatus of claim 1, further comprising:

said processor adapted to aggregate resulting emotional response data for an individual to establish a user profile in connection with at least one aspect of said media presentation.

4. The apparatus of claim 1, said processor further adapted to tie said resulting emotional response data to a media presentation timeline.

5. The apparatus of claim 4, said processor further adapted to tie said resulting emotional response data to a media presentation timeline at a frame level.

6. The apparatus of claim 1, said processor adapted to examine said media presentation content pursuant to said assessment of said at least one aspect of said media presentation.

7. The apparatus of claim 1, said processor adapted to project said resulting emotional response data onto one or more demographic categories.

8. The apparatus of claim 1, said processor adapted to use said emotional responses to said media presentation to identify appropriate and, conversely, inappropriate parts of expression in connection with said media presentation.

9. The apparatus of claim 1, said detector comprising any of:

a microphone for capturing utterances, a video camera for capturing gestures and/or facial expressions, an EKG, or a galvanic skin response device.

10. The apparatus of claim 1, said processor adapted to identify any metadata associated with said media presentation to establish a profile for said at least one individual.

11. The apparatus of claim 1, said emotional responses comprising any of:

laughter, excitement, interest, boredom, joy, sadness, anger, tone, and intensity and/or duration of the emotional response.

12. The apparatus of claim 1, said processor adapted to collect said emotional responses across a large population of individuals and to determine an overall understanding of the emotional effect of each response-invoking portion of said media presentation.

13. A method for using information in the form of passively collected emotional responses to a media presentation that is made to one or more individuals, comprising the steps of:

perceivably exhibiting a media presentation to at least one individual having a profile;

passively identifying detectable, emotional, visceral responses of said at least one individual to at least a portion of said media presentation; and

identifying any of the type of media presentation that was associated with said responses, the time at which said responses occurred during said media presentation, the location of said media presentation that caused said responses, and a value of said responses for either profiling for or for collecting group responses, said identified value being used for the steps of;

associating metadata with individual and group profiles;

producing a correlation that reflects a relationship between said at least one individual'"'"'s emotional responses and at least one portion of said media presentation which contained stimuli that elicited said emotional responses;

identifying at least one group having a profile that is similar to the profile of said at least one individual;

wherein at least one or more of said portions of said media presentation is perceivably exhibited to said group;

wherein the at least one or more of said portions of said media that are perceivably exhibited to said group are selected based upon said correlation between said at least one individual'"'"'s emotional responses and at least one portion of said media presentation that contained stimuli that elicited said emotional response; and

wherein the at least one or more of said portions of said media that are perceivably exhibited to said group are presented to said group with some assurance of a positive reception by said group.

14. The method of claim 13, said identified value being used to establish a profile for the step of:

presenting future targeted advertisements.

15. The method of claim 13, said identified value being used to collect a group response for the step of:

identifying media presentations or portions of media presentations of particular merit.

16. The method of claim 13, said identified value being used for the step of:

building a histogram to identify key aspects of said media presentation with regard to one or more predetermined factors.

17. The method of claim 13, said identifying step further comprising the step of:

identifying specific frames of specific sequences of said media presentation that provoke an emotional response in an individual.

18. The method of claim 13, further comprising the steps of:

in coordination with an individual'"'"'s profile, identifying traits of individual who demonstrated an emotional response to said media presentation; and

correlating said traits to those portions of said media presentation that elicited said emotional response.

19. The method of claim 17, further comprising the step of:

identifying objects within at least one specific frame in connection with said emotional response.

20. The method of claim 13, further comprising the step of:

identifying a temporal sequence within said media presentation that lead up to an emotional response.

21. The method of claim 13, further comprising the step of:

providing said individual with a biometric detection mechanism for use in connection with detection of an emotional response to link an identity for said individual'"'"'s with said emotional response.

22. The method of claim 13, further comprising the step of:

identifying said individual'"'"'s profile.

23. The method of claim 22, further comprising the step of:

using said individual'"'"'s profile to identify any of a network of friends, shared networks, and a demographic into which said individual may be placed.

24. The method of claim 13, further comprising the step of:

targeting advertising in specific markets for delivery to said individual based upon said individual'"'"'s emotional reaction to said media presentation.

25. The method of claim 13, said step of passively identifying detectable, emotional, visceral responses of said at least one individual to at least a portion of said media presentation further comprising the step of:

capturing any of gestures, profiles, movement, and shapes with a 3D camera for any of recognizing an individual, tracking emotional responses of more than one person at a time, tracking of gestures or other body movement as indications of emotional response, linking of gestures and/or body movement to other indications of emotional response, and identifying an emotional response of two or more individuals.

26. An apparatus for using information in the form of passively collected emotional responses to a media presentation that is made to one or more individuals, comprising:

a media presentation perceivably exhibited to a plurality of individuals, each individual having a profile;

a detector for passively and simultaneously capturing emotional responses of multiple individuals to said media presentation during exhibition of said media presentation; and

a processor for producing a correlation that reflects a relationship between said at least one individual'"'"'s emotional responses and at least one portion of said media presentation which contained stimuli that elicited said emotional responses;

said processor identifying at least one group having a profile that is similar to the profile of at least one of said individuals;

wherein at least one or more of said portions of said media presentation is perceivably exhibited to said group;

wherein the at least one or more of said portions of said media that are perceivably exhibited to said group are selected based upon said correlation between said at least one individual'"'"'s emotional responses and at least one portion of said media presentation that contained stimuli that elicited said emotional response; and

wherein the at least one or more of said portions of said media that are perceivably exhibited to said group are presented to said group with some assurance of a positive reception by said group.

27. The apparatus of claim 26, further comprising:

a network device placed in a location, said network device profiled to have a predetermined number of users; and

said processor programmed to use any of face and voice detection to match detected emotional responses with particular individuals who exhibited such emotional responses.

28. The apparatus of claim 26, further comprising:

said processor programmed to perform a calibration procedure to train said apparatus to recognize each individual of said plurality of individuals by any of voice and facial features.

29. A method for using information in the form of passively collected emotional responses of one or more individuals to a media presentation, comprising:

perceivably exhibiting a media presentation to at least one individual having a profile;

passively capturing emotional responses of said at least one individual to said media presentation during exhibition of said media presentation;

producing a correlation that reflects a relationship between said at least one individual'"'"'s emotional responses and at least one portion of said media presentation that contained stimuli that elicited said emotional response;

identifying at least one group having a profile that is similar to the profile of said at least one individual;

perceivably exhibiting at least one or more of said portions of said media presentation to said group;

wherein the at least one or more of said portions of said media that are perceivably exhibited to said group are selected based upon said correlation between said at least one individual'"'"'s emotional responses and at least one portion of said media presentation that contained stimuli that elicited said emotional response; and

wherein the at least one or more of said portions of said media that are perceivably exhibited to said group are presented to said group with some assurance of a positive reception by said group.