USING PHYSICAL SENSORY INPUT TO DETERMINE HUMAN RESPONSE TO MULTIMEDIA CONTENT DISPLAYED ON A MOBILE DEVICE

• US 20140052682A1
• Filed: 08/16/2012
• Published: 02/20/2014
• Est. Priority Date: 08/16/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: PHAN, Thomas, JEON, Won
• Granted Patent: US 9,241,664 B2
• Time in Patent Office: Days
• US Class Current: 706/46
• CPC Class Codes: A61B 5/0022 Monitoring a patient using a global network, e.g. telephone networks, internet
• CPC Class Codes: A61B 5/01 Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue clinical contact thermometers G01K13/20
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/0816 Measuring devices for examining respiratory frequency measuring frequency of electric signals G01R23/00
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/167 Personality evaluation
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: G16H 40/67 for remote operation

Abstract

A mobile handset device collects sensor data about the physiological state of the user of the handset. The mobile handset device receives multimedia content, which is consumed on the mobile handset. In a deployment phase, the sensor data is used to classify the user'"'"'s emotional response to individual pieces of media content consumed on the mobile device. A classification model built in a training phase may be used to map sensor data to classification labels indicative of the user'"'"'s emotional response to multimedia.

Claims

1. A mobile handset device, comprising:

at least one processor and a memory;

a user interface having a display;

a multimedia response module within the mobile handset device configured to collect sensor data from a set of sensors indicative of a physiological response of a user of the mobile handset device; and

the multimedia response module monitoring sensor data associated with the physiological response of the user to multimedia consumed on the mobile handset device and generating an output indicative of the emotional response of the user to the media.

2. The mobile handset device of claim 1, wherein the mobile handset device includes at least one sensor of the set of sensors.

3. The mobile handset device of claim 1, wherein the multimedia response module is configured to determine a classification label of the user'"'"'s emotional state by associating sensory inputs with a classification model and reporting on the classification determination.

4. The mobile handset device of claim 3, wherein the report is utilized to generate aggregate data for a media creator or publisher.

5. The mobile handset device of claim 1, wherein the multimedia response module includes a training phase to determine an association between sensory data and an emotional state of the user.

6. The mobile handset device of claim 5, wherein in the training phase, a user is polled on their emotional response to test multimedia to provide user polling data, and to determine an association between sensor inputs and classification labels for emotional response.

7. The mobile handset device of claim 1, wherein the sensor inputs include sensor input indicative, directly or indirectly, of at least one of a heart rate, respiration rate, galvanic skin response, temperature, pressure, acceleration, motion response, skin flush response, eye blinking response, and a vocal response.

8. A method of analyzing the effectiveness of media content displayed on a mobile handset device, comprising:

providing multimedia to a multiplicity of mobile handset devices, where each mobile handset device is configured to record sensor data indicative of a physiological response of a user of the mobile handset device;

receiving indicator data from each of the multiplicity of mobile handset devices, the indicator data being indicative of the emotional response to a particular piece of media played by the respective mobile handset device; and

determining an aggregated emotional response classification label for at least one piece of media.

9. The method of claim 8, wherein the indicator data includes an emotional classification label within a classification model based on a summary of sensor data.

10. The method of claim 8, further comprising in a training phase requesting test subjects to provide a self-assessment of emotional state in response to test media.

11. The method of claim 8, further comprising generating a classification model mapping a set of classification labels to sensor input data.

12. The method of claim 8, further comprising providing aggregated emotional response information to each provider of the at least one piece of media.

13. A method of analyzing the effectiveness of media content, comprising:

receiving indicator data from a multiplicity of mobile handset devices that is indicative of the emotional response to multimedia consumed by users of individual mobile handset devices; and

aggregating the indicator data and determining an average emotional response within the classification model for at least one piece of media content.

14. The method of claim 13, wherein the indicator data includes a summary of sensor data provided by individual mobile handset devices.

15. The method of claim 14, wherein the indicator data includes a classification label generated by individual mobile handset devices according to a classification model.

16. The method of claim 13, further comprising providing a classification model mapping a set of emotional response classification labels to a tree of physiological sensor data ranges for a user of the mobile handset device.

17. A computer program product comprising computer program code stored on a non-transitory computer readable medium which, when executed on a processor of a mobile handset device, implements a method comprising:

collecting sensor data from a set of sensors proximate to a mobile handset device indicative of a physiological response of a user of the mobile handset device to media consumed on the mobile handset device; and

generating an output indicative of an emotional response of the user to at least one piece of media content.

18. The computer program product of claim 17, further comprising computer program code to associating sensor data with a classification model and determine a classification label for the emotional response of the user to a piece of media content.

19. The computer program product of claim 17, further comprising computer program code to determine an association between sensory data and an emotional response of the user to media content.

20. The computer program product of claim 17, further comprising computer program code to provide user polling data to determine an association between sensor inputs and emotional response to media in a training phase.