Mental state analysis for norm generation

• US 9,959,549 B2
• Filed: 01/15/2015
• Issued: 05/01/2018
• Est. Priority Date: 06/07/2010
• Status: Active Grant

• Current Assignee: Affectiva, Inc. (Smart Eye AB)
• Original Assignee: Affectiva, Inc. (Smart Eye AB)
• Inventors: Kodra, Evan, el Kaliouby, Rana, Peacock, Timothy, Poulin, Gregory
• Primary Examiner(s): TUCKER, WESLEY J
• Application Number:
  US14/598,067
• Publication Number:
  US 20150142553A1
• Time in Patent Office: 1,202 Days
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: G06Q 30/0242 Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0271 Personalized advertisement
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment

Abstract

Mental state data is gathered from a plurality of people and analyzed in order to determine mental state information. Metrics are generated based on the mental state information gathered as the people view media presentations. Norms, defined as the quantitative measures of the mental states of a plurality of people as they view the media presentation, are determined based on the mental state information metrics. The norms can be determined based on various viewer criteria including country of residence, demographic group, or device type on which the media presentation is viewed. Responses to new media are then compared against norms to determine the effectiveness of the new media presentations.

Claims

1. A computer-implemented method for mental state analysis comprising:

obtaining mental state data from a plurality of people by capturing images of the plurality of people and analyzing the images to determine the mental state data, wherein, for at least one person of the plurality of people, the capturing is performed while the at least one person views a media presentation;

analyzing the mental state data to produce mental state information for the plurality of people;

generating a metric based on the mental state information;

generating a statistical analysis for the metric by resampling data from the mental state information by randomly sampling a specific number of times from the mental state information;

modifying the media presentation based on the metric;

evaluating a norm, on a computing device, based on the metric, wherein the evaluating the norm is based on resampling data from the mental state information from the plurality of people multiple times where the resampling includes random selection of data from the mental state information and where individuals from the plurality of people are selectively omitted from the resampling and where a synthetic trace is generated based on the resampling and the synthetic trace indicates emotional reactions for grouped subsets of people within the plurality of people and the synthetic trace is used in the evaluating of the norm; and

rendering an output of the norm on a display.

2. The method of claim 1 wherein the norm is determined for a certain demographic.

3. The method of claim 2 wherein the norm is determined for a country, a region, a socioeconomic group, a race, an ethnicity, a language group, a market, an age group, or a gender.

4. The method of claim 1 wherein the norm is determined for a product category, a media length, a media type, or a gaze direction.

5. The method of claim 1 wherein the norm includes a variance and where the variance of reaction to a certain media presentation is determined to be significantly different from a previous norm.

6. The method of claim 1 wherein the norm includes a mean value and a standard deviation value indicative of a mental state and variability on the mental state where the norm indicates a collective response by the plurality of people to a media presentation.

7. The method of claim 1 wherein the norm includes a propensity to smile, a propensity to concentrate, or a propensity to express.

8. The method of claim 1 wherein the norm is determined on a basis for a device.

9. The method of claim 1 wherein the norm represents an emotion norm for a group.

10. The method of claim 1 wherein the mental state data from the plurality of people is collected while the plurality of people are exposed to a media presentation.

11. The method of claim 10 wherein the norm is determined in response to a category of media presentations.

12. The method of claim 10 wherein the media presentation includes an advertisement.

13. The method of claim 12 further comprising comparing the metric for the advertisement to the norm for other advertisements.

14. The method of claim 1 further comprising evaluating a personal analytic measure for an individual where the personal analytic measure includes a norm for the individual.

15. The method of claim 1 wherein the mental state data includes facial data and the metric and the norm are a reflection of the facial data.

16. The method of claim 1 further comprising collecting additional mental state data and comparing the additional mental state data against the norm.

17. The method of claim 16 further comprising identifying a deviation by the additional mental state data from the norm.

18. The method of claim 1 wherein the metric is calculated based on statistics from a time series of the mental state information.

19. The method of claim 1 wherein the norm is calculated across a number of media presentations.

20. The method of claim 1 wherein the resampling of the data is used to generate a synthetic trace used in generating the norm.

21. The method of claim 1 further comprising inferring mental states based on the mental state data which was obtained wherein the mental states include one or more of sadness, stress, happiness, anger, frustration, confusion, disappointment, hesitation, cognitive overload, focusing, engagement, attention, boredom, exploration, confidence, trust, delight, disgust, skepticism, doubt, satisfaction, excitement, laughter, calmness, and curiosity.

22. The method of claim 1 further comprising determining context during which the mental state data is captured.

23. A computer program product embodied in a non-transitory computer readable medium for mental state analysis, the computer program product comprising code which causes one or more processors to perform operations of:

obtaining mental state data from a plurality of people by capturing images of the plurality of people and analyzing the images to determine the mental state data, wherein, for at least one person of the plurality of people, the capturing is performed while the at least one person views a media presentation;

analyzing the mental state data to produce mental state information for the plurality of people;

generating a metric based on the mental state information;

generating a statistical analysis for the metric by resampling data from the mental state information by randomly sampling a specific number of times from the mental state information;

modifying the media presentation based on the metric;

evaluating a norm, on a computing device, based on the metric, wherein the evaluating the norm is based on resampling data from the mental state information from the plurality of people multiple times where the resampling includes random selection of data from the mental state information and where individuals from the plurality of people are selectively omitted from the resampling and where a synthetic trace is generated based on the resampling and the synthetic trace indicates emotional reactions for grouped subsets of people within the plurality of people and the synthetic trace is used in the evaluating of the norm; and

rendering an output of the norm on a display.

24. A computer system for mental state analysis comprising:

a memory which stores instructions;

one or more processors attached to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to;

obtain mental state data from a plurality of people by capturing images of the plurality of people and analyzing the images to determine the mental state data, wherein, for at least one person of the plurality of people, the capturing is performed while the at least one person views a media presentation;

analyze the mental state data to produce mental state information for the plurality of people;

generate a metric based on the mental state information;

generate a statistical analysis for the metric by resampling data from the mental state information by randomly sampling a specific number of times from the mental state information;

modify the media presentation based on the metric;

evaluate a norm, on a computing device, based on the metric, wherein the evaluating the norm is based on resampling data from the mental state information from the plurality of people multiple times where the resampling includes random selection of data from the mental state information and where individuals from the plurality of people are selectively omitted from the resampling and where a synthetic trace is generated based on the resampling and the synthetic trace indicates emotional reactions for grouped subsets of people within the plurality of people and the synthetic trace is used in the evaluating of the norm; and

render an output of the norm on a display.

25. A computer-implemented method for mental state analysis comprising:

obtaining mental state data from a plurality of people by capturing images of the plurality of people and analyzing, using one or more processors, the images to determine the mental state data, wherein, for at least one person of the plurality of people, the capturing is performed while the at least one person views a media presentation;

analyzing the mental state data to produce mental state information for the plurality of people;

generating a metric based on the mental state information;

generating a statistical analysis for the metric by resampling data from the mental state information by randomly sampling a specific number of times from the mental state information;

evaluating a norm, on a computing device, based on the metric, wherein the evaluating the norm is based on resampling data from the mental state information from the plurality of people multiple times where the resampling includes random selection of data from the mental state information and where individuals from the plurality of people are selectively omitted from the resampling and where a synthetic trace is generated based on the resampling and the synthetic trace indicates emotional reactions for grouped subsets of people within the plurality of people and the synthetic trace is used in the evaluating of the norm;

modifying the media presentation based on the metric; and

displaying the modified media presentation.