Methods for creating a situation dependent library of affective response

• US 20120290512A1
• Filed: 06/25/2011
• Published: 11/15/2012
• Est. Priority Date: 05/11/2011
• Status: Active Grant

• Current Assignee: Affectomatics ltd.
• Original Assignee: Affectivon Limited
• Inventors: Frank, Ari M., Thieberger, Gil, Ben-Haim, Anat Thieberger
• Granted Patent: US 9,230,220 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

Generating a situation-dependent library comprising a user'"'"'s expected response to tokens representing stimuli that influence the user'"'"'s affective state, including: receiving samples comprising temporal windows of token instances to which the user was exposed, wherein the token instances have overlapping instantiation periods and are spread over a long period of time that spans different situations; wherein at least one token is expected to elicit from the user a noticeably different affective response in the different situations; receiving target values corresponding to the temporal windows of token instances; the target values represent the user'"'"'s responses to the token instances from the temporal windows of token instances; training a machine learning-based user response model using the samples and the corresponding target values; and analyzing the machine learning-based user response model to generate the situation-dependent library comprising the user'"'"'s expected response to tokens, which accounts for the variations in the user'"'"'s affective response in the different situations.

Claims

1. A method for generating a situation-dependent library comprising a user'"'"'s expected response to tokens representing stimuli that influence the user'"'"'s affective state, the method comprising:

receiving samples comprising temporal windows of token instances to which the user was exposed, wherein the token instances have overlapping instantiation periods and are spread over a long period of time that spans different situations;

wherein at least one token is expected to elicit from the user a noticeably different affective response in the different situations;

receiving target values corresponding to the temporal windows of token instances;

the target values represent the user'"'"'s responses to the token instances from the temporal windows of token instances;

training a machine learning-based user response model using the samples and the corresponding target values; and

analyzing the machine learning-based user response model to generate the situation-dependent library comprising the user'"'"'s expected response to tokens, which accounts for the variations in the user'"'"'s affective response in the different situations.

2. The method of claim 1, wherein the target values are affective response annotations, and the situation-dependent library is a situation-dependent library comprising the user'"'"'s expected affective response to tokens.

3. The method of claim 1, wherein the target values are derived from user measurement channels of the user, and the situation-dependent library expresses the expected response to tokens in terms of the user measurement channels'"'"' values.

4. The method of claim 1, wherein the tokens comprise representations of elements extracted from digital media content.

5. The method of claim 1, wherein the situation-dependent library is generated by analyzing at least two different machine learning-based user response models that were trained on data collected over periods during which the user was in different situations;

and the situation-dependent library comprises the user'"'"'s expected responses to essentially the same tokens in the different situations.

6. The method of claim 1, wherein the different situations are characterized by the user being exposed to different subsets of tokens.

7. The method of claim 1, wherein different situations are characterized by a noticeably different user emotional state baseline value.

8. The method of claim 1, wherein different situations are characterized by a noticeably different user measurement channel baseline value.

9. The method of claim 1, further comprising clustering of the samples, wherein samples falling into different clusters belong to different situations.

10. The method of claim 1, wherein the step of training the machine learning-based user response model comprises training a regression model;

and

the situation-dependent library comprises a regression parameter vector.

11. The method of claim 1, wherein the step of training the machine learning-based user response model comprises training a Naive Bayes classifier;

and the situation-dependent library comprises class prior and class conditional probabilities;

the class prior probabilities describe the prior probability for a temporal window of token instances to be labeled with a target value that is an emotional state category;

the class conditional probabilities describe the probability of token instances belonging to temporal window of token instances that is labeled with a specific user response category.

12. The method of claim 1, wherein the step of training the machine learning-based user response model comprises training a maximum entropy model classifier;

and the situation-dependent library comprises feature function weights.

13. The method of claim 1, wherein the step of training the machine learning-based user response model comprises training a neural network;

and the situation-dependent library comprises partial derivatives of the neural networks output values, with respect to the input neurons that correspond to token instances.

14. The method of claim 1, wherein the step of training the machine learning-based user response model comprises training a random forest model;

and the situation-dependent library comprises importance parameters for features.

15. A method for generating a situation-dependent library comprising a user'"'"'s expected response when there are significantly more samples than target values available, the method comprising:

receiving samples comprising temporal windows of token instances to which the user was exposed, wherein the token instances have overlapping instantiation periods and are spread over a long period of time that spans different situations;

wherein at least one token is expected to elicit from the user a noticeably different response in the different situations;

receiving intermittent target values corresponding to a subset of the temporal windows of token instances;

the target values represent the user'"'"'s response after being exposed to the token instances from the subset of the temporal windows of token instances;

training a semi-supervised machine learning-based user response model on the samples and the intermittent corresponding target values to account for the variations in the user'"'"'s response in the different situations;

analyzing the machine learning-based user response model to generate the situation-dependent library comprising the user'"'"'s expected response to tokens, which accounts for the variations in the user'"'"'s response in the different situations;

the situation-dependent library is more accurate than a situation-dependent library generated when the machine learning training uses only the samples that have corresponding target values, since the training procedure of the machine learning-based user response model is able to leverage additional information from the samples comprising temporal windows of token instances without corresponding target values.

16. The method of claim 15, wherein the token instances originate from different token sources, and are detected by the user using essentially different sensory pathways;

and the machine learning-based predictor accommodates for the variety of attributes and characteristic values typical of the different token sources.

19. The method of claim 15, wherein the different situations are characterized by the user being exposed to different subsets of tokens.

20. The method of claim 15, wherein the situation-dependent library is generated by analyzing at least two different machine learning-based user response models that were trained on data collected over periods during which the user was in different situations;

and the situation-dependent library comprises the user'"'"'s expected responses to essentially the same tokens in the different situations.

17. A method for generating an affective-response library comprising a user'"'"'s expected affective response to tokens representing stimuli that influence the user'"'"'s affective state, the method comprising:

receiving samples comprising temporal windows of token instances to which the user was exposed, wherein the token instances have overlapping instantiation periods and are spread over a long period of time that spans different situations;

wherein at least one token is expected to elicit from the user a noticeably different affective response in the different situations;

receiving affective response annotations corresponding to the temporal windows of token instances;

training a machine learning-based user response model using the samples and the corresponding affective response annotations; and

analyzing the machine learning-based user response model to generate the affective response library comprising the user'"'"'s expected response to tokens.

18. The method of claim 17, wherein the affective-response library is generated by analyzing at least two different machine learning-based user response models that were trained on data collected over periods during which the user was in different situations;

and the affective-response library comprises the user'"'"'s expected responses to essentially the same tokens in the different situations.