Situation-dependent libraries of affective response

• US 9,230,220 B2
• Filed: 06/25/2011
• Issued: 01/05/2016
• Est. Priority Date: 05/11/2011
• Status: Active Grant

• Current Assignee: Affectomatics ltd.
• Original Assignee: Anat Thieberger Ben-Haim, Ari M. Frank, Gil Thieberger
• Inventors: Frank, Ari M., Thieberger, Gil, Thieberger Ben-Haim, Anat
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): FINK, THOMAS ANDREW
• Application Number:
  US13/168,962
• Publication Number:
  US 20120290512A1
• Time in Patent Office: 1,655 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

Generating a situation-dependent library comprising a user'"'"'s expected response to tokens representing stimuli that influence the user'"'"'s affective state, including: receiving samples comprising temporal windows of token instances to which the user was exposed, wherein the token instances have overlapping instantiation periods and are spread over a long period of time that spans different situations; wherein at least one token is expected to elicit from the user a noticeably different affective response in the different situations; receiving target values corresponding to the temporal windows of token instances; the target values represent the user'"'"'s responses to the token instances from the temporal windows of token instances; training a machine learning-based user response model using the samples and the corresponding target values; and analyzing the machine learning-based user response model to generate the situation-dependent library comprising the user'"'"'s expected response to tokens, which accounts for the variations in the user'"'"'s affective response in the different situations.

Claims

1. A system configured to generate a situation-dependent library, comprising:

at least one processor and at least one memory, the at least one processor and the at least one memory cooperating to function as;

a machine learning trainer configured to receive samples comprising temporal windows of token instances to which a user was exposed and situation identifiers corresponding to situations the user was in while being exposed to the temporal windows;

wherein the token instances have overlapping instantiation periods and are spread over a period of time that spans at least a day, and the situation identifiers comprise situation identifiers for first and second situations;

the machine learning trainer is further configured to receive target values corresponding to the temporal windows of token instances;

the target values represent affective responses of the user to the token instances from the temporal windows of token instances;

the machine learning trainer is further configured to train first and second situation-dependent machine learning-based user response models based on respective first and second datasets;

wherein the first and second datasets are obtained from partitioning the samples according to the situations the user was in when exposed to the temporal windows of token instances corresponding to the samples; and

wherein the first dataset comprises samples comprising situation identifiers for the first situation and corresponding target values, and the second dataset comprises samples comprising situation identifiers for the second situation and corresponding target values; and

a model analyzer configured to generate, based on the first and second situation-dependent machine learning-based user response models, the situation-dependent library which describes a first set of expected affective responses of the user to certain tokens while being in the first situation, and a second set of expected affective responses of the user to the certain tokens while being in the second situation;

wherein the first set is not identical to the second set, and wherein at least some of the token instances from the temporal windows of token instances are instantiations of the certain tokens.

2. The system of claim 1, wherein the target values are emotional response annotations, and the situation-dependent library is a situation-dependent library comprising expected emotional responses of the user to the tokens.

3. The system of claim 1, wherein the target values are derived from values of user measurement channels of the user, and the situation-dependent library expresses the expected responses to the tokens in terms of values of the user measurement channels.

4. The system of claim 1, wherein the tokens comprise representations of elements extracted from digital media content.

5. The system of claim 1, wherein the machine learning-based user response model comprises a regression model, and the situation-dependent library comprises a regression parameter vector.

6. The system of claim 1, wherein the machine learning trainer is further configured to train a Naive Bayes classifier, and the situation-dependent library comprises class prior and class conditional probabilities;

the class prior probabilities describe the prior probability for a temporal window of token instances to be labeled with a target value that is an emotional state category;

the class conditional probabilities describe the probability of token instances belonging to a temporal window of token instances that is labeled with a specific user response category.

7. The system of claim 1, wherein the machine learning trainer is further configured to train a maximum entropy model classifier, and the situation-dependent library comprises feature function weighting parameters.

8. The system of claim 1, wherein the machine learning trainer is further configured to train a neural network, and the situation-dependent library comprises partial derivatives of the neural network'"'"'s output values, with respect to the input neurons that correspond to token instances.

9. The system of claim 1, wherein the machine learning trainer is further configured to train a random forest model, and the situation-dependent library comprises importance parameters for variables of the random forest model.

10. The system of claim 2, further comprising a target value generator configured to predict the affective response annotations from values of a user measurement channel of the user.

11. The system of claim 10, wherein the target value generator is further configured to receive a baseline value derived from a value of a user measurement channel of the user, and to utilize the baseline value to predict the affective response annotations.

12. A system configured to generate a situation-dependent library, comprising:

at least one processor and at least one memory, the at least one processor and the at least one memory cooperating to function as;

a machine learning trainer configured to receive samples comprising temporal windows of token instances to which a user was exposed and situation identifiers corresponding to situations the user was in while being exposed to the temporal windows;

wherein the token instances have overlapping instantiation periods and are spread over a period of time that spans at least a day, and the situation identifiers comprise situation identifiers for first and second situations;

the machine learning trainer is further configured to receive target values corresponding to a proper subset of the temporal windows of token instances;

the target values represent affective responses of the user after being exposed to the token instances from the proper subset of the temporal windows of token instances;

the machine learning trainer is further configured to perform semi-supervised training of first and second situation-dependent machine learning-based user response models based on respective first and second datasets;

wherein the first and second datasets are obtained from partitioning the samples according to the situations the user was in when exposed to the temporal windows of token instances corresponding to the samples; and

wherein the first dataset comprises samples comprising situation identifiers for the first situation and corresponding target values, and the second dataset comprises samples comprising situation identifiers for the second situation and corresponding target values; and

a model analyzer configured to generate, based on the first and second situation-dependent machine learning-based user response models, the situation-dependent library which describes a first set of expected affective responses of the user to certain tokens while being in the first situation, and a second set of expected affective responses of the user to the certain tokens while being in the second situation;

wherein the first set is not identical to the second set, and wherein at least some of the token instances from the temporal windows of token instances are instantiations of the certain tokens.

13. The system of claim 12, wherein the token instances originate from different token sources, and are detected by the user using different sensory pathways;

and wherein the token instances comprise representations of elements extracted from digital media content.

14. The system of claim 12, wherein the target values are emotional response annotations, and the situation-dependent library is a situation-dependent library comprising expected emotional responses of the user to the tokens.

15. The system of claim 12, wherein the target values are derived from values of user measurement channels of the user, and the situation-dependent library expresses the expected responses to the tokens in terms of values of the user measurement channels.

16. The system of claim 13, further comprising a target value generator configured to predict the affective response annotations from values of a user measurement channel of the user.

17. The system of claim 16, wherein the target value generator is further configured to receive a baseline value derived from a value of a user measurement channel of the user, and to utilize the baseline value to predict the affective response annotations.

18. A system configured to generate a situation-dependent affective response library, comprising:

at least one processor and at least one memory, the at least one processor and the at least one memory cooperating to function as;

a machine learning trainer configured to receive samples comprising temporal windows of token instances to which a user was exposed and situation identifiers corresponding to situations the user was in while being exposed to the temporal windows;

wherein the token instances have overlapping instantiation periods and are spread over a period of time that spans at least a day, and the situation identifiers comprise situation identifiers for first and second situations;

the machine learning trainer is further configured to receive affective response annotations corresponding to the temporal windows of token instances;

wherein the annotations are expressed as user measurement channel values;

the machine learning trainer is further configured to train first and second situation-dependent machine learning-based user affective response models based on respective first and second datasets;

wherein the first and second datasets are obtained from partitioning the samples according to the situations the user was in when exposed to the temporal windows of token instances corresponding to the samples; and

wherein the first dataset comprises samples comprising situation identifiers for the first situation and corresponding target values, and the second dataset comprises samples comprising situation identifiers for the second situation and corresponding target values; and

a model analyzer configured to generate, based on the first and second situation-dependent machine learning-based user affective response models, the situation-dependent affective-response library, which describes a first set of expected affective responses of the user to certain tokens while being in the first situation, and a second set of expected affective responses of the user to the certain tokens while being in the second situation;

wherein the first set is not identical to the second set, and wherein at least some of the token instances from the temporal windows of token instances are instantiations of the certain tokens.

19. The system of claim 18, wherein the first and second situations are characterized by at least one of:

the user being exposed to different subsets of tokens in the first situation than in the second situation, a different emotional state baseline value of the user when the user in the first situation compared to when the user is in the second situation, and a different user measurement channel baseline value of the user when the user in the first situation compared to when the user is in the second situation.

20. The system of claim 18, wherein the first and second situations are characterized by at least one of:

the user being exposed to token instances instantiated from different subsets of tokens when the user is in the first situation compared to when the user is in the second situation.