Training a predictor of emotional response based on explicit voting on content and eye tracking to verify attention

• US 9,477,993 B2
• Filed: 10/13/2013
• Issued: 10/25/2016
• Est. Priority Date: 10/14/2012
• Status: Active Grant

• Current Assignee: IP3 2017, Series 200 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: Ari M. Frank, Gil Thieberger
• Inventors: Frank, Ari M, Thieberger, Gil
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): OLUDE AFOLABI, OLATOYOSI
• Application Number:
  US14/052,759
• Publication Number:
  US 20140108309A1
• Time in Patent Office: 1,108 Days
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G07C 13/00 Voting apparatus
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45

Abstract

Utilizing eye tracking to collect naturally expressed affective responses for training an emotional response predictor, comprising: receiving a vote of a user on a segment of content consumed by the user; receiving eye tracking data of the user taken while the user consumed the segment of content; determining, based on the eye tracking data, that a gaze-based attention level to the segment reaches a predetermined threshold; utilizing the vote to generate a label related to an emotional response to the segment; receiving an affective response measurement of the user taken substantially while the user consumed the segment of content; and training a measurement emotional response predictor with the label and the affective response measurement.

Claims

1. A system configured to utilize eye tracking for collecting a naturally expressed affective response for training an emotional response predictor, comprising:

a memory storing computer executable modules; and

a processor configured to execute the computer executable modules;

the computer executable modules comprising;

a label generator configured to receive first and second votes of a user on first and second segments of content consumed by the user, respectively;

the label generator is further configured to utilize the first and second votes to generate first and second labels related to first and second emotional responses to the first and second segments, respectively;

wherein the first and second votes are generated via a voting mechanism that comprises one or more of the following;

a like voting mechanism, a dislike voting mechanism, a star rating mechanism, a numerical rating mechanism, an up voting mechanism, a down voting mechanism, and a ranking mechanism;

a gaze analyzer configured to receive first and second eye tracking data of the user acquired while the user consumed the first and second segments, and to make a first determination that a first gaze-based attention level to the first segment reaches a first predetermined threshold, and to make a second determination that a second gaze-based attention level to the second segment does not reach a second predetermined threshold; and

a sample generator configured to utilize the first and second determinations to assign, for purpose of training a measurement emotional response predictor, a higher weight to a first sample comprising the first label and a first affective response measurement of the user, than to a second sample comprising the second label and a second affective response measurement of the user;

wherein the first measurement was taken by a sensor while the user consumed the first segment, and the second measurement was taken by the sensor while the user consumed the second segment.

2. The system of claim 1, wherein the gaze analyzer is further configured to receive third eye tracking data of the user taken while the user consumed a third segment of content, and to make a third determination that a third gaze-based attention level to the third segment reaches a third predetermined threshold;

the label generator does not receive a third vote of the user on the third segment, and the sample generator is further configured to assign, for the purpose of training a measurement emotional response predictor, a higher weight to the second sample, than to a third sample comprising an affective response measurement of the user taken by the sensor while the user consumed the third segment.

3. The system of claim 1, further comprising a training module configured to train the measurement emotional response predictor with the first sample, and not to train the measurement emotional response predictor with the second sample.

4. The system of claim 1, wherein the first and second segments consumed by the user on a social network, and the voting mechanism belongs to the social network.

5. The system of claim 4, wherein the first and second affective response measurements are taken during periods that end in temporal vicinity of when the first and second votes are made, respectively.

6. The system of claim 1, wherein the sample generator is further configured to select durations, during which the first and second affective response measurements are to be taken, in proportion to the first and second gaze-based attention levels, such that the duration during which the first affective response measurement is to be taken is longer than the duration during which the second affective response measurement is to be taken.

7. The system of claim 1, wherein the gaze analyzer is further configured to estimate gaze stability from the first eye tracking data, and the sample generator is further configured to select duration during which the first affective response measurement is to be taken as a function of the gaze stability.

8. The system of claim 1, wherein the gaze analyzer is further configured to estimate pupil diameter from the first eye tracking data, and the sample generator is further configured to select duration during which the first affective response measurement is to be taken as function of the pupil diameter.

9. The system of claim 1, wherein the label generator is further configured to receive data derived from an analysis of the content and to utilize the data to generate the label.

10. A method for utilizing eye tracking for collecting a naturally expressed affective response for training an emotional response predictor, comprising:

receiving first and second votes of a user on first and second segments of content consumed by the user;

wherein the first and second votes are generated via a voting mechanism that comprises one or more of the following;

a like voting mechanism, a dislike voting mechanism, a star rating mechanism, a numerical rating mechanism, an up voting mechanism, a down voting mechanism, and a ranking mechanism;

utilizing the first and second votes to generate first and second labels related to first and second emotional responses to the first and second segments, respectively;

receiving first and second eye tracking data of the user acquired while the user consumed the first and second segments, respectively;

making a first determination, based on the first eye tracking data, that a first gaze-based attention level to the first segment reaches a first predetermined threshold;

making a second determination, based on the second eye tracking data, that a second gaze-based attention level to the second segment does not reach a second predetermined threshold;

utilizing the first and second determinations to assign, for purpose of training a measurement emotional response predictor, a higher weight to a first sample comprising the first label and a first affective response measurement of the user, than a second sample comprising the second label and a second affective response measurement of the user;

wherein the first measurement was taken by a sensor while the user consumed the first segment, and the second measurement was taken by the sensor while the user consumed the second segment.

11. The method of claim 10, further comprising receiving third eye tracking data of the user taken while the user consumed a third segment, and making a third determination that a third gaze-based attention level to the third segment reaches a third predetermined threshold;

not receiving a third vote of the user on the third segment, and assigning, for the purpose of training a measurement emotional response predictor, a higher weight to the second sample, than to a third sample comprising a third affective response measurement of the user taken by the sensor while the user consumed the third segment for training.

12. The method of claim 10, further comprising training a measurement emotional response predictor with the first sample, and not training the measurement emotional response predictor with the second sample.

13. The method of claim 10, further comprising selecting durations during which the first and second affective response measurements are to be taken in proportion to the first and second gaze-based attention levels, such that the duration during which the first affective response measurement is to be taken is longer than the duration during which the second affective response measurement.

14. The method of claim 10, further comprising estimating gaze stability from the first eye tracking data, and selecting duration during which the first affective response measurement is to be taken as function of the gaze stability.

15. The method of claim 10, further comprising estimating pupil diameter from the first eye tracking data, and selecting duration during which the first affective response measurement is to be taken as function of the pupil diameter.

16. The method of claim 10, wherein the first and second segments are consumed by the user on a social network, the first and second votes are generated via a voting mechanism belonging to the social network, and the voting mechanism is offered independently of whether the vote is used to acquire the sensor to take affective response measurements.

17. The method of claim 16, further comprising selecting durations during which the first and second affective response measurements are to be taken, to be in respective temporal vicinities to when the first and second votes are made;

whereby temporal vicinity to a vote is no longer than 30 seconds before and 10 seconds after the vote is made; and

wherein the voting mechanism comprises one or more of the following;

a like voting mechanism, a dislike voting mechanism, a star rating mechanism, a numerical rating mechanism, an up voting mechanism, a down voting mechanism, and a ranking mechanism.

18. A system configured to utilize eye tracking to train an emotional response predictor, comprising:

a memory storing computer executable modules; and

a processor configured to execute the computer executable modules;

the computer executable modules comprising;

a label generator configured to receive a vote of a user on a segment of content consumed by the user on a social network;

wherein the vote is generated via a voting mechanism belonging to the social network; and

wherein the voting mechanism comprises one or more of the following;

a like voting mechanism, a dislike voting mechanism, a star rating mechanism, a numerical rating mechanism, an up voting mechanism, a down voting mechanism, and a ranking mechanism;

the label generator is further configured to utilize the vote to generate a label related to an emotional response to the segment;

a gaze analyzer configured to receive eye tracking data of the user taken while the user consumed the segment, and to determine, based on the eye tracking data, whether a gaze-based attention level to the segment reaches a predetermined threshold, and to indicate thereof to a training module; and

the training module is configured to receive the label and an affective response measurement of the user taken while the user consumed the segment, and to train a measurement emotional response predictor with the measurement and the label;

wherein the measurement is taken by a sensor coupled to the user.

19. The system of claim 18, wherein the label generator is further configured to receive, via the voting mechanism, a second vote of the user on a second segment of content consumed by the user on the social network;

the label generator is further configured to utilize the second vote to generate a second label related to an emotional response to the second segment;

the gaze analyzer is further configured to receive second eye tracking data of the user taken while the user consumed the second segment, and to determine, based on the second eye tracking data, that a gaze-based attention level to the second segment does not reach a second predetermined threshold, and to indicate thereof to the training module; and

wherein the training module is further configured not to utilize a second affective response measurement of the user taken by the sensor while the user consumed the second segment, to train the measurement emotional response predictor with the second measurement and the second label.

20. The system of claim 18, wherein the gaze analyzer is further configured to estimate from the eye tracking data at least one of the following values:

(i) a value of gaze stability, and (ii) a value of a pupil diameter.