User modeling with salience

• US 9,946,795 B2
• Filed: 01/27/2014
• Issued: 04/17/2018
• Est. Priority Date: 01/27/2014
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Marvit, David L., Ubois, Jeffrey
• Primary Examiner(s): Bullock, Joshua
• Application Number:
  US14/165,347
• Publication Number:
  US 20150213020A1
• Time in Patent Office: 1,541 Days
• Field of Search: 707748, 707769
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

A method of using a user model with salience includes providing content to a user through a user interface. The method also includes receiving physiological data corresponding to a physiological response of the user as the user interacts with the content. The method may also include determining a salience score for the content based at least in part on the physiological response of the user, and updating a user model with the salience score.

Claims

1. A method of using a user model with salience, the method comprising:

providing a content item to a user through a user interface;

receiving physiological data corresponding to a physiological response of the user as the user interacts with the content item provided through the user interface;

determining a salience score for the content item based at least in part on the physiological response of the user;

updating a user model with the salience score for the content item;

determining a salience ranking for one or more keywords of the content item based on the salience score for the content item;

determining a relevance ranking for the one or more keywords of the content item based on the relevance of the one or more keywords to the content item; and

updating the user model to include a combined score for the one or more keywords of the content item based on the salience ranking and the relevance ranking;

wherein the salience ranking is at least partially based on the timing of determining the salience score for the content item.

2. The method according to claim 1, wherein the method further comprises receiving eye tracking data corresponding to a plurality of viewing angles of an eye of the user over time as the user interacts with the content item over time, wherein determining a salience score is based at least in part on the viewing angle data.

3. The method according to claim 1, wherein updating the user model with the salience score comprises associating the salience score with the content item within the user model.

4. The method according to claim 1, the method further comprising:

determining one or more keywords from the content item; and

associating the salience score with the one or more keywords within the user model.

5. The method according to claim 1, the method further comprising performing an action on a document based on the user model.

6. The method according to claim 1, the method further comprising recommending another content item to a user based on the user model.

7. The method according to claim 1, wherein the physiological data comprises brain activity data.

8. The method according to claim 1, the method further comprising:

maintaining, at a server, an aggregate user model database having a plurality of user models associated with a plurality of users, each of the plurality of user models associated with a user and associated with a content item consumed by the user with a salience score of the user when viewing the content item;

clustering, at the server, at least a first user model of the plurality of user models associated with a first user of the plurality of users and a second user model of the plurality of user models associated with a second user of the plurality of users based at least in part on the similarity of salience measures of a similar content item within the first user model and the second user model; and

determining, at the sever, a recommended content item for the first user based at least in part on the salience measures of the recommended content item of the second user as indicated in the second user model.

9. The method according to claim 8, wherein each of the salience measures are based at least in part on either or both of a physiological response of the user as the user views at least a portion of the content item and eye tracking data as the user views at least a portion of the content item.

10. The method according to claim 8, wherein the plurality of user models ranks a plurality of content items based on a plurality of salience scores.

11. The method according to claim 8, wherein the recommended content item is selected from a list consisting of an advertisement, a video, music, a document, and a webpage.

12. The method according to claim 8, further comprising sending the recommended content item to the first user from the server.

13. The method according to claim 8, further comprising highlighting a portion of a document related to the recommended content item based at least in part on the salience measures of the recommended content item of the second user as indicated in the second user model.

14. The method according to claim 8, further comprising highlighting a portion of a document related to the recommended content item based at least in part on keywords associated with the recommended content item.

15. The method according to claim 8, further comprising at least one of:

advancing to a specific portion of a video that is related to the recommended content item based at least in part on the salience measures of the recommended content item of the second user as indicated in the second user model;

advancing to a specific portion of a video that is related to the recommended content item based at least in part on keywords associated with the recommended content item; and

advancing to a specific portion of a video that is related to the recommended content item based at least in part on keywords associated with the recommended content item.

16. A system of associating salience with a content item, comprising:

a user interface configured to present the content item to a user;

a physiological sensor configured to record a physiological response of the user over time as the user interacts with the content item; and

a controller communicatively coupled with the user interface and the physiological sensor, the controller configured to;

provide the content item to the user interface for presentation to the user;

associate the physiological response of the user with the content item as the user engages with the content item;

determine a salience score for the content item based at least in part on the physiological response of the user;

update a user model with the salience score for the content item;

determine a salience ranking for one or more keywords of the content item based on the salience score for the content item;

determine a relevance ranking for the one or more keywords of the content item based on the relevance of the one or more keywords to the content item; and

update the user model to include a combined score for the one or more keywords of the content item based on the salience ranking and the relevance ranking;

wherein the salience ranking is at least partially based on the timing of determining the salience score for the content item.

17. The system according to claim 16, further comprising an eye tracking system configured to record eye tracking data that associates the viewing angle corresponding to a plurality of viewing angles of an eye of a user over time as the user interacts with the content item, and wherein the controller is configured to determine the salience score for the content item based at least in part on the eye tracking data.

18. The system according to claim 16, wherein the controller is further configured to:

determine one or more keywords from the content item; and

associate the salience score with the one or more keywords within the user model.

19. The system according to claim 16, wherein the controller is further configured to perform an action on a document based on the user model.

20. The system according to claim 16, wherein the controller is further configured to recommend another content item to a user based on the user model.