DOCUMENT SEARCHING USING SALIENCE

• US 20150213012A1
• Filed: 01/27/2014
• Published: 07/30/2015
• Est. Priority Date: 01/27/2014
• Status: Abandoned Application

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: MARVIT, David L., UBOIS, Jeffrey
• Application Number:
  US14/165,353
• Publication Number:
  US 20150213012A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A method of associating salience with content includes displaying a document having a plurality of content. The method also includes receiving eye tracking data corresponding to a plurality of viewing angles of an eye of a user over time as the user views at least a portion of the plurality of the content within the document on the display over time, and receiving physiological data corresponding to a physiological response of the user as the user views at least a portion of the plurality of the content within the document on the display over time. The method may also include associating at least a portion of the viewing angle data with a location of at least one of the plurality of content within the document, and associating the physiological response of the user with one or more of the plurality of content of the document using the viewing angle data.

Claims

1. A system of associating salience with content, comprising:

a display device for displaying a document having a plurality of content items;

an eye tracking subsystem configured to record viewing angle data corresponding to a plurality of viewing angles of an eye of a user over time as the user views at least a portion of the plurality of the content items within the document on the display;

a physiological sensor subsystem configured to record a physiological response of the user over time as the user views at least a portion of the plurality of the content items within the document on the display; and

a controller coupled with the display device, the eye tracking subsystem, and the physiological sensor subsystem, the controller configured to;

provide the document to the display device for displaying to the user;

associate at least a portion of the viewing angle data with a location of at least one of the plurality of content items within the document; and

associate the physiological response of the user with one or more of the plurality of content items of the document using the viewing angle data.

2. The system according to claim 1, wherein the controller is further configured to determine a salience score for at least one of the plurality of content items based on the viewing angle data and the physiological response of the user.

3. The system according to claim 2, wherein the controller is further configured to generate metadata for the document that includes the salience score.

4. The system according to claim 2, wherein the controller is further configured to generate metadata for the document that associates a salience score with each of the plurality of content items in the document.

5. The system according to claim 2, wherein the controller is further configured to cluster the document relative to a plurality of documents based at least in part on the salience score.

6. The system according to claim 2, wherein the controller is further configured to filter the document relative to a plurality of other documents based at least in part on the salience score and the salience scores of the other documents.

7. The system according to claim 2, wherein the controller is further configured to:

determine one or more keywords associated with each of a subset of the plurality of content items; and

weight the one or more keywords with the salience score.

8. The system according to claim 1, wherein the eye tracking subsystem comprises one or more imager and one or more illumination subsystems.

9. The system according to claim 1, wherein the controller is further configured to:

determine a salience score for at least one of the plurality of content items based on the viewing angle data and the physiological response of the user;

highlight one or more content items in the document based on the salience score; and

provide the document to the display device for displaying to the user with one or more content items highlighted.

10. A method of associating salience with content, the method comprising:

displaying a document having a plurality of content items on a display;

receiving eye tracking data corresponding to a plurality of viewing angles of an eye of a user over time as the user views at least a portion of the plurality of the content items within the document on the display over time;

receiving physiological data corresponding to a physiological response of the user as the user views at least a portion of the plurality of the content items within the document on the display over time;

associating at least a portion of the viewing angle data with a location of at least one of the plurality of content items within the document; and

associating the physiological response of the user with one or more of the plurality of content items of the document using the viewing angle data.

11. The method according to claim 10, further comprising determining a salience score for at least one of the plurality of content items based on the viewing angle data and the physiological response of the user.

12. The method according to claim 11, further comprising generating metadata for the document that associates a salience score with each of the plurality of content items in the document.

13. The method according to claim 10, further comprising:

determining one or more keywords associated with each of a subset of the plurality of content items; and

weighting the one or more keywords with the salience score.

14. A non-transitory computer-readable medium having encoded therein programming code executable by a processor to perform operations comprising:

displaying a document having a plurality of content items on a display;

receiving eye tracking data corresponding to a plurality of viewing angles of an eye of a user over time as the user views at least a portion of the plurality of the content items within the document on the display over time;

receiving physiological data corresponding to a physiological response of the user as the user views at least a portion of the plurality of the content items within the document on the display over time;

associating at least a portion of the viewing angle data with a location of at least one of the plurality of content items within the document; and

associating the physiological response of the user with one or more of the plurality of content items of the document using the viewing angle data.

15. The non-transitory computer-readable medium according to claim 14, wherein the operations further comprise determining a salience score for at least one of the plurality of content items based on the viewing angle data and the physiological response of the user.

16. The non-transitory computer-readable medium according to claim 14, wherein the operations further comprise generating metadata for the document that associates a salience score with each of the plurality of content items in the document.

17. The non-transitory computer-readable medium according to claim 14, wherein the operations further comprise ranking the document relative to a plurality of documents based at least in part on the salience score.

18. The non-transitory computer-readable medium according to claim 14, wherein the operations further comprise:

determining one or more keywords associated with each of a subset of the plurality of content items; and

weighting the one or more keywords with the salience score.