System, method, and computer program product for using eye movement tracking for retrieval of observed information and of related specific context

• US 9,483,480 B2
• Filed: 04/23/2012
• Issued: 11/01/2016
• Est. Priority Date: 04/23/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Jacob, Michal
• Primary Examiner(s): Nguyen, Loan T
• Application Number:
  US13/983,930
• Publication Number:
  US 20140006463A1
• Time in Patent Office: 1,653 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Methods and systems to identify a portion of content, for each of multiple instances of content presented at a display, based on eye contact of a user, populate records of the portions of content with contextual information, and search a data field(s) of the record to identify content of interest to the user amongst the portions of content. The search may be based on a, without limitation, a user-specified application program, file type, URL, time, display position, content presented at the display prior to, concurrently with, and/or subsequent to presentation of content of interest to the user, and/or an action performed by the user prior to and/or subsequent to a time at which the content of interest to the user had been presented at the display.

Claims

1. A method, comprising:

identifying a portion of content, for each of multiple instances of content presented at a display, based on based on eye fixations and saccadic eye movements of a user;

recording the portions of content in a data repository;

populating records of the portions of content with contextual information related to the respective portions of content;

searching one or more data fields of the records to identify content of interest to the user amongst the recorded portions of content, based a user-specified one or more of,

an application program that caused the content of interest to the user to be presented at the display,

a file type of the content of interest to the user,

a uniform resource locator (URL) associated with the content of interest to the user,

a time at which content of interest to the user had been presented at the display,

a position of the display at which the content of interest to the user had been presented at the display,

information related to content presented at the display prior to, concurrently with, and/or subsequent to presentation of the content of interest to the user, and

an indication of an action performed by the user prior to and/or subsequent to a time at which the content of interest to the user had been presented at the display; and

presenting one or more of the portions of content for which the respective data record satisfies the user query, at the display.

2. The method of claim 1, wherein:

the populating includes populating an application program field of a record with an indication of an application program that caused the respective portion of content to be present at the display; and

the searching includes searching the application program field of the record based on the user-specified application program that caused the content of interest to the user to be presented at the display.

3. The method of claim 1, wherein

the populating includes populating a file type field of a record with a file type of the respective portion of content;

and

the searching includes searching the file type field of the record based on the user-specified file type of the content of interest to the user.

4. The method of claim 1, wherein:

the populating includes populating a URL field of a record with a URL of the respective portion of content; and

the searching includes searching the URL field of the record based on the user-specified URL associated with the content of interest to the user.

5. The method of claim 1, wherein:

the populating includes populating a time field of a record with a time at which the respective portion of content had been presented at the display; and

the searching includes searching the time field of the record based on the user-specified time at which the content of interest to the user had been presented at the display.

6. The method of claim 1, wherein:

the populating includes populating a position field of a record with a position of the display at which the respective portion of content had been presented; and

the searching includes searching the position field of the record based on the user-specified position at which the content of interest to the user had been presented at the display.

7. The method of claim 1, wherein:

the populating includes populating a temporal context field of a record with information related to content presented at the display prior to and/or subsequent to presentation of the respective portion of content; and

the searching includes searching the temporal context field of the record based on the user-specified information related to content presented at the display prior to and/or subsequent to presentation of the content of interest to the user.

8. The method of claim 1, wherein:

the populating includes populating a spatial context field of a record with information related to content presented at the display concurrently with presentation of the respective portion of content; and

the searching includes searching the spatial context field of the record based on the user-specified information related to content presented at the display concurrently with presentation of the content of interest to the user.

9. The method of claim 1, wherein:

the populating includes populating a user action field of a record with an indication of an action performed by the user prior to and/or subsequent to a time at which the respective portion of content had been presented at the display; and

the searching includes searching the user action field of the record based on the user-specified indication of an action performed by the user prior to and/or subsequent to a time at which the content of interest to the user had been presented at the display.

10. The method of claim 1, further including:

performing the searching and the presenting while performing the identifying, the recording, and the populating.

11. A system, comprising, a processor and memory configured to:

identify a portion of content, for each of multiple instances of content presented at a display, based on based on eye fixations and saccadic eye movements of a user;

record the portions of content in a data repository;

populate records of the portions of content with contextual information related to the respective portions of content;

search one or more data fields of the records to identify content of interest to the user amongst the recorded portions of content, based a user-specified one or more of,

an application program that caused the content of interest to the user to be presented at the display,

a file type of the content of interest to the user,

a uniform resource locator (URL) associated with the content of interest to the user,

a time at which content of interest to the user had been presented at the display,

a position of the display at which the content of interest to the user had been presented at the display,

information related to content presented at the display prior to, concurrently with, and/or subsequent to presentation of the content of interest to the user, and

an indication of an action performed by the user prior to and/or subsequent to a time at which the content of interest to the user had been presented at the display; and

present one or more of the portions of content for which the respective data record satisfies the user query, at the display.

12. The system of claim 11, wherein the processor and memory are further configured to:

populate an application program field of a record with an indication of an application program that caused the respective portion of content to be presented at the display; and

search the application program field of the record based on the user-specified application program that caused the content of interest to the user to be presented at the display.

13. The system of claim 11, wherein the processor and memory are further configured:

populate a file type field of a record with a file type of the respective portion of content; and

search the file type field of the record based on the user-specified file type of the content of interest to the user.

14. The system of claim 11, wherein the processor and memory are further configured to:

populate a URL field of a record with a URL of the respective portion of content; and

search the URL field of the record based on the user-specified URL associated with the content of interest to the user.

15. The system of claim 11, wherein the processor and memory are further configured to:

populate a time field of a record with a time at which the respective portion of content had been presented at the display; and

search the time field of the record based on the user-specified time at which the content of interest to the user had been presented at the display.

16. The system of claim 11, wherein processor and memory are further configured to:

populate a position field of a record with a position of the display at which the respective portion of content had been presented; and

search the position field of the record based on the user-specified position at which the content of interest to the user had been presented at the display.

17. The system of claim 11, wherein the processor and memory are further configured to:

populate a temporal context field of a record with information related to content presented at the display prior to and/or subsequent to presentation of the respective portion of content; and

search the temporal context field of the record based on the user-specified information related to content presented at the display prior to and/or subsequent to presentation of the content of interest to the user.

18. The system of claim 11, wherein the processor and memory are further configured to:

populate a spatial context field of a record with information related to content presented at the display concurrently with presentation of the respective portion of content; and

search the spatial context field of the record based on the user-specified information related to content presented at the display concurrently with presentation of the content of interest to the user.

19. The system of claim 11, wherein the processor and memory are further configured to:

populate a user action field of a record with an indication of an action performed by the user prior to and/or subsequent to a time at which the respective portion of content had been presented at the display; and

search the user action field of the record based on the user-specified indication of an action performed by the user prior to and/or subsequent to a time at which the content of interest to the user had been presented at the display.

20. The system of claim 11, wherein the processor and memory are further configured to:

search the one or more fields of the records and present the one or more portion of content, while identifying, recording, and populating a record with respect to a portion of content presented at the display.

21. A non-transitory computer readable medium encoded with a computer program that includes instructions to cause a processor to:

identify a portion of content, for each of multiple instances of content presented at a display, based on based on eye fixations and saccadic eye movements of a user;

record the portions of content in a data repository;

populate records of the portions of content with contextual information related to the respective portions of content;

search one or more data fields of the records to identify content of interest to the user amongst the recorded portions of content, based a user-specified one or more of,

an application program that caused the content of interest to the user to be presented at the display,

a file type of the content of interest to the user,

a uniform resource locator (URL) associated with the content of interest to the user,

a time at which content of interest to the user had been presented at the display,

a position of the display at which the content of interest to the user had been presented at the display,

information related to content presented at the display prior to, concurrently with, and/or subsequent to presentation of the content of interest to the user, and

an indication of an action performed by the user prior to and/or subsequent to a time at which the content of interest to the user had been resented at the display; and

present one or more of the portions of content for which the respective data record satisfies the user query, at the display.

22. The non-transitory computer readable medium of claim 21, further including instructions to cause the processor to:

populate an application program field of a record with an indication of an application program that caused the respective portion of content to be presented at the display; and

search the application program field of the record based on the user-specified application program that caused the content of interest to the user to be presented at the display.

23. The non-transitory computer readable medium of claim 21, further including instructions to cause the processor to:

populate a file type field of a record with a file type of the respective portion of content; and

search the file type field of the record based on the user-specified file type of the content of interest to the user.

24. The non-transitory computer readable medium of claim 21, including instructions further including instructions to cause the processor to:

populate a URL field of a record with a URL of the respective portion of content; and

search the URL field of the record based on the user-specified URL associated with the content of interest to the user.

25. The non-transitory computer readable medium of claim 21, further including instructions to cause the processor to:

populate a time field of a record with a time at which the respective portion of content had been presented at the display; and

search the time field of the record based on the user-specified time at which the content of interest to the user had been presented at the display.

26. The non-transitory computer readable medium of claim 21, further including instructions to cause the processor to:

populate a position field of a record with a position of the display at which the respective portion of content had been presented; and

search the position field of the record based on the user-specified position at which the content of interest to the user had been presented at the display.

27. The non-transitory computer readable medium of claim 21, further including instructions to cause the processor to:

populate a temporal context field of a record with information related to content presented at the display prior to and/or subsequent to presentation of the respective portion of content; and

search the temporal context field of the record based on the user-specified information related to content presented at the display prior to and/or subsequent to presentation of the content of interest to the user.

28. The non-transitory computer readable medium of claim 21, further including instructions to cause the processor to:

populate a spatial context field of a record with information related to content presented at the display concurrently with presentation of the respective portion of content; and

search the spatial context field of the record based on the user-specified information related to content presented at the display concurrently with presentation of the content of interest to the user.

29. The non-transitory computer readable medium of claim 21, further including instructions to cause the processor to:

populate a user action field of a record with an indication of an action performed by the user prior to and/or subsequent to a time at which the respective portion of content had been presented at the display; and

search the user action field of the record based on the user-specified indication of an action performed by the user prior to and/or subsequent to a time at which the content of interest to the user had been presented at the display.

30. The non-transitory computer readable medium of claim 21, further including instructions to cause the processor to:

search the one or more fields of the records and present the one or more portion of content, while identifying, recording, and populating a record with respect to a portion of content presented at the display.