SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT FOR USING EYE MOVEMENT TRACKING FOR RETRIEVAL OF OBSERVED INFORMATION AND OF RELATED SPECIFIC CONTEXT

• US 20140006463A1
• Filed: 04/23/2012
• Published: 01/02/2014
• Est. Priority Date: 04/23/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Jacob, Michal
• Granted Patent: US 9,483,480 B2
• Time in Patent Office: Days
• US Class Current: 707/825
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

Methods, systems, and computer program products that relate to recording content that a user has observed in a particular area of a display. This area may be referred to herein as a region of interest (ROI). The content observed by the user may be recorded, along with context information pertaining to the content observed. This context information may include a processing context. The recorded information may then be used subsequently to recall content that had been previously observed in the ROI. The user may enter search criteria such as information identifying the location of the ROI and/or the time at which the ROI had been observed. In alternative embodiments, additional or different criteria may be entered by the user, such as a key word, user action context, temporal context, and/or spatial context. The content that had been observed in the ROI may then be presented to the user. This content may be presented within the same processing context in which it had been previously observed, e.g., using the same application. This allows the user to get back to information that he had previously seen on screen, by referring to a particular area on the display, the time at which it had been observed, or other criteria.

Claims

1-27. -27. (canceled)

28. A method, comprising:

determining a region of interest (ROI) observed by a user on a display;

recording content observed in the region;

recording a processing context of the content;

storing the recorded information in a repository; and

making the recorded information accessible to the user upon request at a later time.

29. The method of claim 28, wherein the content comprises one or more of text information and image information.

30. The method of claim 28, wherein the processing context comprises one or more of:

an application observed in the region;

a location in the application;

a file open in the region;

a location in the file, where the location in the file is in the region;

a web page observed in the region; and

a location in the web page, where the location in the web page is in the region.

31. The method of claim 28, further comprising:

recording the time at which the region of interest is observed,

performed before said storing.

32. The method of claim 28, further comprising:

recording screen coordinates of the region of interest observed,

performed before said storing.

33. The method of claim 28, wherein said storing comprises:

creating a record; and

storing the recorded information in the record.

34. The method of claim 33, further comprising:

storing, in the record, links to one or more related records.

35. The method of claim 34, wherein the one or more related records comprise at least one of:

a record for a previously observed region of interest;

a record for a subsequently observed region of interest; and

a record for a region adjacent to the observed region of interest.

36. The method of claim 33, further comprising storing, in the record, one or more of:

identification of a user action taken prior to the observation; and

identification of a user action taken after the observation.

37. The method of claim 33, further comprising storing, in the record, the time duration through which the user visually dwelled on the region.

38. The method of claim 28, wherein said making the information accessible comprises:

receiving search criteria from the user;

searching the repository for recorded information that matches for the search criteria;

returning, to the user, one or more references to any recorded information that matches the search criteria;

receiving, from the user, a selection of one or more of the references; and

presenting, to the user, content corresponding to the selection in a recreated context associated with the corresponding content.

39. The method of claim 38, wherein the search criteria comprises one or more of:

a description of the content in the region of interest;

a reference to the time or range of time at which the region of interest was observed;

a reference to the location or area on the display at which the region of interest was observed;

a reference to a user action taken prior to the observation;

a reference to a user action taken after the observation;

a reference to a previously observed region of interest;

a reference to a subsequently observed region of interest; and

a reference to a region adjacent to the observed region of interest.

40. A system, comprising:

a processor; and

a memory in communication with said processor, said memory for storing a plurality of processing instructions configured to direct said processor to;

determine a region of interest (ROI) observed by a user on a display;

record content observed in the region;

record a processing context of the content;

store the recorded information in a repository; and

make the recorded information accessible to the user upon request at a later time.

41. The system of claim 40, wherein the content comprises one or more of text information and image information.

42. The system of claim 40, wherein the processing context comprises one or more of:

an application observed in the region;

a location in the application;

a file open in the region;

a location in the file, where the location in the file is in the region;

a web page observed in the region; and

a location in the web page, where the location in the web page is in the region.

43. The system of claim 40, wherein the plurality of processing instructions is further configured to direct said processor to:

record the time at which the region of interest is observed, performed before said storing.

44. The system of claim 40, wherein the plurality of processing instructions is further configured to direct said processor to:

record screen coordinates of the region of interest observed, performed before said storing.

45. The system of claim 40, wherein said storing comprises:

creating a record; and

storing the recorded information in the record.

46. The system of claim 45, wherein the plurality of processing instructions is further configured to direct said processor to:

store, in the record, links to one or more related records.

47. The system of claim 46, wherein the one or more related records comprise at least one of:

a record for a previously observed region of interest;

a record for a subsequently observed region of interest; and

a record for a region adjacent to the observed region of interest.

48. The system of claim 45, wherein the plurality of processing instructions configured to direct said processor to store, in the record, the recorded information, comprises a plurality of processing instructions configured to direct said processor to store one or more of:

identification of a user action taken prior to the observation; and

identification of a user action taken after the observation.

49. The system of claim 45, wherein the plurality of processing instructions configured to direct said processor to store, in the record, the recorded information, comprises a plurality of processing instructions configured to direct said processor to store:

the time duration through which the user visually dwelled on the region.

50. The system of claim 40, wherein the plurality of processing instructions configured to direct said processor to make the information accessible comprises a plurality of processing instructions configured to direct said processor to:

receive search criteria from the user;

search the repository for recorded information that matches for the search criteria;

return, to the user, one or more references to any recorded information that matches the search criteria;

receive, from the user, a selection of one or more of the references; and

present, to the user, content corresponding to the selection in a recreated context associated with the corresponding content.

51. The system of claim 50, wherein the search criteria comprises one or more of:

a description of the content in the region of interest;

a reference to the time or range of time at which the region of interest was observed;

a reference to the location or area on the display at which the region of interest was observed;

a reference to a user action taken prior to the observation;

a reference to a user action taken after the observation;

a reference to a previously observed region of interest;

a reference to a subsequently observed region of interest; and

a reference to a region adjacent to the observed region of interest.

52. The system of claim 40, further comprising a non-volatile memory configured to store said repository.

53. The system of claim 40, further comprising a connection to a network that is configured to provide remote access to a non-volatile memory configured to store said repository.

54. A computer program product comprising a non-transitory computer useable medium having control logic stored therein, the computer control logic comprising logic to cause a processor to:

determine a region of interest (ROI) observed by a user on a display;

record content observed in the region;

record a processing context of the content;

store the recorded information in a repository; and

make the recorded information accessible to the user upon request at a later time.

55. The computer program product of claim 54, wherein the content comprises one or more of text information and image information.

56. The computer program product of claim 54, wherein the processing context comprises one or more of :

an application observed in the region;

a location in the application;

a file open in the region;

a location in the file, where the location in the file is in the region;

a web page observed in the region; and

a location in the web page, where the location in the web page is in the region.

57. The computer program product of claim 54, the computer control logic further comprising logic to cause the processor to:

record the time at which the region of interest is observed,

performed before said storing.

58. The computer program product of claim 54, the computer control logic further comprising logic to cause the processor to:

record screen coordinates of the region of interest observed,

performed before said storing.

59. The computer program product of claim 54, wherein the storing comprises:

creating a record; and

storing the recorded information in the record.

60. The computer program product of claim 59, wherein the logic to cause the processor to store the recorded information comprises logic to cause the processor to:

store, in the record, links to one or more related records.

61. The computer program product of claim 60, wherein the one or more related records comprise at least one of:

a record for a previously observed region of interest;

a record for a subsequently observed region of interest; and

a record for a region adjacent to the observed region of interest.

62. The computer program product of claim 59, wherein the logic to cause the processor to store the recorded information comprises logic to cause the processor to store one or more of:

identification of a user action taken prior to the observation; and

identification of a user action taken after the observation.

63. The computer program product of claim 59, wherein the logic to cause the processor to store the recorded information comprises logic to cause the processor to store, in the record, the time duration through which the user visually dwelled on the region.

64. The computer program product of claim 54, wherein the logic to cause the processor to make the information accessible comprises logic to cause the processor to:

receive search criteria from the user;

search the repository for recorded information that matches for the search criteria;

return, to the user, one or more references to any recorded information that matches the search criteria;

receive, from the user, a selection of one or more of the references; and

present, to the user, content corresponding to the selection in a recreated context associated with the corresponding content.

65. The computer program product of claim 64, wherein the search criteria comprises one or more of:

a description of the content in the region of interest;

a reference to the time or range of time at which the region of interest was observed;

a reference to the location or area on the display at which the region of interest was observed;

a reference to a user action taken prior to the observation;

a reference to a user action taken after the observation;

a reference to a previously observed region of interest;

a reference to a subsequently observed region of interest; and

a reference to a region adjacent to the observed region of interest.