Eye event detection for electronic documents

• US 9,817,477 B1
• Filed: 12/31/2015
• Issued: 11/14/2017
• Est. Priority Date: 03/11/2013
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Taylor, Dennis James, Wright, Joshua Daniel, Reinitz, Jed Baraban, Bennett, Tommy Lee
• Primary Examiner(s): Hausmann, Michelle
• Application Number:
  US14/986,405
• Time in Patent Office: 684 Days
• Field of Search: None
• CPC Class Codes: G06F 15/0291 for reading, e.g. e-books constructional details of portable computers G06F1/1613
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

Techniques and systems described herein locate possibly-significant portions of electronic documents. In some examples, gaze movements, e.g., of users'"'"' eyes, are tracked while the users read a formatted version of an electronic document. The gaze movement may be analyzed to determine if a reading interruption occurs (e.g., a reading pause or irregularity in a regular reading rate for the user). The reading interruption may occur when the user, while reading, encounters or notices a problem or point of interest in the text. A reading location associated with the reading interruption may be determined in text that is currently displayed. The reading location may be mapped to a location in a standard version of the document. The location may be reported to a centralized entity where statistical analysis can be performed to determine if there is a problem in the document.

Claims

1. A system comprising:

a processor;

a display;

an eye sensor; and

computer-readable storage media storing a reading model and computer-executable instructions that, when executed by the processor, cause the processor to;

present a formatted version of a document on the display;

receive, from the eye sensor, information related to a gaze direction;

determine, based on an analysis of the information related to the gaze direction, first data of a reading location in the formatted version of the document;

determine, based on an analysis of the information related to the gaze direction, a reading rate associated with the reading location;

determine that the reading rate is less than a regular reading rate indicated by the reading model; and

determine second data of a standard location that corresponds to the reading location, the standard location being within a standard version of the document, wherein the first data of the reading location is different from the second data of the standard location.

2. The system as recited in claim 1, wherein the computer-executable instructions further cause the processor to determine the regular reading rate based at least in part on tracking the gaze direction of a user as the user reads at least a second document.

3. The system as recited in claim 1, wherein the computer-executable instructions further cause the processor to send information associated with the standard location to a network entity.

4. The system as recited in claim 3, wherein the computer-executable instructions further cause the processor to:

receive, from the network entity, in response to the information associated with the standard location, a recommendation comprising at least one of an advertisement, an offer, or a coupon; and

present the recommendation on the display.

5. The system as recited in claim 1, wherein the eye sensor comprises a front-facing camera.

6. The system as recited in claim 1, wherein the gaze direction corresponds to a gaze direction of eyes of a user and the reading model is associated with the user.

7. A method, comprising:

presenting text of a document on a display screen of a computing device, the text being presented based at least in part on a formatted version of the document;

tracking gaze movement;

detecting an interruption to the gaze movement;

determining that the interruption to the gaze movement occurs in association with a display screen location that contains a portion of the text, wherein the display screen location is represented by first data; and

determine, within a standard version of the document, a standard location corresponding to the display screen location, the standard location represented by second data that is different from the first data.

8. The method as recited in claim 7, further comprising sending, via a network interface, to one or more network-accessible devices, information associated with the standard location.

9. The method as recited in claim 8, further comprising:

parsing the portion of the text to determine a user interest of a user;

sending the user interest in association with the information associated with the standard location;

receiving, at the computing device, a recommendation based at least in part on the user interest; and

presenting at least part of the recommendation on the display screen.

10. The method as recited in claim 9, wherein the recommendation comprises at least one of an advertisement, an offer, or a coupon.

11. The method as recited in claim 7, wherein the gaze movement comprises a rate of change in gaze direction in relation to the text of the document, and the method further comprises:

accessing reading information that defines a regular rate of change in gaze direction;

comparing the rate of change in gaze direction to the regular rate of change in gaze direction; and

detecting the interruption to the gaze movement based at least in part on a determination that the rate of change in gaze direction is a threshold amount less than the regular rate of change in gaze direction.

12. The method as recited in claim 7, further comprising presenting the text of the document based at least in part on at least a text size, line spacing, text spacing, display screen size, or type of font of the formatted version.

13. The method as recited in claim 7, further comprising determining the standard location based at least in part on the display screen location and a setting of the computing device used in presenting the text, the setting corresponding to at least a text size, line spacing, text spacing, display screen size, or type of font.

14. A computer-implemented method comprising:

receiving, at a network-accessible computing device, information on a plurality of reading events, wherein at least one reading event (i) is associated with a user, (ii) comprises first data associated with a reading location in a formatted version of a document, and (iii) is indicative of an interruption to eye gaze movement as the user reads text of the formatted version of the document;

determining, at the network-accessible computing device, based at least in part on the information, second data representing one or more standard locations within a standard version of the document, individual ones of the one or more standard locations corresponding to respective ones of the plurality of reading events, wherein the first data of the at least one reading event is different from the second data of a corresponding one of the one or more standard locations;

determining, at the network-accessible computing device, based at least in part on the information, a first standard location of the one or more standard locations, the first standard location associated with a threshold number of the reading events; and

communicating, by the network-accessible computing device, to one or more network-accessible devices, information of the first standard location.

15. The method as recited in claim 14, further comprising, at the network-accessible computing device:

accessing, in the document, information of an author of the document; and

communicating the information of the first standard location to the author based at least in part on the information of the author of the document.

16. The method as recited in claim 14, wherein the at least one reading event indicates that a rate of change in eye gaze direction of the respective user is a threshold amount less than a regular rate of change in eye gaze direction of the respective user.

17. The method as recited in claim 16, wherein the regular rate of change in the eye gaze direction is determined based on tracking other eye gaze movement while the respective user reads at least one other document.

18. The method as recited in claim 14, wherein a plurality of the reading events are associated with respective users of a plurality of users.

19. The method as recited in claim 18, wherein the determining the first standard location comprises determining that the first standard location is associated with at least a selected percentage or number of the users of the plurality of users.

20. The method as recited in claim 18, wherein the determining the first standard location comprises correlating ones of the reading events from multiple, different ones of the plurality of users.