Method and system for the recognition of reading skimming and scanning from eye-gaze patterns

• US 6,873,314 B1
• Filed: 08/29/2000
• Issued: 03/29/2005
• Est. Priority Date: 08/29/2000
• Status: Expired due to Term

• Current Assignee: Tobii Technology AB
• Original Assignee: International Business Machines Corporation
• Inventors: Campbell, Christopher S.
• Primary Examiner(s): NGUYEN, CHANH DUY
• Application Number:
  US09/649,608
• Time in Patent Office: 1,673 Days
• Field of Search: 345/156, 345/157, 345/158, 345/7, 345/8, 351/210, 702/151
• US Class Current: 345/156
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 5/4863 Measuring or inducing nystagmus
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06Q 30/0267 Wireless devices
• CPC Class Codes: G06Q 30/0273 Determination of fees for advertising

Abstract

Accurately recognizing from eye-gaze patterns when a user is reading, skimming, or scanning on a display filled with heterogeneous content, and then supplying information tailored to meet individual needs. Heterogeneous content includes objects normally encountered on computer monitors, such as text, images, hyperlinks, windows, icons, and menus. Three distinct mechanisms are used: (1) coarse or quantized representation of eye-movements, (2) accumulation of pooled numerical evidence based detection, and (3) mode switching. Analysis of text the user is reading or skimming may infer user interest and adapt to the user'"'"'s needs.

Claims

1. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, said method comprising the steps of:

quantizing eye movements of a user viewing heterogeneous content in both X and Y axes;

accumulating a numerical evidence of reading until a predetermined threshold is reached, said numerical evidence independent of gaze time and factoring both positive and negative values; and

detecting reading when said numerical evidence of reading exceeds said threshold.

2. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 1, wherein said quantizing step includes averaging over a predetermined period of time.

3. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 1, wherein said heterogeneous content includes at least one of the following:

text, images, hyperlinks, windows, icons, and menus.

4. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 1, comprising the further step of switching modes from a skimming mode and a scanning mode to a reading mode when said reading is detected.

5. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 4, comprising the further step of adapting to said user'"'"'s need if said reading is detected.

6. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 5, wherein said adapting step comprises the further steps of:

recording said heterogeneous content that is of interest to said user;

finding relevant information from a database using said recorded heterogeneous content; and

utilizing said found relevant information to adapt to a user'"'"'s needs.

7. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 6, wherein said utilizing step comprises the further step of developing accurate models of said users.

8. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 6, wherein said utilizing step comprises the further step of determining fine-grained information regarding said user'"'"'s interest in Internet advertising.

9. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 6, wherein said utilizing step comprises the further step of providing more accurate help regarding computer applications.

10. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 6, wherein said database includes at least one of the following:

a local machine database, a local area network database, and a wide area network database.

11. A method for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 10, wherein said wide area network database is the world wide web.

12. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, said system comprising:

an eye-movement quantizer that quantizes eye movements of a user viewing heterogeneous content in both X and Y axes;

a reading-evidence accumulator that accumulates a numerical evidence of reading, said numerical evidence independent of gaze time and factoring both positive and negative values;

a threshold-comparator that compares said numerical evidence of reading against a predetermined threshold; and

a reading-detector that detects reading when said numerical evidence of reading exceeds said predetermined threshold.

13. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 12, wherein said eye-movement quantizer averages over a predetermined period of time.

14. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 12, wherein said heterogeneous content includes at least one of the following:

text, images, hyperlinks, windows, icons, and menus.

15. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 12, further comprising a mode-switcher that switches modes from a skimming mode and a scanning mode to a reading mode when said reading is detected.

16. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 15, wherein said mode-switcher generates a user adaptation command signal when switching to said reading mode.

17. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 16, wherein, in response to said user adaptation command signal:

records said heterogeneous content that is of interest to said user;

finds relevant information from a database using said recorded heterogeneous content; and

utilizes said found relevant information to adapt to a user'"'"'s needs.

18. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 17, wherein the system develops accurate models of said users.

19. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 17, wherein the system determines fine-grained information regarding said user'"'"'s interest in Internet advertising.

20. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 17, wherein the system provides more accurate help regarding computer applications.

21. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 17, wherein said database includes at least one of the following:

a local machine database, a local area network database, and a wide area network database.

22. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, as per claim 21, wherein said wide area network database is the world wide web.

23. A system for recognizing reading, skimming, and scanning modes from eye-gaze patterns, said system comprising:

means for quantizing eye movements of a user viewing heterogeneous content in both X and Y axes;

means for accumulating a numerical evidence of reading, said numerical evidence independent of gaze time and factoring both positive and negative value;

means for comparing said numerical evidence of reading against a predetermined threshold; and

means for detecting reading when said numerical evidence of reading exceeds said predetermined threshold.

24. An article of manufacture comprising a computer program product having a machine-readable medium including computer program instructions embodied therein for recognizing reading, skimming, and scanning from eye-gaze patterns with:

computer program instructions for quantizing eye movements of a user viewing heterogeneous content in both X and Y axes, said quantizing based on averaging over a predetermined period of time;

computer program instructions for accumulating a numerical evidence of reading until a predetermined threshold is reached, said numerical evidence independent of gaze time and factoring both positive and negative values;

computer program instructions for detecting reading when said numerical evidence of reading exceeds said predetermined threshold; and

computer program instructions for switching modes from a scanning mode and a skimming mode to a reading mode when reading is detected.

25. An article of manufacture comprising a computer program product having a machine-readable medium including computer program instructions embodied therein for utilizing user interest information to adapt a computer to a user'"'"'s needs with:

computer program instructions for recording eye-gaze patterns of said user viewing heterogeneous content;

computer program instructions for determining from said recorded patterns whether said user is reading, skimming, and scanning based upon accumulated numerical evidence, said numerical evidence independent of gaze time and factoring both positive and negative values;

computer programs instructions for recording heterogeneous content of interest to said user upon detection of said reading;

computer program instructions for finding relevant information from a database using said recorded heterogeneous content of interest; and

computer program instructions for adapting said computer to said user'"'"'s needs using said revelant information.