Medical diagnostic gaze tracker

• US 9,888,842 B2
• Filed: 05/31/2012
• Issued: 02/13/2018
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: White, Sean M., Nguyen, David H., Lyons, Kent M., Ashbrook, Daniel L.
• Primary Examiner(s): Pichler, Marin
• Application Number:
  US13/484,442
• Publication Number:
  US 20130321772A1
• Time in Patent Office: 2,084 Days
• Field of Search: 351209, 351210, 600558
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A method to track a user'"'"'s gaze for medical diagnostics is described. The method includes detecting a gaze behavior of a user in response to a stimulus. The stimulus is a non-diagnostic stimulus. The method also includes determining a type of the stimulus and generating an indication of the gaze behavior and the type of the stimulus. The indication may then be used to generate an expected gaze pattern describing expected gaze behavior to given stimuli. Additionally, the indication may be used to determine whether the gaze behavior is indicative of a potential problem and, in response, an alert may be generated. Apparatus and computer readable media are also described.

Claims

1. A method comprising:

detecting a gaze behavior of a user in response to a stimulus, where the stimulus is a non-diagnostic stimulus that is a visual stimulus that is not specifically created or generated for a diagnostic testing machine or method;

determining a type of the non-diagnostic visual stimulus by using a camera to image a scene being viewed by the user and by using a processor-implemented object recognition technique to identify an object or objects in the imaged scene to which the gaze of the user is directed;

assigning the identified object or objects to a particular non-diagnostic visual stimulus classification and type of non-diagnostic visual stimulus;

associating the detected gaze behavior with the identified object or objects; and

generating an indication of the gaze behavior and the determined type of the non-diagnostic visual stimulus.

2. The method of claim 1, further comprising determining a gaze pattern based on a plurality of gaze behaviors and associated types of non-diagnostic visual stimuli, where the plurality of gaze behaviors and associated types of non-diagnostic visual stimuli comprise the gaze behavior and the type of non-diagnostic visual stimulus.

3. The method of claim 1, further comprising determining whether the gaze behavior to the type of non-diagnostic visual stimulus deviates from a gaze pattern.

4. The method of claim 1, further comprising, in response determining that the gaze behavior to the type of non-diagnostic visual stimulus matches a known behavior, generating an alert.

5. The method of claim 1, where determining the type of the non-diagnostic visual stimulus comprises using at least one of optical character recognition, template matching, weak classifier cascades, bag of features, edge-matching, and histogram matching, and further comprising classifying an identified object in the scene as to a type of non-diagnostic visual stimulus.

6. The method of claim 1, where determining the type of the non-diagnostic visual stimulus further comprises determining a type of an image shown on a display.

7. The method of claim 1, where determining the type of the non-diagnostic visual stimulus further comprises determining a type of an image shown on a display, and where the display is a near eye display device.

8. The method of claim 1, further comprising storing the indication of the gaze behavior and the type of non-diagnostic visual stimulus.

9. The method of claim 1, where the non-diagnostic visual stimulus that is detected by the method is a real world non-diagnostic visual stimulus.

10. An apparatus, comprising:

at least one processor;

a gaze tracker having an output coupled to the processor;

at least one camera having an output coupled to the processor; and

at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, control the apparatus to perform, as a result of execution of the computer program code by the at least one processor, operations that comprise;

detect based at least in part on an output of the gaze tracker a gaze behavior of a user in response to a stimulus, where the stimulus is a non-diagnostic stimulus that is a visual stimulus that is not specifically created or generated for a diagnostic testing machine or method;

determine using the at least one camera and the at least one processor a type of the non-diagnostic visual stimulus by using the at least one camera to image a scene being viewed by the user and by using an object recognition technique executed by the at least one processor to identify an object or objects in the imaged scene to which the gaze of the user is directed;

assign using the at least one processor the identified object or objects to a particular non-diagnostic visual stimulus classification and type of non-diagnostic visual stimulus;

associate the detected gaze behavior with the identified object or objects; and

generate and store in the at least one memory an indication of the gaze behavior and the determined type of the non-diagnostic visual stimulus.

11. The apparatus of claim 10, where the operations that result from the execution of the computer program code by the at least one processor further comprise:

determine a gaze pattern based on a plurality of gaze behaviors and associated types of non-diagnostic visual stimuli, where the plurality of gaze behaviors and associated types of non-diagnostic visual stimuli comprise the gaze behavior and the type of non-diagnostic visual stimulus.

12. The apparatus of claim 10, where the operations that result from the execution of the computer program code by the at least one processor further comprise:

generate an alert in response determining that the gaze behavior to the type of non-diagnostic visual stimulus matches a known gaze behavior.

13. The apparatus of claim 10, where the operations that result from the execution of the computer program code by the at least one processor further comprise:

when determining the type of the non-diagnostic visual stimulus, control the apparatus to receive an image from the at least one camera that is disposed to be facing in the direction of the gaze; and

determine the type of the non-diagnostic visual stimulus in the image in accordance with least one of optical character recognition, template matching, weak classifier cascades, bag of features, edge-matching, and histogram matching.

14. The apparatus of claim 10, where the operations that result from the execution of the computer program code by the at least one processor further comprise:

when determining the type of the non-diagnostic visual stimulus, control the apparatus to determine a type of an image shown on a display.

15. A computer readable medium tangibly encoded with a computer program executable by a processor to perform actions comprising:

detecting a gaze behavior of a user in response to a stimulus, where the stimulus is a non-diagnostic stimulus that is a visual stimulus that is not specifically created or generated for a diagnostic testing machine or method;

determining a type of the non-diagnostic visual stimulus by using a camera to image a scene being viewed by the user and by using an object recognition technique to identify an object or objects in the imaged scene to which the gaze of the user is directed;

assigning the identified object or objects to a particular non-diagnostic visual stimulus classification and type of non-diagnostic visual stimulus;

associating the detected gaze behavior with the identified object or objects; and

generating an indication of the gaze behavior and the determined type of the non-diagnostic visual stimulus.

16. The computer readable medium of claim 15, the actions further comprising determining a gaze pattern based on a plurality of gaze behaviors and associated types of non-diagnostic visual stimuli, where the plurality of gaze behaviors and associated types of non-diagnostic visual stimuli comprise the gaze behavior and the type of non-diagnostic visual stimulus.

17. The computer readable medium of claim 15, the actions further comprising, in response determining that the gaze behavior to the type of non-diagnostic visual stimulus matches a known behavior, generating an alert.

18. The computer readable medium of claim 15, where determining the type of the non-diagnostic visual stimulus comprises using at least one of optical character recognition, template matching, weak classifier cascades, bag of features, edge-matching, and histogram matching, and further comprising classifying an identified object or objects in the scene as to a type of non-diagnostic visual stimulus.

19. A method, comprising:

in response to a user wearing a headset that comprises a gaze tracker that faces in a direction of the user'"'"'s eyes, a transparent lens through which the user can view a scene and a forward facing camera configured to image the scene viewed by the user;

identifying, using a processor-implemented object recognition technique, an object or objects in the imaged scene to which the gaze of the user is currently being directed;

classifying an identified object or objects in the scene as to being a determined one of a plurality of types of non-diagnostic visual stimulus, where a non-diagnostic visual stimulus is a visual stimulus that is not specifically created or generated for a diagnostic testing machine or method;

determining a gaze pattern of the user in response to the non-diagnostic visual stimulus resulting from the object or objects in the scene being viewed by the user;

associating the determined gaze pattern with the identified object or objects; and

comparing the determined gaze pattern to historically determined gaze patterns of the user that correspond to a same determined type of non-diagnostic visual stimulus.

20. The method as in claim 19, where identifying the object or objects in the images scene comprises using at least one of processor-executed optical character recognition, template matching, weak classifier cascades, bag of features, edge-matching, and histogram matching.