Portable eye tracking device

• US 10,310,597 B2
• Filed: 09/27/2016
• Issued: 06/04/2019
• Est. Priority Date: 09/03/2013
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Biedert, Ralf, Thunstrom, Robin, Andersson, Jonas
• Primary Examiner(s): McLoone, Peter D
• Application Number:
  US15/277,225
• Publication Number:
  US 20170017299A1
• Time in Patent Office: 980 Days
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: H04R 1/32 for obtaining desired directional characteristic only specially adapted for hearing aids H04R25/40
• CPC Class Codes: H04R 2201/40 Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups

Abstract

A portable eye tracker device is disclosed which includes a frame, at least one optics holding member, a movement sensor, and a control unit. The frame may be a frame adapted for wearing by a user. The at least one optics holding member may include at least one illuminator configured to selectively illuminate at least a portion of at least one eye of the user, and at least one image sensor configured to capture image data representing images of at least a portion of at least one eye of the user. The movement sensor may be configured to detect movement of the frame. The control unit may be configured to control the at least one illuminator for the selective illumination of at least a portion of at least one eye of the user, receive the image data from the image sensors, and receive information from the movement sensor.

Claims

1. A method for calibrating a gaze detection system, wherein the method comprises:

displaying a plurality of multipurpose virtual objects as part of an interactive virtual experience in a virtual reality environment on a display of a wearable device worn by a user;

selecting a multipurpose virtual object of the plurality of multipurpose virtual objects based at least in part on a determination that the user is anticipated to interact with the multipurpose virtual object during the interactive virtual experience;

detecting a user interaction of the user with the multipurpose virtual object during the interactive virtual experience;

determining that the user interaction of the user with the multipurpose virtual object causes a change to the multipurpose virtual object; and

responsive to determining that the user interaction of the user with the multipurpose virtual object causes the change to the multipurpose virtual object, initiating, during the interactive virtual experience, a calibration process of the gaze detection system, the calibration process comprising;

determining a gaze direction of the user relative to the multipurpose virtual object with the gaze detection system of the wearable device; and

calibrating the gaze detection system based at least in part on the gaze direction.

2. The method for calibrating a gaze detection system of claim 1, wherein:

the method further comprises receiving an input; and

calibrating the gaze detection system occurs in response to the input.

3. The method for calibrating a gaze detection system of claim 2, wherein the input comprises:

an input from a hand held controller.

4. The method for calibrating a gaze detection system of claim 2, wherein the input comprises:

an interaction by the user with the multipurpose virtual object that moves the multipurpose virtual object closer to a particular portion of a virtual body of the user in the virtual reality environment.

5. The method for calibrating a gaze detection system of claim 2, wherein the input comprises:

the gaze direction remaining consistent for a predetermined amount of time.

6. The method for calibrating a gaze detection system of claim 1, wherein:

the multipurpose virtual object comprises a salient feature; and

determining the gaze direction of the user relative to the multipurpose virtual object comprises determining the gaze direction of the user relative to the salient feature.

7. The method for calibrating a gaze detection system of claim 1, wherein determining that the user interaction of the user with the multipurpose virtual object causes the change to the multipurpose virtual object comprises:

determining that the user has picked up the multipurpose virtual object in the virtual reality environment.

8. The method for calibrating a gaze detection system of claim 1, wherein determining that the user interaction of the user with the multipurpose virtual object causes the change to the multipurpose virtual object comprises:

determining that the user has moved the multipurpose virtual object in the virtual reality environment.

9. The method for calibrating a gaze detection system of claim 8, wherein the determining that the user has moved the multipurpose virtual object comprises:

determining that the user has moved the multipurpose virtual object closer to a particular portion of a virtual body of the user in the virtual reality environment.

10. The method for calibrating a gaze detection system of claim 1, wherein the multipurpose virtual object is used in the virtual reality environment for one or more purposes in addition to the calibration process.

11. The method for calibrating a gaze detection system of claim 1, wherein the calibration process is a recalibration process.

12. The method for calibrating a gaze detection system of claim 1, wherein a first purpose of the multipurpose virtual object is for the calibration process of the gaze detection system and a second purpose of the multipurpose virtual object is for the interactive virtual experience other than the calibration process.

13. A non-transitory machine-readable medium having instructions stored thereon for calibrating a gaze detection system, wherein the instructions are executable by at least one processor to at least:

displaying a plurality of multipurpose virtual objects as part of an interactive virtual experience in a virtual reality environment on a display of a wearable device worn by a user;

selecting a multipurpose virtual object of the plurality of multipurpose virtual objects based at least in part on a determination that the user is anticipated to interact with the multipurpose virtual object during the interactive virtual experience;

detect a user interaction of the user with the multipurpose virtual object during the interactive virtual experience;

determine that the user interaction of the user with the multipurpose virtual object causes a change to the multipurpose virtual object; and

responsive to determining that the user interaction of the user with the multipurpose virtual object causes the change to the multipurpose virtual object, initiate, during the interactive virtual experience, a calibration process of the gaze detection system, the calibration process comprising;

determine a gaze direction of the user relative to the multipurpose virtual object with the gaze detection system of the wearable device; and

calibrate the gaze detection system based at least in part on the gaze direction.

14. The non-transitory machine-readable medium of claim 13, wherein the instructions are further executable to at least:

receive an input, wherein calibrating the gaze detection system occurs in response to the input.

15. The non-transitory machine-readable medium of claim 14, wherein the input comprises:

the gaze direction remaining consistent for a predetermined amount of time.

16. The non-transitory machine-readable medium of claim 13, wherein:

the multipurpose virtual object comprises a salient feature; and

determining the gaze direction of the user relative to the multipurpose virtual object comprises determining the gaze direction of the user relative to the salient feature.

17. The non-transitory machine-readable medium of claim 13, wherein determining that the user interaction of the user with the multipurpose virtual object causes the change to the multipurpose virtual object comprises:

determining that the user has picked up the multipurpose virtual object in the virtual reality environment.

18. A system for determining a gaze direction of a user, wherein the system comprises:

a wearable device having a display and a gaze detection system; and

one or more processors configured to at least;

display a plurality of multipurpose virtual objects as part of an interactive virtual experience in a virtual reality environment on the display;

select a multipurpose virtual object of the plurality of multipurpose virtual objects based at least in part on a determination that the user is anticipated to interact with the multipurpose virtual object during the interactive virtual experience;

detect a user interaction of a user with the multipurpose virtual object during the interactive virtual experience;

determine that the user interaction of the user with the multipurpose virtual object causes a change to the multipurpose virtual object;

in response to determining that the user interaction of the user with the multipurpose virtual object causes the change to the multipurpose virtual object, initiate, during the interactive virtual experience, a calibration process of the gaze detection system, the calibration process comprising;

determining a gaze direction of the user relative to the multipurpose virtual object with the gaze detection system; and

calibrating the gaze detection system based at least in part on the gaze direction.

19. The system of claim 18, wherein:

the multipurpose virtual object comprises a salient feature; and

determining the gaze direction of the user relative to the multipurpose virtual object comprises determining the gaze direction of the user relative to the salient feature.

20. The system of claim 18, wherein determining that the user interaction of the user with the multipurpose virtual object causes the change to the multipurpose virtual object comprises:

determining that the user has picked up the multipurpose virtual object in the virtual reality environment.