AUTOMATIC CALIBRATION OF EYE TRACKING FOR OPTICAL SEE-THROUGH HEAD MOUNTED DISPLAY

• US 20150049013A1
• Filed: 01/23/2014
• Published: 02/19/2015
• Est. Priority Date: 08/19/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: RAHMAN, Md Sazzadur, LIU, Kexi, RENSCHLER, Martin H.
• Granted Patent: US 10,073,518 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An apparatus for calibrating an eye tracking system of a head mounted display displays a moving object in a scene visible through the head mounted display. The object is displayed progressively at a plurality of different points (P) at corresponding different times (T). While the object is at a first point of the plurality of different points in time, the apparatus determines whether an offset between the point P and an eye gaze point (E) satisfies a threshold. The eye-gaze point (E) corresponds to a point where a user is determined to be gazing by the eye tracking system. If the threshold is not satisfied, the apparatus performs a calibration of the eye tracking system when the object is at a second point of the plurality of different points in time. The apparatus then repeats the determining step when the object is at a third point of the plurality of different points in time.

Claims

1. A method of calibrating an eye tracking module of an optical see-through head mounted display (HMD) being worn by a user, said method comprising:

automatically displaying an object on the HMD, the object displayed progressively at a plurality of different object points (P) on the HMD at corresponding different times (T);

for an accuracy point (A) corresponding to one of the plurality of different object points (P), determining whether an offset between the accuracy point (A) and an eye gaze point (E) satisfies a threshold, the eye-gaze point (E) corresponding to a point on the HMD where the user is determined to be gazing by the eye tracking module;

if the threshold is not satisfied, performing a calibration of the eye tracking module for a calibration point (C) corresponding to one of the plurality of different object points (P); and

repeating the determining for a subsequent accuracy point (A) corresponding to one of the plurality of different object points (P).

2. The method of claim 1, wherein automatically displaying comprises displaying the object independent of user input.

3. The method of claim 1, wherein the calibration point (C) and accuracy point (A) are different object points (P).

4. The method of claim 1, wherein the threshold is satisfied when the offset between the accuracy point (A) and the eye-gaze point (E) is less than a predetermined value.

5. The method of claim 4, wherein:

the HMD has an associated screen coordinate system;

the accuracy point (A) is characterized by coordinates that represent a location of the object within the screen coordinate system;

the eye gaze point (E) is characterized by coordinates that represent a location of the eye gaze within the screen coordinate system; and

the offset is derived from the coordinates of the accuracy point (A) and the eye gaze point (E).

6. The method of claim 1, wherein the HMD has an associated screen coordinate system, and performing the calibration comprises:

obtaining coordinates of the calibration point (C) that represent a location of the object within the screen coordinate system;

obtaining coordinates of the eye gaze point (E) that represent a location of the eye gaze within the screen coordinate system;

determining an offset between the calibration point (C) and the eye gaze point (E) based on the respective coordinates; and

adjusting the coordinates of subsequent eye gaze points (E) output by the eye tracking module based on the offset.

7. An apparatus for calibrating an eye tracking system of a head mounted display, said apparatus comprising:

means for automatically displaying a moving object in a scene visible through the head mounted display, where the object is displayed progressively at a plurality of different points (P) at corresponding different times (T);

means for, at a first point of the plurality of different points in time, determining whether an offset between the point P and an eye gaze point (E) satisfies a threshold, where the eye-gaze point (E) correspond to a point where a user is determined to be gazing by the eye tracking system;

means for performing a calibration of the eye tracking system at a second point of the plurality of different points in time, if the threshold is not satisfied; and

means for repeating the determining step at a third point of the plurality of different points in time.

8. The apparatus of claim 7, wherein the means for automatically displaying is configured to display the object independent of user input.

9. The apparatus of claim 7, wherein the calibration point (C) and accuracy point (A) are different object points (P).

10. The apparatus of claim 7, wherein the threshold is satisfied when the offset between the accuracy point (A) and the eye-gaze point (E) is less than a predetermined value.

11. The apparatus of claim 10, wherein:

the HMD has an associated screen coordinate system;

the accuracy point (A) is characterized by coordinates that represent a location of the object within the screen coordinate system;

the eye gaze point (E) is characterized by coordinates that represent a location of the eye gaze within the screen coordinate system; and

the offset is derived from the coordinates of the accuracy point (A) and the eye gaze point (E).

12. The apparatus of claim 7, wherein the HMD has an associated screen coordinate system, and the means for performing the calibration is configured to:

obtain coordinates of the calibration point (C) that represent a location of the object within the screen coordinate system;

obtain coordinates of the eye gaze point (E) that represent a location of the eye gaze within the screen coordinate system;

determine an offset between the calibration point (C) and the eye gaze point (E) based on the respective coordinates; and

adjust the coordinates of subsequent eye gaze points (E) output by the eye tracking module based on the offset.

13. An apparatus for calibrating an eye tracking system of a head mounted display, said apparatus comprising:

a memory; and

at least one processor coupled to the memory and configured to;

display a moving object in a scene visible through the head mounted display, the object displayed progressively at a plurality of different points (P) at corresponding different times (T);

at a first point of the plurality of different points in time, determine whether an offset between the point P and an eye gaze point (E) satisfies a threshold, the eye-gaze point (E) corresponding to a point where a user is determined to be gazing by the eye tracking system;

if the threshold is not satisfied, perform a calibration of the eye tracking system at a second point of the plurality of different points in time; and

repeat the determining step at a third point of the plurality of different points in time.

14. The apparatus of claim 13, wherein the processor is configured to automatically display the object independent of user input.

15. The apparatus of claim 13, wherein the calibration point (C) and accuracy point (A) are different object points (P).

16. The apparatus of claim 13, wherein the threshold is satisfied when the offset between the accuracy point (A) and the eye-gaze point (E) is less than a predetermined value.

17. The apparatus of claim 16, wherein:

the HMD has an associated screen coordinate system;

the accuracy point (A) is characterized by coordinates that represent a location of the object within the screen coordinate system;

the eye gaze point (E) is characterized by coordinates that represent a location of the eye gaze within the screen coordinate system; and

the offset is derived from the coordinates of the accuracy point (A) and the eye gaze point (E).

18. The apparatus of claim 13, wherein the HMD has an associated screen coordinate system, and the processor performs calibration by being further configured to:

obtain coordinates of the calibration point (C) that represent a location of the object within the screen coordinate system;

obtain coordinates of the eye gaze point (E) that represent a location of the eye gaze within the screen coordinate system;

determine an offset between the calibration point (C) and the eye gaze point (E) based on the respective coordinates; and

adjust the coordinates of subsequent eye gaze points (E) output by the eye tracking module based on the offset.

19. A computer program product for calibrating an eye tracking system of a head mounted display, said product comprising:

a computer-readable medium comprising code for;

displaying a moving object in a scene visible through the head mounted display, the object displayed progressively at a plurality of different points (P) at corresponding different times (T);

at a first point of the plurality of different points in time, determining whether an offset between the point P and an eye gaze point (E) satisfies a threshold, the eye-gaze point (E) corresponding to a point where a user is determined to be gazing by the eye tracking system;

if the threshold is not satisfied, performing a calibration of the eye tracking system at a second point of the plurality of different points in time; and

repeating the determining step at a third point of the plurality of different points in time.