Gaze tracking system, eye-tracking assembly and an associated method of calibration

• US 6,943,754 B2
• Filed: 09/27/2002
• Issued: 09/13/2005
• Est. Priority Date: 09/27/2002
• Status: Expired due to Fees

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Vorst, Carl J., Swaine, Steven D., Aughey, John H., Rohr, Michael V.
• Primary Examiner(s): Casler, Brian L.
• Assistant Examiner(s): Sanders, John R.
• Application Number:
  US10/259,103
• Publication Number:
  US 20040061831A1
• Time in Patent Office: 1,082 Days
• Field of Search: 351/200, 351/205, 351/206, 351/209, 351/210, 351/221, 353/28, 353/30, 353/39, 353/46, 353/121, 353/122, 345/700, 345/7, 345/8, 345/9, 600/558, 382/103, 348/115, 348/116, 348/121, 348/135, 434/44
• US Class Current: 345/8
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 2027/0198 System for aligning or maintaining alignment of an image in a predetermined direction
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A system for tracking a gaze of an operator includes a head-mounted eye tracking assembly, a head-mounted head tracking assembly and a processing element. The head-mounted eye tracking assembly comprises a visor having an arcuate shape including a concave surface and an opposed convex surface. The visor is capable of being disposed such that at least a portion of the visor is located outside a field of view of the operator. The head-mounted head tracking sensor is capable of repeatedly determining a position of the head to thereby track movement of the head. In this regard, each position of the head is associated with a position of the at least one eye. Thus, the processing element can repeatedly determine the gaze of the operator, based upon each position of the head and the associated position of the eyes, thereby tracking the gaze of the operator.

Claims

1. A head-mounted eye tracking assembly for tracking movement of at least one eye of an operator, said eye tracking assembly comprising:

a visor having an arcuate shape including a concave surface and an opposed convex surface, wherein said visor is capable of being disposed such that the at least one eye is located proximate the concave surface, wherein said visor is at least partially reflective such that said visor is capable of reflecting an image of the at least one eye, and wherein said visor is capable of being disposed such that at least a portion of said visor is located outside a field of view of the operator;

an image capture device disposed proximate the concave surface of said visor, wherein said image capture device is capable of tracking movement of the at least one eye based upon the image of the at least one eye reflected by said visor; and

a lens disposed in an optical path of said image capture device wherein an effective focus of said image capture device is based upon an optical power of said visor and an optical power of said lens.

2. A head-mounted eye tracking assembly according to claim 1, wherein the concave surface of said visor includes a reflective coating.

3. A head-mounted eye tracking assembly according to claim 1, further comprising an illuminator capable of illuminating the at least one eye.

4. A head-mounted eye tracking assembly according to claim 3, wherein said illuminator is capable of illuminating the at least one eye with infrared light.

5. A head-mounted eye tracking assembly according to claim 1, wherein said image capture device is located outside the field of view of the operator.

6. A head-mounted eye tracking assembly according to claim 1, wherein said visor is capable of reflecting an image of a pupil of at least one eye, and wherein said image capture device is capable of tracking movement of the at least one eye based upon the image of the pupil of the at least one eye.

7. A system for tracking a gaze of an operator, said gaze tracking system comprising:

a head-mounted eye tracking assembly comprising;

a visor having an arcuate shape including a concave surface and opposed an convex surface, wherein said visor is capable of being disposed such that the at least one eye is located proximate the concave side, wherein said visor is at least partially reflective such that said visor is capable of reflecting an image of the at least one eye, wherein said visor is capable of being disposed such that at least a portion of said visor is located outside a field of view of the operator, and wherein the head-mounted eye tracking assembly is capable of repeatedly determining a position of the at least one eye based upon the image of the at least one eye reflected by said visor to thereby track movement of the at least one eye;

an image capture device disposed proximate the concave surface of said visor, wherein the image capture device is capable of tracking movement of the at least one eye based upon the image of the at least one eye reflected by said visor; and

a lens disposed in an optical path of the image capture device, wherein an effective focus of the image capture device is based upon an optical power of the visor and an optical power of the lens;

a head-mounted head tracking sensor capable of repeatedly determining a position of the head to thereby track movement of the head, wherein each position of the head associated with a position of the at least one eye; and

a processing element capable of repeatedly determining the gaze of the operator to thereby track the gaze of the operator, wherein said processing element repeatedly determines the gaze of the operator based upon each position of the head and the associated position of the at least one eye.

8. A system according to claim 7, wherein the concave surface of the visor of said head-mounted eye tracking assembly includes a reflective coating.

9. A system according to claim 7, wherein said head-mounted eye tracking assembly further comprises an illuminator capable of illuminating the at least one eye.

10. A system according to claim 9, wherein the illuminator is capable of illuminating the at least one eye with infrared light.

11. A system according to claim 7, wherein the image capture device located outside the field of view of the operator.

12. A system according to claim 7, wherein said visor is capable of reflecting an image of a pupil of at least one eye, and wherein the head-mounted eye tracking assembly is capable of repeatedly determining a position of the at least one eye based upon the image of the pupil of the at least one eye.

13. A visual display system comprising:

a background image projector having a predetermined field of view, said background image projector generating a background image having a predetermined size an resolution that at least partially fills the field of view;

at least one inset image projector for generating an area of interest (AOI) image having a smaller size and a higher resolution than the background image, said inset image projector having a predetermined field of view that is capable of being movably disposed at least partially within the field of view of said background image projector; and

a gaze tracking system capable of directing the movement of the field of view of said inset image projector based upon the direction in which an operator is looking, said gaze tracking system comprising a head-mounted eye tracking assembly comprising;

a visor having an arcuate shape including a concave surface and an opposed convex surface, wherein said visor is capable of being disposed such that the at least one eye is located proximate the concave surface, wherein said visor is at least partially reflective such that said visor is capable of reflecting an image of the at east one eye, and wherein said visor is capable of being disposed such that at least a portion of said visor is located outside a field of view of the operator;

an image capture device disposed proximate the concave surface of said visor, wherein said image capture device is capable of tracking movement of the at least one eye based upon the image of the at least one eye reflected by said visor;

a lens disposed in an optical path of said image capture device, wherein an effective focus of said image capture device is based upon an optical power of said visor and an optical power of said lens.

14. A visual display system according to claim 13, wherein said gaze tracking system further comprises:

a head-mounted head tracking sensor capable of repeatedly determining a position of the head to thereby track movement of the head, wherein each position of the head is associated with a position of the at least one eye; and

a processing element capable of repeatedly determining the gaze of the operator to thereby track the gaze of the operator, wherein said processing element repeatedly determines the gaze of the operator based upon each position of the head and the associated position of the at least one eye.

15. A visual display system according to claim 13, where the concave surface of the visor of said head-mounted eye tracking assembly includes a reflective coating.

16. A visual display system according to claim 13, wherein said head-mounted eye tracking assembly further comprises an illuminator capable of illuminating the at least one eye.

17. A visual display system according to claim 13, wherein said head-mounted eye tracking assembly further comprises a lens disposed in an optical path of said image capture device, wherein an effective focus of said image capture device is based upon an optical power of said visor and an optical power of said lens.

18. A visual display system according to claim 13, wherein said visor is capable of reflecting an image of a pupil of at least one eye, and wherein said image capture device is capable of tracking movement of the at least one eye based upon the image of the pupil of the at least one eye.

19. A visual display system according to claim 13, wherein said inset image projector has a predetermined field of view that is capable of being movably disposed based upon electronic raster deflection.