Gaze tracking system, eye-tracking assembly and an associated method of calibration

• US 20040061831A1
• Filed: 09/27/2002
• Published: 04/01/2004
• Est. Priority Date: 09/27/2002
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Vorst, Carl J., Swaine, Steven D., Aughey, John H., Rohr, Michael V.
• Granted Patent: US 6,943,754 B2
• Time in Patent Office: Days
• US Class Current: 351/209
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 2027/0198 System for aligning or maintaining alignment of an image in a predetermined direction
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A system for tracking a gaze of an operator includes a head-mounted eye tracking assembly, a head-mounted head tracking assembly and a processing element. The head-mounted eye tracking assembly comprises a visor having an arcuate shape including a concave surface and an opposed convex surface. The visor is capable of being disposed such that at least a portion of the visor is located outside a field of view of the operator. The head-mounted head tracking sensor is capable of repeatedly determining a position of the head to thereby track movement of the head. In this regard, each position of the head is associated with a position of the at least one eye. Thus, the processing element can repeatedly determine the gaze of the operator, based upon each position of the head and the associated position of the eyes, thereby tracking the gaze of the operator.

Claims

1. A head-mounted eye tracking assembly for tracking movement of at least one eye of an operator, said eye tracking assembly comprising:

a visor having an arcuate shape including a concave surface and an opposed convex surface, wherein said visor is capable of being disposed such that the at least one eye is located proximate the concave surface, wherein said visor is at least partially reflective such that said visor is capable of reflecting an image of the at least one eye, and wherein said visor is capable of being disposed such that at least a portion of said visor is located outside a field of view of the operator; and

a camera disposed proximate the concave surface of said visor, wherein said camera is capable of tracking movement of the at least one eye based upon the image of the at least one eye reflected by said visor.

2. A head-mounted eye tracking assembly according to claim 1, wherein the concave surface of said visor includes a reflective coating.

3. A head-mounted eye tracking assembly according to claim 1, further comprising an illuminator capable of illuminating the at least one eye.

4. A head-mounted eye tracking assembly according to claim 3, wherein said illuminator is capable of illuminating the at least one eye with infrared light.

5. A head-mounted eye tracking assembly according to claim 1 further comprising a lens disposed in an optical path of said camera, wherein an effective focus of said camera is based upon an optical power of said visor and an optical power of said lens.

6. A head-mounted eye tracking assembly according to claim 1, wherein said camera is located outside the field of view of the operator.

7. A system for tracking a gaze of an operator, said gaze tracking system comprising:

a head-mounted eye tracking assembly comprising a visor having an arcuate shape including a concave surface and an opposed convex surface, wherein said visor is capable of being disposed such that the at least one eye is located proximate the concave side, wherein said visor is at least partially reflective such that said visor is capable of reflecting an image of the at least one eye, wherein said visor is capable of being disposed such that at least a portion of said visor is located outside a field of view of the operator, and wherein the head-mounted eye tracking assembly is capable of repeatedly determining a position of the at least one eye based upon the image of the at least one eye reflected by said visor to thereby track movement of the at least one eye;

a head-mounted head tracking sensor capable of repeatedly determining a position of the head to thereby track movement of the head, wherein each position of the head is associated with a position of the at least one eye; and

a processing element capable of repeatedly determining the gaze of the operator to thereby track the gaze of the operator, wherein said processing element repeatedly determines the gaze of the operator based upon each position of the head and the associated position of the at least one eye.

8. A system according to claim 7, wherein the concave surface of the visor of said head-mounted eye tracking assembly includes a reflective coating.

9. A system according to claim 7, wherein said head-mounted eye tracking assembly further comprises an illuminator capable of illuminating the at least one eye.

10. A system according to claim 9, wherein the illuminator is capable of illuminating the at least one eye with infrared light.

11. A system according to claim 7, wherein said head-mounted eye tracking assembly further comprises a camera disposed proximate the concave surface of said visor, and wherein the camera is capable of tracking movement of the at least one eye based upon the image of the at least one eye reflected by said visor.

12. A system according to claim 11, wherein said head-mounted eye tracking assembly further comprises a lens disposed in an optical path of the camera, wherein an effective focus of the camera is based upon an optical power of the visor and an optical power of the lens.

13. A system according to claim 11, wherein the camera is located outside the field of view of the operator.

14. A method of calibrating a gaze tracking system relative to an object, wherein the gaze tracking system includes an eye tracking assembly capable of tracking movement of at least one eye of an operator and a head tracking sensor capable of tracking movement of a head of the operator, said method comprising:

repeatedly moving the head tracking sensor to a plurality of different positions;

measuring a position of the at least one eye and an angle offset of a line of sight of the at least one eye relative to a first reference point, and measuring the position of the head tracking sensor and an angle offset of the head tracking sensor relative to a second reference position, wherein measuring occurs for each of the plurality of different positions; and

calibrating the gaze tracking system based upon the measured positions of the at least one eye, the offset angles of the at least one eye, the positions of the head tracking sensor and the offset angles of the head tracking sensor relative to the second reference position.

15. A method according to claim 14, wherein calibrating comprises calibrating the gaze tracking system with an artificial neural network.

16. A method according to claim 14, wherein calibrating comprises calibrating the gaze tracking system according to an interpolation technique.

17. A method according to claim 14 further comprising aligning the line of sight of the at least one eye with the first reference point and setting a position of the head tracking sensor as the second reference position, wherein the aligning step occurs before moving the head tracking sensor.

18. A method according to claim 17 further comprising moving the first reference point to at least one additional position, wherein aligning the line of sight, moving the head tracking sensor, and measuring a position occur for each additional position of the first reference point.

19. A visual display system comprising:

a background image projector having a predetermined field of view, said background image projector generating a background image having a predetermined size and resolution that at least partially fills the field of view;

at least one inset image projector for generating an area of interest (AOI) image having a smaller size and a higher resolution than the background image, said inset image projector having a predetermined field of view that is capable of being movably disposed at least partially within the field of view of said background image projector; and

a gaze tracking system capable of directing the movement of the field of view of said inset image projector based upon the direction in which an operator is looking, said gaze tracking system comprising a head-mounted eye tracking assembly comprising;

a visor having an arcuate shape including a concave surface and an opposed convex surface, wherein said visor is capable of being disposed such that the at least one eye is located proximate the concave surface, wherein said visor is at least partially reflective such that said visor is capable of reflecting an image of the at least one eye, and wherein said visor is capable of being disposed such that at least a portion of said visor is located outside a field of view of the operator; and

a camera disposed proximate the concave surface of said visor, wherein said camera is capable of tracking movement of the at least one eye based upon the image of the at least one eye reflected by said visor.

20. A visual display system according to claim 19, wherein said gaze tracking system further comprises:

a head-mounted head tracking sensor capable of repeatedly determining a position of the head to thereby track movement of the head, wherein each position of the head is associated with a position of the at least one eye; and

a processing element capable of repeatedly determining the gaze of the operator to thereby track the gaze of the operator, wherein said processing element repeatedly determines the gaze of the operator based upon each position of the head and the associated position of the at least one eye.

21. A visual display system according to claim 19, wherein the concave surface of the visor of said head-mounted eye tracking assembly includes a reflective coating.

22. A visual display system according to claim 19, wherein said head-mounted eye tracking assembly further comprises an illuminator capable of illuminating the at least one eye.

23. A visual display system according to claim 19, wherein said head-mounted eye tracking assembly further comprises a lens disposed in an optical path of said camera, wherein an effective focus of said camera is based upon an optical power of said visor and an optical power of said lens.