Eye controllable screen pointer
First Claim
1. An eye controllable pointer system for determining a point of true gaze of a user with respect to a screen, comprising:
- a screen beacon including a plurality of light sources, the light sources fixed with respect to the screen and arranged noncollinearly;
a helmet secured to the head of the user;
a screen beacon tracking camera mounted on the helmet and situated to receive an image of the plurality of light sources, the screen beacon tracking camera producing a beacon tracking signal corresponding to the image received by the beacon tracking camera;
an eye gaze tracker mounted on the helmet, the eye gaze tracker detecting a gaze direction of at least one eye of the user, and producing an eye tracking signal corresponding to the detected gaze direction; and
a processor that receives the beacon tracking signal and the eye tracking signal and calculates a point of computed gaze of the user with respect to the screen as a function of the eye tracking signal and the beacon tracking signal.
0 Assignments
0 Petitions
Accused Products
Abstract
An eye controllable screen pointer system is provided that combines eye gaze tracking and screen tracking from the point of view of the user. Screen tracking is performed by a screen tracking camera, attached to a helmet the user is wearing. The screen tracking camera is aided by either light emitted from screen beacon located near the screen, or by a light pattern emitted from the screen itself as a screen beacon signal. The screen tracking means provides a screen tracking signal. Eye gaze tracking is performed as is known in the art, or according to a novel way described herein. An eye gaze tracking means is attached to the helmet, and provides an eye tracking signal. The information carried in the screen tracking signal and in the eye tracking signal are combined in a calculation by a processing means residing in a processor to produce a point of computed gaze on the screen. In an alternate embodiment of the invention, the input device of both the eye gaze tracking means and the screen tracking camera are combined into a single video camera, thus resulting in a simplified apparatus. Optionally, the system further projects a mark at the point of computed gaze.
135 Citations
30 Claims
-
1. An eye controllable pointer system for determining a point of true gaze of a user with respect to a screen, comprising:
-
a screen beacon including a plurality of light sources, the light sources fixed with respect to the screen and arranged noncollinearly;
a helmet secured to the head of the user;
a screen beacon tracking camera mounted on the helmet and situated to receive an image of the plurality of light sources, the screen beacon tracking camera producing a beacon tracking signal corresponding to the image received by the beacon tracking camera;
an eye gaze tracker mounted on the helmet, the eye gaze tracker detecting a gaze direction of at least one eye of the user, and producing an eye tracking signal corresponding to the detected gaze direction; and
a processor that receives the beacon tracking signal and the eye tracking signal and calculates a point of computed gaze of the user with respect to the screen as a function of the eye tracking signal and the beacon tracking signal. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. An eye controllable pointer system for determining a point of true gaze of a user with respect to a screen, comprising:
-
a screen beacon including a plurality of light sources, the light sources fixed with respect to the screen and arranged noncollinearly in a predetermined shape in a plane substantially parallel to the screen;
a helmet secured to the head of the user;
a screen beacon tracking camera mounted on the helmet and situated to receive an image of the plurality of light sources, the screen beacon tracking camera producing a beacon tracking signal corresponding to the image received by the beacon tracking camera;
an eye gaze tracker mounted on the helmet, the eye gaze tracker detecting a gaze direction of at least one eye of the user, and producing an eye tracking signal corresponding to the detected gaze direction; and
a processor that receives the beacon tracking signal and the eye tracking signal and calculates a point of computed gaze of the user with respect to the screen as a function of the eye tracking signal and the beacon tracking signal, wherein the image of the plurality of light sources has a distorted shape, and the processor calculates a transformation that transforms the distorted shape into a shape proportional to the predetermined shape in order to determine the point of computed gaze. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. A method for determining a point of true gaze of a user with respect to a screen, comprising:
-
fixing at least three noncollinearly arranged light sources with respect to the screen;
producing a screen beacon tracking signal corresponding to an image of the light sources;
detecting a gaze direction of at least one eye of the user, and producing an eye tracking signal corresponding to the detected gaze direction; and
calculating a point of computed gaze of the user with respect to the screen as a function of the screen beacon tracking signal and the eye tracking signal. - View Dependent Claims (23, 24, 25, 26, 27, 28)
-
-
29. An eye controllable pointer system for determining a point of true gaze of a user with respect to a screen, comprising:
-
four light sources, fixed with respect to the screen and arranged at corners of a rectangle having dimensions proportional to corresponding dimensions of the screen;
a helmet securable to the head of the user;
a screen beacon tracking camera mounted on the helmet, the beacon tracking camera including a beacon lens having a beacon lens axis, the screen beacon tracking camera situated to receive an image of the rectangle of light sources, the screen beacon tracking camera producing a beacon tracking signal corresponding to the image;
an eye gaze tracker mounted on the helmet for detecting a gaze direction of at least one eye of the user, the eye gaze tracker including an eye gaze tracking camera and an eye gaze lens having an eye gaze lens axis, the eye gaze tracking camera receiving an image of at least a portion of an eye of the user along the eye gaze lens axis and producing an eye tracking signal corresponding to a detected gaze direction, wherein the beacon lens axis and the eye gaze lens axis are not coaxial;
a processor that receives the screen beacon tracking signal and the eye tracking signal, the processor calculating a transformation that transforms the distorted image received by the screen beacon camera into a rectangular image with dimensions corresponding to the dimensions of the rectangle of light sources, and calculates a point of computed gaze of the user with respect to the screen as a function of the eye tracking signal and the transformation;
a controller, adjustable to align the point of computed gaze and the point of true gaze; and
a screen driver that displays a marker on the screens the marker indicative of the point of true gaze.
-
-
30. An eye controllable pointer system for determining a point of true gaze of a user with respect to a screen, comprising:
-
a screen beacon including a plurality of light sources, the light sources fixed with respect to the screen and arranged in a predetermined shape;
a helmet secured to the head of the user;
a screen beacon tracking camera mounted on the helmet and situated to receive an image of the plurality of light sources having a distorted shape, the screen beacon tracking camera producing a beacon tracking signal corresponding to the image received by the beacon tracking camera;
an eye gaze tracker mounted on the helmet, the eye gaze tracker detecting a gaze direction of at least one eye of the user, and producing an eye tracking signal corresponding to the detected gaze direction; and
a processor that receives the beacon tracking signal and the eye tracking signal and calculates a point of computed gaze of the user with respect to the screen as a function of the eye tracking signal and the beacon tracking signal based on the predetermined shape and the distorted shape.
-
Specification