Fast wake-up in a gaze tracking system

• US 9,766,699 B2
• Filed: 08/08/2013
• Issued: 09/19/2017
• Est. Priority Date: 08/09/2012
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Skog, Mrten, Olsson, Anders, Elvesj, John Mikael, Yu, Aron
• Primary Examiner(s): Eisen, Alexander
• Assistant Examiner(s): ALMEIDA, CORY A
• Application Number:
  US13/962,151
• Publication Number:
  US 20140043227A1
• Time in Patent Office: 1,503 Days
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence

Abstract

A gaze tracking system, leaving a low power mode in response to an activation signal, captures an initial burst of eye pictures in short time by restricting the image area of a sensor, with the purpose of enabling an increased frame rate. Subsequent eye pictures are captured at normal frame rate. The first gaze point value is computed memorylessly based on the initial burst of eye pictures and no additional imagery, while subsequent values may be computed recursively by taking into account previous gaze point values or information from previous eye pictures. The restriction of the image area may be guided by a preliminary overview picture captured using the same or a different sensor. From the gaze point values, the system may derive a control signal to be supplied to a computer device with a visual display.

Claims

1. A method in a gaze tracking system for controlling a computer device associated with a visual display, wherein the gaze tracking system tracks a gaze point on said visual display, said gaze tracking system comprising at least one sensor with a light-sensitive surface, the method comprising:

in response to a gaze tracking activation signal from input means of said computer device at said gaze tracking system, leaving a low power mode of said gaze tracking system;

capturing an overview picture of at least a portion of the head of a viewer and locating an eye region of the viewer in the overview picture;

capturing, at a first frame rate, a plurality of pictures of at least a portion of the viewer'"'"'s head using a first region of the light-sensitive surface based on the overview picture, which are positioned in accordance with the eye region;

determining a first gaze point value of said viewer based on said plurality of pictures only and optionally the overview picture;

providing a first control signal to the computer device based on said first gaze point value;

capturing, at a second frame rate, being lower than said first frame rate, a plurality of further pictures of at least a portion of the viewer'"'"'s head using a second region of the light-sensitive surface after providing the first control signal, wherein the second region of the light-sensitive surface is larger than the first region of the light-sensitive surface;

determining a further gaze point value of said viewer based on a further picture from said plurality of further pictures and a picture from said plurality of pictures; and

providing a further control signal to the computer device based on said further gaze point value.

2. The method of claim 1, wherein the ratio between the first frame rate and the second frame rate is at least 2.

3. The method of claim 2, wherein:

said first gaze point value is determined by jointly filtering pictures in said plurality of pictures or a quantity derived from said pictures and optionally based on additional information from the overview picture; and

said further gaze point value is determined by jointly filtering said further picture and said picture or a quantity derived from said pictures.

4. The method of claim 1, wherein:

said first gaze point value is determined by jointly filtering pictures in said plurality of pictures or a quantity derived from said pictures and optionally based on additional information from the overview picture; and

said further gaze point value is determined by jointly filtering said further picture and said picture or a quantity derived from said pictures.

5. The method of claim 4, wherein:

the step of determining said further gaze point value includes filtering gaze point guesses derived from one or more pictures at a time; and

the filtering in the step of determining said further gaze point value is conditional upon a maximal drift of the gaze point guesses.

6. The method of claim 1, wherein the overview picture is captured by reading out data from a region representing at least 50% of the area of the light-sensitive surface of the at least one sensor, which at least one sensor is preferably operated in a reduced resolution mode.

7. The method of claim 1, wherein the overview picture is captured by an imaging device which is distinct from the at least one sensor, preferably by one of the following:

receiving data from a depth sensing system;

receiving data from an RGB-type camera;

receiving data from a near-infrared camera.

8. The method of claim 1, wherein the gaze tracking activation signal directly triggers an interrupt in the gaze tracking system, or wherein the gaze tracking activation signal is a USB request.

9. The method of claim 1, wherein the gaze tracking activation signal is received from input means which are of a non-gaze type, and preferably of a body gesture detector, an acoustic transducer, a touch detector, a pointing device or a body proximity detector type.

10. A computer program product to be executed by a gaze tracking system for controlling a computer device associated with a visual display, said product comprising a computer-readable medium with instructions for causing the gaze tracking system to perform the method of claim 1.

11. The method of claim 1, wherein the first region occupies at most 10% of the area of the light-sensitive surface.

12. A gaze tracking system for controlling a computer device associated with a visual display, wherein the gaze tracking system tracks a gaze point on said visual display, the system comprising a processor and being communicatively connected:

to at least one sensor with a light-sensitive surface, the at least one sensor being selectively operable to capture pictures at a first frame rate using a first region of the light-sensitive surface or at a second frame rate using a second region of the light-sensitive surface, wherein said second frame rate is lower than said first frame rate, and wherein said second region is larger than said first region; and

to an overview imaging device configured to capture an overview picture of at least a portion of the head of a viewer,

wherein the processor is configured to respond to receipt, in a low power mode of the gaze tracking system, of a gaze tracking activation signal from input means of said computer device by performing the following;

causing the gaze tracking system to leave the low power mode;

capturing an overview picture of at least a portion of the head of the viewer using said overview imaging device and locating an eye region of the viewer in the overview picture;

capturing, at the first frame rate, a plurality of pictures of at least a portion of the viewer'"'"'s head using said first region of the light-sensitive surface, in which an image of an eye of the viewer is to be found;

determining a first gaze point value of said viewer based on said plurality of pictures only and optionally the overview picture;

providing a first control signal to the computer device based on said first gaze point value;

capturing, at the second frame rate, a plurality of further pictures of at least a portion of the viewer'"'"'s head using said second region of the light-sensitive surface after providing the first control signal;

determining a further gaze point value of said viewer on the basis of a further picture from said plurality of further pictures and a picture from said plurality of pictures; and

providing a further control signal to the computer device based on said further gaze point value.

13. The gaze tracking system of claim 12, further comprising a filter configured to provide a gaze point value on the basis of a plurality of pictures of the viewer'"'"'s head or a quantity derived from a plurality of such pictures and optionally further on the basis of additional information from the overview picture.

14. The gaze tracking system of claim 12, wherein the overview imaging device coincides with the at least one sensor and the overview picture is preferably captured by operating the at least one sensor in a reduced resolution mode.

15. The gaze tracking system of claim 12, further comprising an interrupt pin configured to receive the gaze tracking activation signal from the input means of the computer device.