DETERMINING GAZE TARGET BASED ON FACIAL FEATURES

• US 20170177079A1
• Filed: 03/03/2017
• Published: 06/22/2017
• Est. Priority Date: 03/01/2013
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: George-Svahn, Erland, Henderek, David Figgins, Lannsj, Rebecka, Skog, Mrten, Elvesj, John
• Granted Patent: US 10,545,574 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0227 Cooperation and interconnection of the input arrangement with other functional units of a computer G06F3/023 - G06F3/037 take precedence
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning

Abstract

A computer system can be controlled with non-contact inputs, such as eye-tracking devices. A visual indicator can be presented on a display to indicate the location where a computer function will take place (e.g., a common cursor). The visual indicator can be moved to a gaze target in response to continued detection of an action (e.g., touchpad touch) by a user for a predetermined period of time. A delay between the action and the movement of the visual indicator can allow a user time to “abort” movement of the visual indicator. Additionally, once the visual indicator has moved, the visual indicator can be controlled with additional precision as the user moves the gaze while continuing the action (e.g., continued holding of the touchpad).

Claims

1. A computing device, comprising:

a display device for presenting a display;

an image sensor configured to capture images of at least a portion of a user of the computing device; and

at least one processor;

wherein the processor is configured to;

determine multiple facial features of the user based on at least one captured image; and

determine a gaze target of the user based on the facial features, the gaze target having a vertical component and a horizontal component, wherein the vertical component is determined based on a first facial feature, and the horizontal component is determined based on a second facial feature.

2. The computing device of claim 1, wherein the first facial feature comprises an eye lid position.

3. The computing device of claim 2, wherein the processor is further configured to determine the vertical component based on a detected openness of the eye lid position.

4. The computing device of claim 1, wherein the second facial feature comprises a head pose.

5. The computing device of claim 4, wherein the processor is further configured to determine the horizontal component based on an offset of the head pose from a direct forward line of sight.

6. The computing device of claim 1, wherein the processor is further configured to determine changes in the vertical component based on the first facial feature.

7. The computing device of claim 1, wherein the processor is further configured to determine changes in the horizontal component based on the second facial feature.

8. The computing device of claim 1, wherein the processor is further configured to determine that the gaze target is within a scroll zone on the display.

9. The computing device of claim 8, wherein the processor is further configured to determine, based on the second facial feature, that the gaze target is within the scroll zone.

10. The computing device of claim 1, further comprising an infrared light source.

11. The computing device of claim 10, wherein:

the image sensor is further configured to detect the infrared light emitted by the infrared light source, and

the processor is further configured to determine the gaze target based on a reflection, from at least one eye, of the infrared light emitted by the infrared light source.

12. A method of providing user input to a computer device, the method comprising:

determining, with a processor, multiple facial features of a user based on at least one image captured by an image sensor; and

determining, with the processor, a gaze target of the user based on the facial features, the gaze target having a vertical component and a horizontal component, wherein the vertical component is determined based on a first facial feature, and the horizontal component is determined based on a second facial feature.

13. The method of claim 12, wherein the first facial feature comprises an eye lid position, and the processor determines the vertical component based on a detected openness of the eye lid position.

14. The method of claim 12, wherein the second facial feature comprises a head pose, and the processor determines the horizontal component based on an offset of the head pose from a direct forward line of sight.

15. The method of claim 12, wherein the processor further determines changes in the vertical component based on the first facial feature.

16. The method of claim 12, wherein the processor further determines changes in the horizontal component based on the second facial feature.

17. The method of claim 12, wherein the processor further determines, based on the second facial feature, that the gaze target is within a scroll zone on a display device.

18. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions, when executed by one or more processors, perform at least the steps of:

determining multiple facial features of a user based on at least one image captured by an image sensor; and

determining a gaze target of the user based on the facial features, the gaze target having a vertical component and a horizontal component, wherein the vertical component is determined based on a first facial feature, and the horizontal component is determined based on a second facial feature.

19. The non-transitory computer-readable storage medium of claim 18, wherein the first facial feature comprises an eye lid position, and determining the vertical component is based on a detected openness of the eye lid position.

20. The non-transitory computer-readable storage medium of claim 18, wherein the second facial feature comprises a head pose, and determining the horizontal component is based on an offset of the head pose from a direct forward line of sight.