Systems and methods of eye tracking control

• US 9,829,971 B2
• Filed: 01/20/2014
• Issued: 11/28/2017
• Est. Priority Date: 01/21/2013
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: San Agustin Lopez, Javier, Sztuk, Sebastian, Tall, Martin Henrik
• Primary Examiner(s): Ngo, Tony N
• Application Number:
  US14/159,168
• Publication Number:
  US 20140204029A1
• Time in Patent Office: 1,408 Days
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Methods and systems to facilitate eye tracking control are provided. A user input is received at a computing device. Point of regard information associated with a user of the computing device is determined while the user input is being received. The point of regard information indicates a location on a display of the computing device at which the user is looking. An operation associated with a display object identified based on the point of regard information is performed when receipt of the user input is determined to have terminated.

Claims

1. A method, comprising:

receiving a user touch input at a computing device;

while continuing to receive the user touch input;

periodically determining, by a processor and at a plurality of time instances while the user touch input is being continuously received, point of regard information associated with a user of the computing device, the point of regard information comprising a plurality of point of regard measurements obtained during the plurality of time instances and indicating a location on a display of the computing device at which the user is looking during continuous reception of the user touch input; and

activating, in response to the user touch input, an eye control mode at the computing device, the eye control mode providing a plurality of display objects on the display while the user touch input is being received, each display object positioned at a different location within the display and associated with a respective operation capable of being performed based on a point of regard measurement of the plurality of point of regard measurements obtained at a time when the receipt of the user touch input has terminated;

determining whether receipt of the user touch input has terminated; and

performing an operation associated with a display object of the plurality of display objects identified by the point of regard at a moment the user touch input selecting the display object has terminated.

2. The method of claim 1, wherein the user touch input is any one of a touch sensor input, a push button input, a mouse input, a keyboard input, or a biometric signal input.

3. The method of claim 1, wherein the operation associated with the display object includes moving or selecting the display object.

4. The method of claim 1, wherein the operation associated with the display object includes launching an application.

5. The method of claim 1, further comprising:

terminating the eye control mode when the receipt of the user touch input has terminated.

6. A non-transitory machine-readable storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving a user touch input at a computing device;

while continuing to receive the user touch input;

periodically determining, by the one or more processors and at a plurality of time instances while the user touch input is being continuously received, point of regard information associated with a user of the computing device, the point of regard information comprising a plurality of point of regard measurements obtained during the plurality of time instances and indicating a location on a display of the computing device at which the user is looking during continuous reception of the user touch input; and

activating, in response to the user touch input, an eye control mode at the computing device, the eye control mode providing a plurality of display objects on the display while the user touch input is being received, each display object positioned at a different location within the display and associated with a respective operation capable of being performed based on a point of regard measurement of the plurality of point of regard measurements obtained at a time when the receipt of the user touch input has terminated;

determining whether receipt of the user touch input has terminated; and

performing an operation associated with a display object of the plurality of display objects identified by the point of regard at a moment the user touch input selecting the display object has terminated.

7. The non-transitory machine-readable storage medium of claim 6, wherein the user touch input is any one of a touch sensor input, a push button input, a mouse input, a keyboard input, or a biometric signal input.

8. The non-transitory machine-readable storage medium of claim 6, wherein the operation associated with the display object includes moving or selecting the display object.

9. The non-transitory machine-readable storage medium of claim 6, wherein the operation associated with the display object includes launching an application.

10. A computing device, comprising:

a memory having instructions embodied thereon;

one or more processors configured by the instructions to perform operations comprising;

receiving a user touch input at the computing device;

while continuing to receive the user touch input;

periodically determining at a plurality of time instances while the user touch input is being continuously received, point of regard information associated with a user of the computing device, the point of regard information comprising a plurality of point of regard measurements obtained during the plurality of time instances and indicating a location on a display of the computing device at which the user is looking during continuous reception of the user touch input; and

activating, in response to the user touch input, an eye control mode at the computing device, the eye control mode providing a plurality of display objects on the display while the user touch input is being received, each display object positioned at a different location within the display and associated with a respective operation capable of being performed based on a point of regard measurement of the plurality of point of regard measurements obtained at a time when the receipt of the user touch input has terminated;

determining whether receipt of the user touch input has terminated; and

performing an operation associated with a display object of the plurality of display objects identified by the point of regard at a moment the user touch input selecting the display object has terminated.

11. The computing device of claim 10, wherein the user touch input is any one of a touch sensor input, a push button input, a mouse input, a keyboard input, or a biometric signal input.

12. The computing device of claim 10, wherein the operation associated with the display object includes moving or selecting the display object.

13. The computing device of claim 10, wherein the operation associated with the display object includes launching an application.

14. The computing device of claim 10, wherein the operations further comprise terminating the eye control mode when the receipt of the user touch input has terminated.