Methods and systems for gesture based switch for machine control

• US 9,965,042 B2
• Filed: 03/30/2015
• Issued: 05/08/2018
• Est. Priority Date: 03/30/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Jules, Anthony Sean, Edsinger, Aaron
• Primary Examiner(s): MARINELLI, PATRICK
• Application Number:
  US14/672,750
• Publication Number:
  US 20160291697A1
• Time in Patent Office: 1,135 Days
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 2221/2133 Verifying human interaction, e.g., Captcha
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Examples described may relate to methods and systems for gesture based switch for machine control. Some machines may be operated by computing devices that include touchscreens, and a graphical user interface (GUI) is provided for enabling control of the machines. Within examples, a disconnect switch can be incorporated into the GUI. As one example, to initiate operation of a machine, such as a robotic device, a user may be required to contact the touchscreen at a location and then trace out a pattern. To enable continued operation of the machine, the GUI may require the user to maintain contact with the touchscreen at a “goal” position. If contact with the touchscreen is broken (e.g., for more than a threshold amount of time), the machine operation may be halted, and the process to initiate operation can be performed again on the touchscreen to cause the machine to resume operation.

Claims

1. A method comprising:

displaying a pattern on a touchscreen of a computing device that includes a graphical user interface (GUI) for enabling control of an operation of a machine;

receiving an input gesture on the touchscreen of the computing device, the input gesture including movement along the pattern on the touchscreen from a first location of the pattern to a second location of the pattern; and

the GUI enabling the computing device to control the operation of the machine based on;

an authentication of the input gesture, and

detecting persistent contact within a defined location on the touchscreen, wherein the defined location includes the second location,

wherein the computing device is configured to display on the touchscreen, based on the authentication of the input gesture, boundaries for input of contact that qualifies as the persistent contact within the defined location,

wherein an operating system of the computing device is configured to;

interpret touch input for the persistent contact within the defined location,

enable the computing device to control the operation of the machine,

provide control of the operation of the machine to an application of the computing device in response to the authentication of the input gesture and detecting the persistent contact within the defined location, and

based on a lack of detection of the persistent contact within the defined location, disable control of the operation of the machine by the computing device by locking the touchscreen and returning the touchscreen to an initial state requiring the authentication of the input gesture, and

wherein the application of the computing device is configured to;

distinguish the touch input for the persistent contact within the defined location from other touch inputs representing contact outside of the defined location, and

using the other touch inputs for controlling the operation of the machine while detecting the touch input for persistent contact within the defined location.

2. The method of claim 1, wherein the input gesture includes input of a passcode on the touchscreen using swipe motions, followed by the movement along the pattern on the touchscreen.

3. The method of claim 1, further comprising:

determining that the input gesture concluded within the defined location on the touchscreen and continual contact within the defined location on the touchscreen is detected without interruption of contact.

4. The method of claim 1, further comprising:

based on the lack of detection of the persistent contact within the defined location of the touchscreen, pausing the operation of the machine.

5. The method of claim 1, further comprising:

determining that the touch input within the defined location corresponding to the persistent contact is lacking for more than a threshold amount of time; and

responsively pausing the operation of the machine.

6. The method of claim 5, further comprising, to resume the operation of the machine:

receiving a second input gesture on the touchscreen of the computing device; and

resuming the operation of the machine based on the second input gesture being authenticated and on persistent contact at the defined location being detected.

7. The method of claim 1, further comprising:

based on the lack of detection of the persistent contact within the defined location, causing one or more actions to occur including changing a display on the computing device to a locked state, displaying a message indicating the operation of the machine is paused, and providing at the computing device a vibration or audio alert to indicate the operation of machine has paused.

8. The method of claim 1, wherein the GUI enabling the computing device to control the operation of the machine is further based on the computing device and the machine exchanging communications at regular intervals to maintain a heartbeat.

9. The method of claim 1, wherein the other touch inputs are indicative of instructions for the machine to perform one or more actions.

10. A non-transitory computer readable medium having stored thereon instructions that, upon execution by a computing device, cause the computing device to perform functions comprising:

displaying a pattern on a touchscreen of a computing device that includes a graphical user interface (GUI) for enabling control of an operation of a machine;

receiving an input gesture on the touchscreen of the computing device, the input gesture including movement along the pattern on the touchscreen from a first location of the pattern to a second location of the pattern; and

the GUI enabling the computing device to control the operation of the machine based on;

an authentication of the input gesture, and

detecting persistent contact within a defined location on the touchscreen, wherein the defined location includes the second location,

wherein the computing device is configured to display on the touchscreen, based on the authentication of the input gesture, boundaries for input of contact that qualifies as the persistent contact within the defined location,

wherein an operating system of the computing device is configured to;

interpret touch input for the persistent contact within the defined location,

enable the computing device to control the operation of the machine,

provide control of the operation of the machine to an application of the computing device in response to the authentication of the input gesture and detecting the persistent contact within the defined location, and

based on a lack of detection of the persistent contact within the defined location, disable control of the operation of the machine by the computing device by locking the touchscreen and returning the touchscreen to an initial state requiring the authentication of the input gesture, and

wherein the application of the computing device is configured to;

distinguish the touch input for the persistent contact within the defined location from other touch inputs representing contact outside of the defined location, and

using the other touch inputs for controlling the operation of the machine while detecting the touch input for persistent contact within the defined location.

11. The non-transitory computer readable medium of claim 10, wherein the functions further comprise:

based on the lack of detection of the persistent contact within the defined location on the touchscreen, sending instructions to the machine to perform a specific function.

12. The non-transitory computer readable medium of claim 10, wherein the functions further comprise:

determining that the touch input for the persistent contact is lacking for about 1-2 seconds; and

responsively pausing the operation of the machine.

13. The non-transitory computer readable medium of claim 10, wherein the GUI enabling the computing device to control the operation of the machine is further based on the computing device and the machine exchanging communications at regular intervals to maintain a heartbeat.

14. The non-transitory computer readable medium of claim 10, wherein the other touch inputs are indicative of instructions for the machine to perform one or more actions.

15. A system comprising:

a computing device including a touchscreen, an operating system, an application, and a graphical user interface (GUI) for enabling control of an operation of a machine;

one or more processors; and

data storage including instructions executable by the one or more processors to cause the system to perform functions comprising;

displaying a pattern on the touchscreen;

receiving an input gesture on the touchscreen of the computing device, the input gesture including movement along the pattern on the touchscreen from a first location of the pattern to a second location of the pattern; and

the GUI enabling the computing device to control of the operation of the machine based on;

an authentication of the input gesture, and

detecting persistent contact within a defined location on the touchscreen, wherein the defined location includes the second location,

wherein the computing device is configured to display on the touchscreen, based on the authentication of the input gesture, boundaries for input of contact that qualifies as the persistent contact within the defined location,

wherein the operating system of the computing device is configured to;

interpret touch input for the persistent contact within the defined location,

enable the computing device to control the operation of the machine,

provide control of the operation of the machine to the application in response to the authentication of the input gesture and detecting the persistent contact within the defined location, and

based on a lack of detection of the persistent contact within the defined location, disable control of the operation of the machine by the computing device by locking the touchscreen and returning the touchscreen to an initial state requiring the authentication of the input gesture, and

wherein the application of the computing device is configured to;

distinguish the touch input for the persistent contact within the defined location from other touch inputs representing contact outside of the defined location, and

using the other touch inputs for controlling the operation of the machine while detecting the touch input for persistent contact within the defined location.

16. The system of claim 15, wherein the functions further comprise, to resume the operation of the machine:

receiving a second input gesture on the touchscreen; and

resuming the operation of the machine based on the second input gesture being authenticated and on persistent contact at the defined location being detected.

17. The system of claim 15, wherein the functions further comprise:

based on the authentication of the input gesture, displaying on the touchscreen an icon informing of the defined location being a deadman switch contact area.

18. The system of claim 15, wherein the GUI enabling the computing device to control the operation of the machine is further based on the computing device and the machine exchanging communications at regular intervals to maintain a heartbeat.

19. The system of claim 15, wherein the other touch inputs are indicative of instructions for the machine to perform one or more actions.