Control of a device using gestures

• US 8,793,624 B2
• Filed: 05/18/2011
• Issued: 07/29/2014
• Est. Priority Date: 05/18/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Santoro, David T.
• Primary Examiner(s): Kim, Matt
• Assistant Examiner(s): Nguyen, Tuan S
• Application Number:
  US13/110,358
• Publication Number:
  US 20120293421A1
• Time in Patent Office: 1,168 Days
• Field of Search: None
• US Class Current: 715/863
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

In an operating system running on a processing device, detecting a gesture input via a user interface; identifying an operating system operation that corresponds to the gesture; performing the operating system operation; identifying an application running on the operating system that has subscribed to gesture input; and passing data corresponding to the gesture to the application for use by the application.

Claims

1. A computer-implemented method comprising:

receiving, by a computing device, an indication that a first gesture has been performed on a touch sensitive display of the computing device;

identifying, by the computing device, a first application that corresponds to the first gesture;

receiving, by the computing device, an indication that a second gesture has been performed on the touch sensitive display of the computing device after the first gesture has been performed;

identifying, by the computing device in response to receiving the indication that the second gesture has been performed, a first operation that corresponds to the second gesture and that is one of multiple operations assigned to the first application, wherein each of the multiple operations assigned to the first application corresponds to a respective gesture; and

in response to identifying the first application that corresponds to the first gesture and identifying the first operation that corresponds to the second gesture, executing the first application and performing the first operation in the first application.

2. The computer-implemented method of claim 1, further comprising displaying, with the touch sensitive display of the computing device, (i) a representation of the first gesture that shows a path traced by the first gesture on the touch sensitive display, and (ii) a representation of the second gesture that shows a path traced by the second gesture on the touch sensitive display.

3. The computer-implemented method of claim 2, further comprising:

maintaining the display of the representation of the first gesture while displaying the representation of the second gesture; and

clearing the respective displays of the representation of the first gesture and the representation of the second gesture in conjunction with executing the first application and performing the first operation in the first application.

4. The computer-implemented method of claim 1, wherein the first gesture, the second gesture, or both, are performed on the touch sensitive display of the computing device over a grid of displayed graphical representations of applications on the computing device.

5. The computer-implemented method of claim 1, wherein the first gesture, the second gesture, or both, are performed on the touch sensitive display of the computing device within a second application that is different than the first application.

6. The computer-implemented method of claim 1, wherein:

the first application has subscribed to gesture-based control on the computing device, and

identifying the first application comprises accessing, by an operating system of the computing device, a data repository that correlates gestures with applications that have subscribed to the gesture-based control on the computing device, and identifying the first application from the data repository.

7. The computer-implemented method of claim 6, wherein the data repository further correlates, for each application in the data repository, respective gestures with one or more operations for the application, and wherein identifying the first operation comprises identifying, by the operating system of the computing device, the first operation from the data repository.

8. The computer-implemented method of claim 1, wherein at least one of the first gesture and the second gesture comprises an alphanumeric symbol that identifies the first application or the first operation, respectively.

9. The computer-implemented method of claim 1, wherein executing the first application and performing the first operation in the first application comprises opening the first application and performing an operation in the first application that would not be performed without additional user input if the first application had been opened in response to selection of a graphical representation of the first application from a menu on the touch sensitive display of the computing device.

10. The computer-implemented method of claim 1, wherein performing the first operation in the first application comprises passing data indicative of the first operation to the first application using an application programming interface (API) of an operating system of the computing device.

11. The computer-implemented method of claim 1, further comprising:

receiving, by the computing device, an indication that a third gesture has been performed on the touch sensitive display of the computing device;

identifying, by the computing device, a second application that corresponds to the third gesture, wherein the second application is different than the first application;

receiving, by the computing device, a second indication that the second gesture has been performed on the touch sensitive display of the computing device after the third gesture has been performed;

identifying, by the computing device in response to receiving the second indication that the second gesture has been performed, a second operation that corresponds to the second gesture and that is one of multiple operations assigned to the second application, wherein each of the multiple operations assigned to the second application corresponds to a respective gesture, and wherein the second operation is different than the first operation; and

in response to identifying the second application that corresponds to the third gesture and identifying the second operation that corresponds to the second gesture, executing the second application and performing the second operation in the second application.

12. One or more non-transitory machine-readable hardware devices having instructions stored thereon that, when executed by one or more computers, cause performance of operations comprising:

receiving, by one or more computers, an indication that a first gesture has been performed on a touch sensitive display of the one or more computers;

identifying, by the one or more computers, a first application that corresponds to the first gesture;

receiving, by the one or more computers, an indication that a second gesture has been performed on the touch sensitive display of the one or more computers after the first gesture has been performed;

identifying, by the one or more computers in response to receiving the indication that the second gesture has been performed, a first operation that corresponds to the second gesture and that is one of multiple operations assigned to the first application, wherein each of the multiple operations assigned to the first application corresponds to a respective gesture; and

in response to identifying the first application that corresponds to the first gesture and identifying the first operation that corresponds to the second gesture, executing the first application and performing the first operation in the first application.

13. The one or more machine-readable hardware devices of claim 12, wherein the operations further comprise displaying, with the touch sensitive display of the one or more computers, (i) a representation of the first gesture that shows a path traced by the first gesture on the touch sensitive display, and (ii) a representation of the second gesture that shows a path traced by the second gesture on the touch sensitive display.

14. The one or more machine-readable hardware devices of claim 13, wherein the operations further comprise:

maintaining the display of the representation of the first gesture while displaying the representation of the second gesture; and

clearing the respective displays of the representation of the first gesture and the representation of the second gesture in conjunction with executing the first application and performing the first operation in the first application.

15. The one or more machine-readable hardware devices of claim 12, wherein the first gesture, the second gesture, or both, are performed on the touch sensitive display of the one or more computers over a grid of displayed graphical representations of applications on the one or more computers.

16. The one or more machine-readable hardware devices of claim 12, wherein the first gesture, the second gesture, or both, are performed on the touch sensitive display of the one or more computers within a second application that is different than the first application.

17. The one or more machine-readable hardware devices of claim 12, wherein:

the first application has subscribed to gesture-based control on the one or more computers, and

identifying the first application comprises accessing, by an operating system of the one or more computers, a data repository that correlates gestures with applications that have subscribed to the gesture-based control on the one or more computers, and identifying the first application from the data repository.

18. The one or more machine-readable hardware devices of claim 17, wherein the data repository further correlates, for each application in the data repository, respective gestures with one or more operations for the application, and wherein identifying the first operation comprises identifying, by the operating system of the one or more computers, the first operation from the data repository.

19. The one or more machine-readable hardware devices of claim 12, wherein at least one of the first gesture and the second gesture comprises an alphanumeric symbol that identifies the first application or the first operation, respectively.

20. The one or more machine-readable hardware devices of claim 12, wherein executing the first application and performing the first operation in the first application comprises opening the first application and performing an operation in the first application that would not be performed without additional user input if the first application had been opened in response to selection of a graphical representation of the first application from a menu on the touch sensitive display of the one or more computers.

21. The one or more machine-readable hardware devices of claim 12, wherein performing the first operation in the first application comprises passing data indicative of the first operation to the first application using an application programming interface (API) of an operating system of the one or more computers.

22. The one or more machine-readable hardware devices of claim 12, wherein the operations further comprise:

receiving, by the one or more computers, an indication that a third gesture has been performed on the touch sensitive display of the one or more computers;

identifying, by the one or more computers, a second application that corresponds to the third gesture, wherein the second application is different than the first application;

receiving, by the one or more computers, a second indication that the second gesture has been performed on the touch sensitive display of the one or more computers after the third gesture has been performed;

identifying, by the one or more computers in response to receiving the second indication that the second gesture has been performed, a second operation that corresponds to the second gesture and that is one of multiple operations assigned to the second application, wherein each of the multiple operations assigned to the second application corresponds to a respective gesture, and wherein the second operation is different than the first operation; and

in response to identifying the second application that corresponds to the third gesture and identifying the second operation that corresponds to the second gesture, executing the second application and performing the second operation in the second application.