Control of a device using gestures

• US 8,875,059 B2
• Filed: 09/30/2011
• Issued: 10/28/2014
• Est. Priority Date: 05/18/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Santoro, David T.
• Primary Examiner(s): Kim, Matt
• Assistant Examiner(s): Nguyen, Tuan S
• Application Number:
  US13/250,640
• Publication Number:
  US 20120297348A1
• Time in Patent Office: 1,124 Days
• Field of Search: 715/863, 705/27, 345/156, 707/769
• US Class Current: 715/863
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

In an operating system running on a processing device, detecting a gesture input via a user interface; identifying an operating system operation that corresponds to the gesture; performing the operating system operation; identifying an application running on the operating system that has subscribed to gesture input; and passing data corresponding to the gesture to the application for use by the application.

Claims

1. A method, comprising:

displaying, by a computing device, a first block of textual content within a presentation of a first application running on the computing device;

receiving, by the computing device, an indication of a first gesture performed on a touch sensitive display of the computing device;

identifying, by the computing device, that the first gesture selected at least a portion of the first block of textual content that is displayed within the presentation of the first application;

receiving, by the computing device, an indication of a second gesture performed on the touch sensitive display, the second gesture performed after the first gesture;

determining that the second gesture corresponds to a first pre-defined gesture from among a plurality of pre-defined gestures;

in response to determining that the second gesture corresponds to the first pre-defined gesture;

(i) identifying that the first pre-defined gesture is associated with (a) a second application on the computing device, from among a plurality of applications on the computing device, and (b) a first operation of the second application,

(ii) providing, to the second application, the at least portion of the first block of textual content that was displayed within the presentation of the first application and that was selected by the first gesture, and

(iii) executing the second application on the computing device, including performing the first operation of the second application using the at least portion of the first block of textual content that was displayed within the presentation of the first application and that was selected by the first gesture;

displaying, by the computing device, a second block of textual content within a presentation of a third application running on the computing device;

receiving, by the computing device, an indication of a third gesture performed on the touch sensitive display of the computing device;

identifying, by the computing device, that the third gestured selected at least a portion of the second block of textual content that is displayed within the presentation of the third application;

receiving, by the computing device, an indication of a fourth gesture performed on the touch sensitive display, the fourth gesture performed after the third gesture;

determining that the fourth gesture corresponds to a second pre-defined gesture from among the plurality of pre-defined gestures, the second pre-defined gesture being different than the first pre-defined gesture;

in response to determining that the fourth gesture corresponds to the second pre-defined gesture;

(i) identifying that the second pre-defined gesture is associated with (a) the second application on the computing device, and (b) a second operation of the second application that is different than the first operation of the second application,

(ii) providing, to the second application, the at least portion of the second block of textual content that was displayed within the presentation of the third application and that was selected by the third gesture, and

(iii) executing the second application on the computing device including performing the second operation of the second application that is different than the first operation, using the at least portion of the second block of textual content that was displayed within the presentation of the third application and that was selected by the third gesture.

2. The method of claim 1, wherein the first operation of the second application or the second operation of the second application comprises at least one of opening a note, opening a message, pasting data, moving data, and adding a recipient to an existing message.

3. The method of claim 1, wherein determining that the second gesture corresponds to the first pre-defined gesture comprises determining that a path traced by the second gesture matches a path associated with the first pre-defined gesture.

4. The method of claim 1, wherein executing the second application on the computing device including performing the first operation of the second application comprises replacing a display of the first application on the touch sensitive display of the computing device with a display of the second application on the touch sensitive display of the computing device.

5. The method of claim 1, wherein determining that the second gesture corresponds to the first pre-defined gesture from among the plurality of pre-defined gestures comprises accessing a data repository on the computing device that includes information that correlates each of the plurality of pre-defined gestures with a respective application on the computing device.

6. The method of claim 1, further comprising:

receiving, from a fourth application on the computing device, a request to subscribe to gesture-based controls on the computing device;

identifying, in response to receiving the request to subscribe to gesture-based controls, (i) a particular gesture associated with the fourth application, and (ii) an operation of the fourth application; and

storing information that correlates the particular gesture associated with the fourth application and the operation of the fourth application in a data repository on the computing device.

7. The method of claim 1, wherein identifying that the first gesture selected the at least portion of the first block of textual content comprises determining that the first gesture highlighted text displayed within the presentation of the first application.

8. The method of claim 1, wherein a path traced by the second gesture comprises a path for an alphanumeric symbol.

9. One or more non-transitory computer-readable storage devices including instructions that, when executed by one or more processors, cause performance of operations, the operations comprising:

displaying, by a computing device, a first block of textual content within a presentation of a first application running on the computing device;

receiving, by the computing device, an indication of a first gesture performed on a touch sensitive display of the computing device;

identifying, by the computing device, that the first gesture selected at least a portion of the first block of textual content that is displayed within the presentation of the first application;

receiving, by the computing device, an indication of a second gesture performed on the touch sensitive display, the second gesture performed after the first gesture;

determining that the second gesture corresponds to a first pre-defined gesture from among a plurality of pre-defined gestures;

in response to determining that the second gesture corresponds to the first pre-defined gesture;

(i) identifying that the first pre-defined gesture is associated with (a) a second application on the computing device, from among a plurality of applications on the computing device, and (b) a first operation of the second application,

(ii) providing, to the second application, the at least portion of the first block of textual content that was displayed within the presentation of the first application and that was selected by the first gesture, and

(iii) executing the second application on the computing device including performing the first operation of the second application using the at least portion of the first block of textual content that was displayed within the presentation of the first application and that was selected by the first gesture;

displaying, by the computing device, a second block of textual content within a presentation of a third application running on the computing device;

receiving, by the computing device, an indication of a third gesture performed on the touch sensitive display of the computing device;

identifying, by the computing device, that the third gestured selected at least a portion of the second block of textual content that is displayed within the presentation of the third application;

receiving, by the computing device, an indication of a fourth gesture performed on the touch sensitive display, the fourth gesture performed after the third gesture;

determining that the fourth gesture corresponds to a second pre-defined gesture from among the plurality of pre-defined gestures, the second pre-defined gesture being different than the first pre-defined gesture;

in response to determining that the fourth gesture corresponds to the second pre-defined gesture;

(i) identifying that the second pre-defined gesture is associated with (a) the second application on the computing device, and (b) a second operation of the second application that is different than the first operation of the second application,

(ii) providing, to the second application, the at least portion of the second block of textual content that was displayed within the presentation of the third application and that was selected by the third gesture, and

(iii) executing the second application on the computing device including performing the second operation of the second application that is different than the first operation using the at least portion of the second block of textual content that was displayed within the presentation of the third application and that was selected by the third gesture.

10. The one or more computer-readable storage devices of claim 9, wherein:

each of one or more applications from among the plurality of applications on the computing device is associated with multiple gestures, each of the multiple gestures being associated with a different operation of the respective application.

11. The one or more computer-readable storage devices of claim 9, wherein the first operation of the second application or the second operation of the second application comprises at least one of opening a note, opening a message, pasting data, moving data, and adding a recipient to an existing message.

12. The one or more computer-readable storage devices of claim 9, wherein determining that the second gesture corresponds to the first pre-defined gesture comprises determining that a path traced by the second gesture matches a path associated with the first pre-defined gesture.

13. The one or more computer-readable storage devices of claim 9, wherein executing the second application on the computing device including performing the first operation of the second application comprises replacing a display of the first application on the touch sensitive display of the computing device with a display of the second application on the touch sensitive display of the computing device.

14. The one or more computer-readable storage devices of claim 9, wherein determining that the second gesture corresponds to the first pre-defined gesture from among the plurality of pre-defined gestures comprises accessing a data repository on the computing device that includes information that correlates each of the plurality of pre-defined gestures with a respective application on the computing device.

15. The one or more computer-readable storage devices of claim 9, wherein the operations further comprise:

receiving, from a fourth application on the computing device, a request to subscribe to gesture-based controls on the computing device;

identifying, in response to receiving the request to subscribe to gesture-based controls, (i) a particular gesture associated with the fourth application, and (ii) an operation of the fourth application; and

storing information that correlates the particular gesture associated with the fourth application and the operation of the fourth application in a data repository on the computing device.

16. The one or more computer-readable storage devices of claim 9, wherein identifying that the first gesture selected the at least portion of the first block of textual content comprises determining that the first gesture highlighted text displayed within the presentation of the first application.

17. The one or more computer-readable storage devices of claim 9, wherein a path traced by the second gesture comprises a path for an alphanumeric symbol.

18. A computing system comprising:

one or more computer processors;

a touch sensitive display; and

one or more non-transitory computer-readable storage devices including instructions that, when executed by the one or more computer processors, cause performance of operations, the operations comprising;

displaying a first block of textual content within a presentation of a first application running on the computing system;

receiving an indication of a first gesture performed on the touch sensitive display;

identifying that the first gesture selected at least a portion of the first block of textual content that is displayed within the presentation of the first application running on the computing system;

receiving an indication of a second gesture performed on the touch sensitive display, the second gesture performed after the first gesture;

determining that the second gesture corresponds to a first pre-defined gesture from among a plurality of pre-defined gestures;

in response to determining that the second gesture corresponds to the first pre-defined gesture;

(i) identifying that the first pre-defined gesture is associated with (a) a second application on the computing system, from among a plurality of applications on the computing system, and (b) a first operation of the second application;

(ii) providing, to the second application, the at least portion of the first block of textual content that was displayed within the presentation of the first application and that was selected by the first gesture and

(iii) executing the second application with the computing system including performing the first operation of the second application using the at least portion of the first block of textual content that was displayed within the presentation of the first application and that was selected by the first gesture

displaying a second block of textual content within a presentation of a third application running on the computing system;

receiving an indication of a third gesture performed on the touch sensitive display;

identifying that the third gesture selected at least a portion of a second block of textual content that is displayed within the presentation of the third application running on the computing system;

receiving an indication of a fourth gesture performed on the touch sensitive display, the fourth gesture performed after the third gesture;

determining that the fourth gesture corresponds to a second pre-defined gesture from among the plurality of pre-defined gestures;

in response to determining that the fourth gesture corresponds to the second pre-defined gesture;

(i) identifying that the second pre-defined gesture is associated with (a) the second application on the computing system, and (b) a second operation of the second application that is different than the first operation of the second application,

(ii) providing, to the second application, the at least portion of the second block of textual content that was displayed within the presentation of the third application and that was selected by the third gesture, and

(iii) executing the second application with the computing system including performing the second operation of the second application that is different than the first operation of the second application using the at least portion of the second block of textual content that was displayed within the presentation of the third application and that was selected by the third gesture.

19. The system of claim 18, wherein the first operation of the second application or the second operation of the second application comprises at least one of opening a note, opening a message, pasting data, moving data, and adding a recipient to an existing message.