Gesture inferred vocabulary bindings

• US 9,746,932 B2
• Filed: 02/24/2016
• Issued: 08/29/2017
• Est. Priority Date: 12/16/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: James, Alex, Pizzo, Michael, Castro, Pablo, Flasko, Michael Justin, Olson, Lance, Jayadevan, Siddharth, Clark, Jason
• Primary Examiner(s): ABDUL-ALI, OMAR R
• Application Number:
  US15/051,684
• Publication Number:
  US 20160170496A1
• Time in Patent Office: 552 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

The subject disclosure relates to annotating data based on gestures. Gestures include user interaction with a client device or client software. Gestures are tracked and associated with data. In an aspect, client context associated with a gesture is also tracked. The gestures are then employed to determine a global term to associate with the data. In an aspect, a look-up table comprising a pre-defined relationship between gestures and a global term can be employed. In another aspect, an inference component employ context information in conjunction with the tracked gestures to determine a global term to assign to data. After a global term is determined for data based on a gesture, an annotation file for the data can be created associating the data with the global term.

Claims

1. A computing device, comprising:

a memory and a processor that are respectively configured to store and execute instructions, including instructions for causing the computing device to;

track a gesture, wherein the gesture represents a user interaction with the computing device;

associate the gesture with a data item;

identify the gesture from amongst gestures of a set of gestures; and

determine, from a lookup-up table, a global term to assign to the data item based on the gesture and a corresponding data-type, wherein the look-up table associates gestures with global terms based on a data-type corresponding to each gesture.

2. The computing device of claim 1, wherein the instructions are also for causing the computing device to:

determine a structure of the data item based on the global term.

3. The computing device of claim 1, wherein the instructions are also for causing the computing device to:

determine a capability of the data item based on the global term.

4. The computing device of claim 1, wherein the instructions are also for causing the computing device to:

generate an annotation file comprising an assignment of the global term to the data item.

5. The computing device of claim 1, wherein the instructions are also for causing the computing device to:

associate gestures with global terms in the look-up table; and

employ the look-up table in the assigning of the global term to the data item.

6. The computing device of claim 5, wherein the instructions are also for causing the computing device to:

associate gestures with global terms in the look-up table based on a client identity.

7. The computing device of claim 1, the gesture comprising at least one action or command selected from a group consisting of changing a format associated with the data item, playing the data item, moving a pointer over the data item, and performing a call or text message based on the data item.

8. The computing device of claim 1, wherein:

the data item includes a telephone number; and

the instructions are also for causing the computing device to;

track a response to a telephone call based on the telephone number; and

determine a global term to assign to the data item based on a response to the telephone call.

9. The computing device of claim 1, wherein:

the instructions are also for causing the computing device to;

identify a first global term to assign to the data item based on the gesture and a second global term to assign to the data item based on a gesture series, and

the gesture series includes the gesture and at least one other gesture.

10. The computing device of claim 1, wherein the gesture represents a request to initiate communication with another computing device.

11. The computing device of claim 1, wherein:

the instructions are also for causing the computing device to;

track descriptive metadata associated with the gesture or a gesture series; and

determine the global term to assign to the data item based on the descriptive metadata; and

the descriptive metadata comprising at least one of a date of creation of the data item, a creator or author of the data item, a placement of the data item on a computer network, or standards used to create the data item.

12. The computing device of claim 1, wherein the instructions are also for causing the computing device to:

employ a search engine external from the computing device to find suggested global terms based on the global term; and

employ the suggested global terms in the assigning of the global term to the data item.

13. The computing device of claim 1, wherein the instructions are also for causing the computing device to:

track context information associated with the gesture, wherein the determining the global term to assign to the data item further comprises determining the global term to assign to the data item based on the context information.

14. A method, comprising:

tracking a gesture, wherein the gesture represents a user interaction with a computing device;

identifying the gesture from amongst a plurality of gestures;

determining, based at least in part on the gesture, a global term to assign to a data item associated with the gesture; and

assigning the global term to the data item associated with the gesture.

15. The method of claim 14, wherein the determining of the global term is based on use of a look-up table that correlates global terms to data types.

16. The method of claim 14, further comprising:

employing a search engine external from the computing device to find suggested global terms; and

employing the suggested global terms in the assigning of the global term to the data item.

17. The method of claim 14, further comprising determining a structure of the data item based on the global term.

18. A computing device, comprising:

a memory and a processor that are respectively configured to store and execute instructions, including instructions for causing the computing device to perform operations, the operations comprising;

tracking a gesture, wherein the gesture represents a user interaction with the computing device;

identifying the gesture from amongst a plurality of gestures;

determining, based at least in part on the gesture, a global term to assign to a data item associated with the gesture; and

assigning the global term to the data item associated with the gesture.

19. The computing device of claim 18, wherein the operations further comprise:

tracking descriptive metadata associated with the gesture, the descriptive metadata comprising at least one a date of creation of the data item, a creator or author of the data item, a placement of the data on a computer item network, or standards used to create the data item.

20. The computing device of claim 18, wherein the determining the global term to assign to the data item is based on one or more constraints imparted by the client.