Gesture inferred vocabulary bindings

• US 9,292,094 B2
• Filed: 12/16/2011
• Issued: 03/22/2016
• Est. Priority Date: 12/16/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: James, Alex, Pizzo, Michael, Castro, Pablo, Flasko, Michael Justin, Olson, Lance, Clark, Jason, Jayadevan, Siddharth
• Primary Examiner(s): ABDUL-ALI, OMAR R
• Application Number:
  US13/329,165
• Publication Number:
  US 20130159938A1
• Time in Patent Office: 1,558 Days
• Field of Search: 715/863, 715/702
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

The subject disclosure relates to annotating data based on gestures. Gestures include user interaction with a client device or client software. Gestures are tracked and associated with data. In an aspect, client context associated with a gesture is also tracked. The gestures are then employed to determine a global term to associate with the data. In an aspect, a look-up table comprising a pre-defined relationship between gestures and a global term can be employed. In another aspect, an inference component employ context information in conjunction with the tracked gestures to determine a global term to assign to data. After a global term is determined for data based on a gesture, an annotation file for the data can be created associating the data with the global term.

Claims

1. A computer-implemented system, comprising:

a memory and a processor that are respectively configured to store and execute instructions, including instructions organized into;

a tracking component configured to track a gesture and associate the gesture with a data item, wherein the gesture includes a user interaction with a client;

a data store configured to store a look-up table that associates gestures with global terms based on a data-type corresponding to each gesture; and

an analysis component configured to identify a gesture from a set of gestures and employ the data store to determine a global term to assign to the data item based on the gesture and a corresponding data-type.

2. The system of claim 1, the gesture comprising at least one action or command selected from a group consisting of changing a format associated with the data item, playing the data item, moving a mouse over the data item, and performing a call or text message based on the data item.

3. The computer-implemented system of claim 1, wherein the data item includes a telephone number, the tracking component is further configured to track a response to a telephone call based on the telephone number, and the analysis component is configured to determine a global term to assign to the data item based on a response to the telephone call.

4. The computer-implemented system of claim 1, the analysis component configured to identify a first global term to assign to the data item based on the gesture, and a second global term to assign to the data item based on a gesture series, wherein the gesture series includes the gesture and at least one other gesture.

5. The computer-implemented system of claim 1, wherein the gesture comprises initiating a communication with another client.

6. The computer-implemented system of claim 1, wherein the tracking component is further configured to track descriptive metadata associated with the gesture or gesture series, the descriptive metadata comprising at least one of a date of creation of the data item, a creator or author of the data item, a placement of the data item on a computer network, or standards used to create the data item;

the instructions further organized into;

an inference component configured to determine the global term to assign to the data based on the descriptive metadata.

7. A method, comprising:

tracking a gesture, wherein the gesture represents a user interaction with a client;

associating the gesture with data;

identifying the gesture from a set of gestures; and

assigning a global term to the data based on the gesture and on a data type corresponding to the data.

8. The method of claim 7, further comprising:

associating gestures with global terms in a look-up table; and

employing the look-up table in the assigning of the global term to the data.

9. The method of claim 7, further comprising:

associating gestures with global terms in a look-up table based on a client identity; and

employing the look-up table in the assigning of the global term to the data.

10. The method of claim 7, further comprising:

employing a search engine external from the client to find suggested global terms based on the other global term; and

employing the suggested global terms in the assigning of the global term to the data.

11. The method of claim 7, further comprising:

generating an annotation file comprising an assignment of the global term to the data.

12. The method of claim 7, further comprising determining a structure of the data based on the global term.

13. The method of claim 7, further comprising determining a capability of the data based on the global term.

14. The method of claim 7, wherein the assigning of the global term to the data comprises determining the global term to assign to the data based on one or more constraints imparted by the client.

15. The method of claim 7, further comprising:

tracking descriptive metadata associated with the gesture, the descriptive metadata comprising at least one a date of creation of the data, a creator or author of the data, a placement of the data on a computer network, or standards used to create the data.

16. A computer readable storage medium comprising computer executable instructions that, in response to execution, cause a computing system to perform operations, comprising:

tracking a gesture, wherein the gesture includes user interaction with a client;

associating the gesture with data;

determining a global term to assign to the data based on the gesture; and

determining a structure of the data based on the global term.

17. The computer readable storage medium of claim 16, the operations further comprising:

associating gestures with global terms in a look-up table; and

employing the look-up table in the determining the global term to assign to the data based on the gesture.

18. The computer readable storage medium of claim 17, the operations further comprising:

tracking context information associated with the gesture, wherein the determining the global term to assign to the data based on the gesture further comprises determining the global term to assign to the data based on the context information.

19. The computer readable storage medium of claim 16, the global term facilitating interpretation and handling of the data.

20. The computer readable storage medium of claim 16, the operations further comprising:

tracking descriptive metadata associated with the gesture, the descriptive metadata comprising at least one a date of creation of the data, a creator or author of the data, a placement of the data on a computer network, or standards used to create the data.