Gestures, Interactions, And Common Ground In a Surface Computing Environment

• US 20150346837A1
• Filed: 08/10/2015
• Published: 12/03/2015
• Est. Priority Date: 12/15/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Morris, Meredith June, Horvitz, Eric J., Jones, F. David, Hodges, Stephen E., Hinckley, Kenneth P., Butler, David Alexander, Sands, Ian M., Russ, V. Kevin, Benko, Hrvoje, LeProwse, Shawn R., Izadi, Shahram, Kunz, William Ben, Wilson, Andrew Daivd
• Granted Patent: US 10,409,381 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04L 67/01 Protocols

Abstract

Aspects relate to detecting gestures that relate to a desired action, wherein the detected gestures are common across users and/or devices within a surface computing environment. Inferred intentions and goals based on context, history, affordances, and objects are employed to interpret gestures. Where there is uncertainty in intention of the gestures for a single device or across multiple devices, independent or coordinated communication of uncertainty or engagement of users through signaling and/or information gathering can occur.

Claims

1. (canceled)

2. A method performed by one or more processors that are configured with executable instructions, the method comprising:

receiving, by a first computing device, a first gesture input from a touch-sensitive input device;

determining that the first gesture input comprises an undefined gesture that is absent from a first set of gestures stored by the first computing device;

receiving, over a network, a second set of gestures from a second computing device;

determining that the first gesture input is included in the second set of gestures;

determining a command associated with the first gesture input based at least in part on the second set of gestures; and

performing the command associated with the first gesture input.

3. The method of claim 2, wherein:

the second computing device comprises a server; and

the network comprises an Internet.

4. The method of claim 2, further comprising:

receiving a second gesture input within a predetermined time period after receiving the first gesture input;

determining that the second gesture input comprises a duplicate gesture based at least in part on the first gesture input, wherein the first gesture input and the second gesture input are associated with a same software application; and

ignoring the duplicate gesture.

5. The method of claim 2, further comprising:

determining that a second gesture input is included in the first set of gestures;

determining a second command corresponding to the second gesture input based at least in part on the first set of gestures or the second set of gestures; and

performing the second command corresponding to the second gesture input.

6. The method of claim 2, wherein receiving, over the network, the second set of gestures from the second computing device comprises:

determining that the second computing device is connected to the network;

sending a request, over the network, to the second computing device to provide the second set of gestures;

receiving the second set of gestures from the second computing device over the network; and

storing the second set of gestures on the first computing device.

7. The method of claim 2, further comprising:

determining that a second gesture is absent from the first set of gestures; and

displaying a prompt to select an second command;

receiving selection of the second command; and

associating the second command with the second gesture.

8. A first computing device comprising:

an input device;

one or more processors;

one or more memory devices to store instructions that are executable by the one or more processors to perform operations comprising;

receiving a first gesture input from the input device;

determining that the first gesture input is absent from a first set of gestures stored in the one or more memory devices;

receiving, over a network, a second set of gestures from a second computing device;

identifying a first command associated with the first gesture input based at least in part on the second set of gestures; and

sending the first command to a software application.

9. The first computing device of claim 8, wherein the second computing device comprises a server.

10. The first computing device of claim 8, wherein the network comprises an Internet.

11. The first computing device of claim 8, the operations further comprising;

receiving a second gesture input;

determining that the second gesture input is included in the first set of gestures;

determining that a second command is associated with the second gesture input; and

performing the second command associated with the second gesture input.

12. The first computing device of claim 8, wherein receiving, over the network, the second set of gestures from the second computing device comprises:

sending a request over the network from the first computing device to the second computing device;

receiving, over the network, the second set of gestures from the second computing device; and

storing the second set of gestures in the one or more memory devices.

13. The first computing device of claim 8, the operations further comprising:

receiving a third gesture input;

determining that the third gesture input is absent from the first set of gestures;

receiving user input selecting a third command; and

associated the third command with the third gesture input.

14. The first computing device of claim 8, the operations further comprising:

tracking a gaze of a user that is using the first computing device; and

identifying, based on the gaze of the user, an item displayed on a display device, wherein sending the first command to the software application causes the software application to perform the first command on the item.

15. A method comprising:

receiving, by a first computing device, a first gesture input from an input device associated with the first computing device;

determining that the first gesture input is absent from a first set of gestures stored on the first computing device;

receiving, over a network, a second set of gestures from a second computing device;

determining that the first gesture input is included in the second set of gestures;

performing a command associated with the first gesture input;

receiving a second gesture input from the input device within a predetermined time period from receiving the first gesture input;

determining that the second gesture input comprises a duplicate gesture, wherein the first gesture input and the second gesture input are associated with a same software application executing on the first computing device; and

ignoring the second gesture input.

16. The method of claim 15, wherein:

the second computing device comprises a server; and

the network comprises an Internet.

17. The method of claim 15, wherein receiving, over the network, the second set of gestures from the second computing device comprises:

determining that the second computing device is connected to the network;

sending a request, over the network, to the second computing device to provide the second set of gestures;

receiving the second set of gestures from the second computing device over the network; and

storing the second set of gestures on the first computing device.

18. The method of claim 15, further comprising:

receiving a third gesture input;

determining that the third gesture input is included in at least one of the first set of gesture or the second set of gestures; and

performing a third command associated with the third gesture input.

19. The method of claim 15, further comprising:

receiving a third gesture input;

determining that the third gesture input is absent from both the first set of gestures and the second set of gestures;

receiving user input selecting a third command; and

associating the third command with the third gesture input.

20. The method of claim 15, further comprising:

receiving a third gesture input;

determining that the third gesture input is absent from both the first set of gestures and the second set of gestures;

receiving, over the network, a third set of gestures from a third computing device;

determining that the third gesture input is included in the third set of gestures; and

performing a third command associated with the third gesture input.

21. The method of claim 15, further comprising:

sending, over the network, the first set of gestures from the first computing device to the second computing device.