USER-DEFINED GESTURE SET FOR SURFACE COMPUTING

• US 20100031203A1
• Filed: 06/24/2009
• Published: 02/04/2010
• Est. Priority Date: 08/04/2008
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew David, Morris, Meredith J., Wobbrock, Jacob O.
• Application Number:
  US12/490,335
• Publication Number:
  US 20100031203A1
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The claimed subject matter provides a system and/or a method that facilitates generating an intuitive set of gestures for employment with surface computing. A gesture set creator can prompt two or more users with a potential effect for a portion of displayed data. An interface component can receive at least one surface input from the user in response to the prompted potential effect. A surface detection component can track the surface input utilizing a computer vision-based sensing technique. The gesture set creator collects the surface input from the two or more users in order to identify a user-defined gesture based upon a correlation between the respective surface inputs, wherein the user-defined gesture is defined as an input that initiates the potential effect for the portion of displayed data.

Claims

1. A system that facilitates generating an intuitive set of gestures for employment with computing, comprising:

a gesture set creator that prompts two or more users with a potential effect for a portion of displayed data;

an interface component that receives at least one hand motion from the user in response to the prompted potential effect, the hand motion is an attempted replication of the potential effect;

a vision-based detection component that tracks the hand motion;

the gesture set creator collects the hand motion from the two or more users; and

an intelligent component that forms an inference to identify a user-defined gesture in a data store based upon a correlation between the respective hand motions,

wherein the user-defined gesture is defined as an input that initiates the potential effect for the portion of displayed data.

2. The system of claim 1, wherein the hand motion is a hand gesture, a gesture, a hand interaction, an object interaction, a portion of a hand interacting with a surface, or a corporeal object.

3. The system of claim 1, wherein the potential effect is at least one of a data selection, a data set selection, a group selection, a data move, a data pan, a data rotate, a data cut, a data paste, a data duplicate, a data delete, an accept, a help request, a reject, a menu request, an undo, a data enlarge, a data shrink, a zoom in, a zoom out, an open, a minimize, a next, or a previous.

4. The system of claim 1, wherein the vision based detection component detects the user-defined gesture which executes the potential effect for the portion of displayed data.

5. The system of claim 1, wherein the potential effect is communicated to the user as at least one of a portion of verbal instruction, a portion of video, a portion of audio, a portion of text, or a portion of a graphic.

6. The system of claim 1, further comprising two or more user-defined gestures that are a user-defined gesture set, the user-defined gesture set includes a first select single gesture, a second select single gesture, a select group gesture, a first move gesture, a second move gesture, a pan gesture, a cut gesture, a first paste gesture, a second paste gesture, a rotate gesture, a duplicate gesture, a delete gesture, an accept gesture, a reject gesture, a help gesture, a menu gesture, an undo gesture, a first enlarge/shrink gesture, a second enlarge/shrink gesture, a third enlarge/shrink gesture, a fourth enlarge/shrink gesture, an open gesture, a zoom in/out gesture, a minimize gesture, and a next/previous gesture.

7. The system of claim 1, further comprising a tutorial component that provides a portion of instruction in relation to at least one user-defined gesture.

8. The system of claim 6, wherein the gesture set creator provides a periodic adjustment to the generated user-defined gesture set based at least in part upon historic data collection.

9. The system of claim 1, wherein the inference identifies the most ideal or appropriate user-defined gesture from the hand motion.

10. The system of claim 1, wherein the intelligent component forms the inference with an explicitly and/or implicitly trained schemes including a support vector machine, a neural network, an expert system, a Bayesian belief network, fuzzy logic, or a data fusion engine.

11. A method that facilitates generating an intuitive set of gestures for employment with computing, comprising:

prompting two or more users with a potential effect for a portion of displayed data;

receiving at least one hand motion from the user in response to the prompted potential effect, the hand motion is an attempted replication of the potential effect;

tracking the hand motion with a vision-based detector;

collecting the hand motion from the two or more users; and

forming an inference to identify a user-defined gesture in a data store based upon a correlation between the respective hand motions,

wherein the user-defined gesture is defined as an input that initiates the potential effect for the portion of displayed data.

12. The method of claim 11, wherein the hand motion is a hand gesture, a gesture, a hand interaction, an object interaction, a portion of a hand interacting with a surface, or a corporeal object.

13. The method of claim 11, wherein the potential effect is at least one of a data selection, a data set selection, a group selection, a data move, a data pan, a data rotate, a data cut, a data paste, a data duplicate, a data delete, an accept, a help request, a reject, a menu request, an undo, a data enlarge, a data shrink, a zoom in, a zoom out, an open, a minimize, a next, or a previous.

14. The method of claim 11, further comprising executing the potential effect for the portion of displayed data.

15. The method of claim 11, further comprising communicating the potential effect to the user as at least one of a portion of verbal instruction, a portion of video, a portion of audio, a portion of text, or a portion of a graphic.

16. The method of claim 11, further comprising two or more user-defined gestures that are a user-defined gesture set, the user-defined gesture set includes a first select single gesture, a second select single gesture, a select group gesture, a first move gesture, a second move gesture, a pan gesture, a cut gesture, a first paste gesture, a second paste gesture, a rotate gesture, a duplicate gesture, a delete gesture, an accept gesture, a reject gesture, a help gesture, a menu gesture, an undo gesture, a first enlarge/shrink gesture, a second enlarge/shrink gesture, a third enlarge/shrink gesture, a fourth enlarge/shrink gesture, an open gesture, a zoom in/out gesture, a minimize gesture, and a next/previous gesture.

17. The method of claim 11, further comprising providing a portion of instruction in relation to at least one user-defined gesture.

18. The method of claim 16, further comprising providing a periodic adjustment to the generated user-defined gesture set based at least in part upon historic data collection.

19. The method of claim 11, wherein forming an inference further comprises identifying the most ideal or appropriate user-defined gesture from the hand motion.

20. The method of claim 11, wherein forming an inference further comprises using an explicitly and/or implicitly trained schemes including a support vector machine, a neural network, an expert system, a Bayesian belief network, fuzzy logic, or a data fusion engine.

21. A computer-implemented system that facilitates an intuitive set of gestures for employment with surface computing, comprising:

means for prompting two or more users with a potential effect for a portion of displayed data;

means for receiving at least one hand motion from the user in response to the prompted potential effect, the hand motion is an attempted replication of the potential effect;

means for tracking the hand motion with a vision-based detector;

means for collecting the hand motion from the two or more users; and

means for forming an inference to identify a user-defined gesture in a data store based upon a correlation between the respective hand motions,

wherein the user-defined gesture is defined as an input that initiates the potential effect for the portion of displayed data.

means for receiving at least one surface input from a user directed to a portion of displayed data;

means for tracking the surface input;

means for collecting the surface input from the user in order to identify a user-defined gesture within a user-defined gesture set, the user-defined gesture set includes a first select single gesture, a second select single gesture, a select group gesture, a first move gesture, a second move gesture, a pan gesture, a cut gesture, a first paste gesture, a second paste gesture, a rotate gesture, a duplicate gesture, a delete gesture, an accept gesture, a reject gesture, a help gesture, a menu gesture, an undo gesture, a first enlarge/shrink gesture, a second enlarge/shrink gesture, a third enlarge/shrink gesture, a fourth enlarge/shrink gesture, an open gesture, a zoom in/out gesture, a minimize gesture, and a next/previous gesture; and

means for executing a potential effect for the portion of display triggered by the identified user-defined gesture.

22. The computer-implemented system of claim 21, wherein the hand motion is a hand gesture, a gesture, a hand interaction, an object interaction, a portion of a hand interacting with a surface, or a corporeal object.

23. The computer-implemented system of claim 21, wherein the potential effect is at least one of a data selection, a data set selection, a group selection, a data move, a data pan, a data rotate, a data cut, a data paste, a data duplicate, a data delete, an accept, a help request, a reject, a menu request, an undo, a data enlarge, a data shrink, a zoom in, a zoom out, an open, a minimize, a next, or a previous.

24. The computer-implemented system of claim 21, further comprising means for executing the potential effect for the portion of displayed data.

25. The computer-implemented system of claim 21, further comprising means for communicating the potential effect to the user as at least one of a portion of verbal instruction, a portion of video, a portion of audio, a portion of text, or a portion of a graphic.

26. The computer-implemented system of claim 21, further comprising two or more user-defined gestures that are a user-defined gesture set, the user-defined gesture set includes a first select single gesture, a second select single gesture, a select group gesture, a first move gesture, a second move gesture, a pan gesture, a cut gesture, a first paste gesture, a second paste gesture, a rotate gesture, a duplicate gesture, a delete gesture, an accept gesture, a reject gesture, a help gesture, a menu gesture, an undo gesture, a first enlarge/shrink gesture, a second enlarge/shrink gesture, a third enlarge/shrink gesture, a fourth enlarge/shrink gesture, an open gesture, a zoom in/out gesture, a minimize gesture, and a next/previous gesture.

27. The computer-implemented system of claim 21, further comprising means for providing a portion of instruction in relation to at least one user-defined gesture.

28. The computer-implemented system of claim 26, further comprising means for providing a periodic adjustment to the generated user-defined gesture set based at least in part upon historic data collection.

29. The computer-implemented system of claim 21, wherein the means for forming an inference further comprises means for identifying the most ideal or appropriate user-defined gesture from the hand motion.

30. The computer-implemented system of claim 21, wherein the means for forming an inference further comprises means for using an explicitly and/or implicitly trained schemes including a support vector machine, a neural network, an expert system, a Bayesian belief network, fuzzy logic, or a data fusion engine.