GESTURE RECOGNITION INTERACTIVE FEEDBACK

• US 20080178126A1
• Filed: 01/24/2007
• Published: 07/24/2008
• Est. Priority Date: 01/24/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kennedy, Kevin A., Bilow, Charles R., Beeck, Langdon W.
• Granted Patent: US 7,770,136 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G09B 19/003 Repetitive work cycles; Sequence of movements
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied

Abstract

Example embodiments provide for (i) suggesting proper gesture sequences based on a user'"'"'s initial gesture input; and/or (ii) providing visual or other gesture example(s), which may alternatively be compared with the user'"'"'s mimic thereof For example, a user may input a gesture into a touch/display surface. Based on the user'"'"'s movement, recognized or proper application gesture commands are suggested. In other embodiments, a tutorial session may be presented to the user for teaching or educating the user on numerous types of gestures. As such, a visual representation of a gesture example may be presented to the user, and feedback provided based on the user'"'"'s mimicking thereof

Claims

1. At a gesture recognition tutorial system including a motion-sensitive display surface capable of recognizing various forms of user interaction with the surface, a method of suggesting recognized application gesture commands based on the user interaction with the surface in order to give the user various options and/or instructions on how to appropriately use the application for the motion-sensitive display surface, the method comprising:

receiving at a motion-sensitive display surface a user-performed gesture, which includes user movement of an object over the surface that recognizes such user interaction therewith;

based on the received user-performed gesture, identifying one or more gesture commands recognized by a program of the motion-sensitive display surface, wherein the one or more gesture commands recognized by the program represent actual commands of the program; and

displaying visual feedback on the motion-sensitive display surface representing the one or more gesture commands recognized by the program in order to aid the user in understanding appropriate various defined gestures representing actual program commands.

2. The method of claim 1, wherein the user movement includes one or more of tapping, touching, touching and holding for a specified time, sliding, waving, flicking or dragging.

3. The method of claim 1, wherein the object is one or more of a bodily appendage, writing utensil, stylus, dice, paint brushes, cellular telephones, coins, poker chips, or radio frequency identification (RFID) tagged object.

4. The method of claim 1, wherein the user-performed gesture closely resembles one or more gestures commands for one or more programs.

5. The method of claim 4, wherein a pre-defined degree of variance is used to identify the gesture command based on the difference between the user-performed gesture and a recognized gesture.

6. The method of claim 1, further comprising interacting with the user to determine whether the displayed gesture was the gesture intended by the user'"'"'s movement input.

7. The method of claim 1, wherein the displayed visual feedback comprises textual information.

8. The method of claim 1, wherein the displayed visual feedback comprises pictorial information.

9. The method of claim 1, wherein the displayed visual feedback is superimposed over a visual depiction of the user-performed gesture.

10. The method of claim 1, wherein the displayed visual feedback comprises a visual deformation of the motion-sensitive display surface.

11. The method of claim 1, wherein the displayed visual feedback is displayed in real-time, as the user is inputting gestures.

12. The method of claim 1, further comprising allowing the user to create a gesture that is recognized by one or more programs as a program command.

13. The method of claim 1, wherein the gesture recognition tutorial system processes all possible inputs within the context of the detected movement inputs while the gesture is being performed.

14. The method of claim 13, further comprising dynamically displaying the possible interpretations of the user'"'"'s gestures inputs in real-time.

15. At a gesture recognition tutorial system including a motion-sensitive display surface capable of recognizing various forms of user interaction with the surface, a method of providing a gesture tutorial mechanism for instructing one or more users how to properly enter program commands via the motion-sensitive display surface using gestures, the method comprising:

identifying one or more gesture commands recognized by a program of a motion-sensitive display surface, the motion-sensitive display surface being configured to detect user movement of an object over the surface that recognizes such user interaction therewith; and

based on the identified one or more gesture commands, displaying a visual example of a gesture capable of being performed on the motion-sensitive display surface, wherein the visual example of the gesture mimics one or more motions a user can perform for executing the one or more gesture commands of a program associated with the motion-sensitive display surface.

16. The method of claim, 15 further comprising:

at the motion-sensitive display surface, detecting a user-performed gesture;

based on the user-performed gesture, determining if the user-performed gesture appropriately corresponded to the example gesture; and

based upon the determination, indicating whether or not the user-performed gesture was proper.

17. The method of claim 15, wherein the visual example gesture corresponds to a command generally recognized by the gesture recognition tutorial system.

18. The method of claim 15, wherein the gesture recognition tutorial system processes all possible inputs within the context of the detected user movements while the gesture is being performed.

19. The method of claim 15, further comprising labeling the visual example with a label that indicates the gesture command displayed in the visual example.

20. At a gesture recognition tutorial system including a motion-sensitive display surface, a computer program product comprising one or more computer-readable media having thereon computer-executable instructions that, when executed by one or more processors of a computer system, cause the computer system to perform a method for suggesting recognized application gesture commands based on the user interaction with the surface, the method comprising:

receiving at a motion-sensitive display surface a user-performed gesture, which includes user movement of an object over the surface that recognizes such user interaction therewith;

based on the received user-performed gesture, identifying one or more gesture commands recognized by a program of the motion-sensitive display surface, wherein the one or more gesture commands recognized by the program represent actual commands of the program that closely resemble those corresponding to the user-performed gesture; and

displaying visual feedback on the motion-sensitive display surface representing the one or more gesture commands recognized by the program in order to aid the user in understanding appropriate various defined gestures representing actual program commands.