GESTURE LEARNING

• US 20080163130A1
• Filed: 06/15/2007
• Published: 07/03/2008
• Est. Priority Date: 01/03/2007
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: WESTERMAN, WAYNE CARL
• Granted Patent: US 9,311,528 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Systems and methods that incorporate various techniques for teaching gestures to a user of a multi-touch sensitive device are disclosed herein. Such techniques can include presenting visible feedback of gestures, such as animated motion trails and/or hand motions, along with affirmative feedback for correctly performed gestures and negative feedback for incorrectly performed gestures. Such techniques can be expanded to provide training exercises that present tasks requiring a particular gesture or sequence of gestures to be performed. These training exercises can take the form of games or other engaging activities to encourage use.

Claims

1. A method for teaching gestures performable on a multi-touch interface, the method comprising:

presenting to a user a display, the display comprising a first display area and a second display area, the second display area comprising a multi-touch monitor window;

detecting a gesture currently being performed by the user; and

presenting in the multi-touch monitor window an interactive feedback mechanism that indicates the gesture currently being performed by the user.

2. The method of claim 1 wherein the multi-touch monitor window is overlaid atop the first display area.

3. The method of claim 2 wherein the multi-touch monitor window incorporates transparency or translucency.

4. The method of claim 1 wherein the multi-touch monitor window is displayed separately from the first display area.

5. The method of claim 1 wherein the interactive feedback mechanism comprises an animated hand together with one or more motion indications.

6. The method of claim 5 wherein the animated hand comprises one or more items selected from the group consisting of:

a line drawing, a three-dimensional rendering, and a translucent shadow of a hand.

7. The method of claim 5 wherein the one or more motion indications comprise motion trails.

8. The method of claim 7 wherein the motion trails are color-coded.

9. The method of claim 5 wherein the interactive feedback mechanism has superimposed thereon an animated display of a gesture so that the user can perceive the difference between the gesture currently being performed by the user and an ideal gesture.

10. The method of claim 1 wherein the first display area is used in conjunction with a utility, entertainment, or communication application.

11. The method of claim 1 wherein the first display area is used in conjunction with a gesture learning application.

12. The method of claim 11 wherein the gesture learning application is a game.

13. The method of claim 11 wherein:

a gesture to be performed by the user is presented in the first display area; and

wherein the method further comprises;

comparing the detected gesture to the gesture to be performed; and

if the detected gesture does not correspond to the gesture to be performed providing a negative feedback indicator; and

if the detected gesture does correspond to the gesture to be performed, providing a positive feedback indicator.

14. The method of claim 11 wherein:

a gesture to be performed by the user is presented in the first display area; and

wherein the method further comprises;

comparing the detected gesture to the gesture to be performed; and

if the detected gesture does not correspond to the gesture to be performed providing a negative feedback indicator; and

if the detected gesture does correspond to the gesture to be performed, displaying a next gesture to be performed by the user in the first display area.

15. A graphical user interface display for a gesture learning application, the graphical user interface display comprising:

a main window including an indication of a gesture to be performed using a multi-touch interface;

a multi-touch monitor window including an interactive feedback mechanism, the interactive feedback mechanism including an indication of a gesture currently being performed by a user.

16. The graphical user interface of claim 15 wherein the indication of a gesture to be performed displayed in the main window includes one or more items selected from the group consisting of:

an animated hand, one or more motion trails, an iconographic representation, a textual description of the gesture to be performed, and a textual description of a command corresponding to the gesture to be performed.

17. The graphical user interface of claim 15 wherein the indication of the gesture currently being performed by the user includes one or more items selected from the group consisting of:

an animated hand, one or more motion trails, an iconographic representation, a textual description of the gesture currently being performed, a textual description of a command corresponding to the gesture currently being performed, a positive feedback indicator, and a negative feedback indicator.

18. The graphical user interface of claim 17 wherein at least one of the positive feedback indicator and the negative feedback indicator comprise an audible indication.

19. The graphical user interface of claim 15 wherein the indication of a gesture to be performed in the main window comprises a sequence of gestures.

20. The graphical user interface of claim 19 wherein the sequence of gestures is arranged according to a particular chord or sequence of chords.

21. The graphical user interface of claim 19 wherein the sequence of gestures is arranged according to a particular motion or sequence of motions.

22. The graphical user interface of claim 19 further comprising at least one of a positive feedback indicator and a negative feedback indicator displayed in the main window.

23. A computer system having a multi-touch interface and a graphical user interface, wherein the computer system includes a computer memory encoded with executable instructions causing the computer system to:

present to a user a display, the display comprising a first display area and a second display area, the second display area comprising a multi-touch monitor window;

detect a gesture currently being performed by the user; and

present in the multi-touch monitor window an interactive feedback mechanism that indicates the gesture currently being performed by the user.

24. The computer system of claim 23 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

25. The computer system of claim 23 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

26. The computer system of claim 23 wherein the interactive feedback mechanism comprises an animated hand together with one or more motion indications.

27. The computer system of claim 26 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

28. The computer system of claim 26 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

29. The computer system of claim 26 wherein the interactive feedback mechanism has superimposed thereon an animated display of a gesture so that the user can perceive the difference between the gesture currently being performed by the user and an ideal gesture.

30. The computer system of claim 29 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

31. The computer system of claim 29 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

32. A computer system of claim 23 wherein the first display area is used in conjunction with a gesture learning application.

33. The computer system of claim 32 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

34. The computer system of claim 32 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

35. The computer system of claim 32 wherein:

a gesture to be performed by the user is presented in the first display area; and

wherein the executable instructions further cause the computer system to;

compare the detected gesture to the gesture to be performed; and

if the detected gesture does not correspond to the gesture to be performed provide a negative feedback indicator; and

if the detected gesture does correspond to the gesture to be performed, provide a positive feedback indicator.

36. The computer system of claim 35 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

37. The computer system of claim 35 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

38. The computer system of claim 32 wherein:

a gesture to be performed by the user is presented in the first display area; and

wherein the executable instructions further cause the computer system to;

compare the detected gesture to the gesture to be performed; and

if the detected gesture does not correspond to the gesture to be performed provide a negative feedback indicator; and

if the detected gesture does correspond to the gesture to be performed, display a next gesture to be performed by the user in the first display area.

39. The computer system of claim 38 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

40. The computer system of claim 38 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

41. A computer system having a multi-touch interface and a graphical user interface, wherein the computer system includes a computer memory encoded with executable instructions causing the computer system to generate a graphical user interface, the graphical user interface comprising:

a main window including an indication of a gesture to be performed using a multi-touch interface;

a multi-touch monitor window including an interactive feedback mechanism, the interactive feedback mechanism including an indication of a gesture currently being performed by a user.

42. The computer system of claim 41 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

43. The computer system of claim 41 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

44. The computer system of claim 41 wherein the indication of a gesture to be performed displayed in the main window includes one or more items selected from the group consisting of:

an animated hand, one or more motion trails, an iconographic representation, a textual description of the gesture to be performed, and a textual description of a command corresponding to the gesture to be performed.

45. The computer system of claim 44 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

46. The computer system of claim 44 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

47. The computer system of claim 41 wherein the indication of the gesture currently being performed by the user includes one or more items selected from the group consisting of:

an animated hand, one or more motion trails, an iconographic representation, a textual description of the gesture currently being performed, a textual description of a command corresponding to the gesture currently being performed, a positive feedback indicator, and a negative feedback indicator.

48. The computer system of claim 47 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

49. The computer system of claim 47 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

50. A mobile telephone having a multi-touch interface and a graphical user interface, wherein the mobile telephone includes a memory encoded with executable instructions causing the mobile telephone to:

present to a user a display, the display comprising a first display area and a second display area, the second display area comprising a multi-touch monitor window;

detect a gesture currently being performed by the user; and

present in the multi-touch monitor window an interactive feedback mechanism that indicates the gesture currently being performed by the user.

51. The mobile telephone of claim 50 wherein the interactive feedback mechanism comprises an animated hand together with one or more motion indications.

52. The mobile telephone of claim 51 wherein the interactive feedback mechanism has superimposed thereon an animated display of a gesture so that the user can perceive the difference between the gesture currently being performed by the user and an ideal gesture.

53. A mobile telephone of claim 50 wherein the first display area is used in conjunction with a gesture learning application.

54. The mobile telephone of claim 53 wherein:

a gesture to be performed by the user is presented in the first display area; and

wherein the executable instructions further cause the mobile telephone to;

compare the detected gesture to the gesture to be performed; and

if the detected gesture does not correspond to the gesture to be performed provide a negative feedback indicator; and

if the detected gesture does correspond to the gesture to be performed, provide a positive feedback indicator.

55. The mobile telephone of claim 53 wherein:

a gesture to be performed by the user is presented in the first display area; and

wherein the executable instructions further cause the mobile telephone to;

compare the detected gesture to the gesture to be performed; and

if the detected gesture does not correspond to the gesture to be performed provide a negative feedback indicator; and

if the detected gesture does correspond to the gesture to be performed, display a next gesture to be performed by the user in the first display area.

56. A mobile telephone having a multi-touch interface and a graphical user interface, wherein the mobile telephone includes a memory encoded with executable instructions causing the mobile telephone to generate a graphical user interface, the graphical user interface comprising:

a main window including an indication of a gesture to be performed using a multi-touch interface;

a multi-touch monitor window including an interactive feedback mechanism, the interactive feedback mechanism including an indication of a gesture currently being performed by a user.

57. The mobile telephone of claim 56 wherein the indication of a gesture to be performed displayed in the main window includes one or more items selected from the group consisting of:

an animated hand, one or more motion trails, an iconographic representation, a textual description of the gesture to be performed, and a textual description of a command corresponding to the gesture to be performed.

58. The mobile telephone of claim 56 wherein the indication of the gesture currently being performed by the user includes one or more items selected from the group consisting of:

an animated hand, one or more motion trails, an iconographic representation, a textual description of the gesture currently being performed, a textual description of a command corresponding to the gesture currently being performed, a positive feedback indicator, and a negative feedback indicator.