Multi-touch gesture dictionary

• US 7,840,912 B2
• Filed: 01/03/2007
• Issued: 11/23/2010
• Est. Priority Date: 01/30/2006
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Haggerty, Myra Mary, Westerman, Wayne Carl, Elias, John Greer
• Primary Examiner(s): Pesin; Boris
• Assistant Examiner(s): SHIAU, SHEN C
• Application Number:
  US11/619,553
• Publication Number:
  US 20070177803A1
• Time in Patent Office: 1,420 Days
• Field of Search: 715/863, 715/864, 382/126, 382/243
• US Class Current: 715/863
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04895 Guidance during keyboard input operation, e.g. prompting
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A multi-touch gesture dictionary is disclosed herein. The gesture dictionary can include a plurality of entries, each corresponding to a particular chord. The dictionary entries can include a variety of motions associated with the chord and the meanings of gestures formed from the chord and the motions. The gesture dictionary may take the form of a dedicated computer application that may be used to look up the meaning of gestures. The gesture dictionary may also take the form of a computer application that may be easily accessed from other applications. The gesture dictionary may also be used to assign user-selected meanings to gestures. Also disclosed herein are computer systems incorporating multi-touch gesture dictionaries. The computer systems can include, desktop computers, tablet computers, notebook computers, handheld computers, personal digital assistants, media players, mobile telephones, and the like.

Claims

1. A method of providing to a user a dictionary of multi-touch gestures, each of the gestures comprising a chord and a motion associated with the chord, the method comprising:

identifying a chord presented to a multi-touch interface by the user to begin a gesture, the identified chord comprising a combination of hand parts associated with the gesture but distinct from motion associated with the identified chord to complete the gesture; and

in response to the identifying of the chord presented to begin the gesture, and prior to identifying motion associated with the identified chord to complete the gesture, displaying a dictionary entry for the identified chord, the displayed dictionary entry comprising a visual depiction of multiple motions associated with the identified chord and, for each of the multiple motions associated with the identified chord, a meaning of a gesture comprising the identified chord and the corresponding motion.

2. The method of claim 1 further comprising:

identifying a subsequent chord presented to the multi-touch interface by the user; and

displaying a dictionary entry for the identified subsequent chord, the dictionary entry comprising a visual depiction of one or more motions associated with the identified subsequent chord and, for each of the one or more motions associated with the identified subsequent chord, a meaning of a gesture comprising the identified subsequent chord and the motion.

3. The method of claim 1 wherein the dictionary entry comprises multiple motion icons, each motion icon including a graphical depiction of a motion and a textual description of a corresponding meaning.

4. The method of claim 3 wherein the motion icons are arranged consistently for a plurality of dictionary entries corresponding to a plurality of chords.

5. The method of claim 1 wherein the dictionary entry comprises an animation of the multiple motions.

6. The method of claim 1 further comprising:

identifying a motion of the identified chord performed by the user; and

providing feedback to the user indicating a gesture formed by the identified chord and the identified motion.

7. The method of claim 6 wherein the feedback includes visual feedback.

8. The method of claim 7 wherein the visual feedback includes highlighting a motion icon associated with the gesture formed by the identified chord and the identified motion.

9. The method of claim 6 wherein the feedback includes audible feedback.

10. The method of claim 9 wherein the audible feedback includes speaking a meaning of the gesture formed by the identified chord and the identified motion.

11. The method of claim 1 wherein the chord further comprises one or more modifier keys.

12. A method of providing to a user a dictionary of multi-touch gestures, each of the gestures comprising a chord and a motion associated with the chord, the method comprising:

identifying a chord presented to a multi-touch interface by the user to begin a gesture, the identified chord comprising a combination of hand parts associated with the gesture but distinct from motion associated with the identified chord to complete the gesture, wherein the identified chord is presented during use of a non-dictionary application;

detecting an absence of motion associated with the identified chord to complete the gesture for a predetermined period of time after the identifying of the chord presented to begin the gesture; and

in response to the detecting of the absence of motion for the predetermined period of time, displaying a dictionary entry for the identified chord, the displayed dictionary entry comprising a visual depiction of multiple motions associated with the identified chord and, for each of the multiple motions associated with the identified chord, a meaning of a gesture comprising the identified chord and the corresponding motion.

13. The method of claim 12 further comprising:

identifying a subsequent chord presented to the multi-touch interface by the user; and

displaying a dictionary entry for the identified subsequent chord, the dictionary entry comprising a visual depiction of one or more motions associated with the identified subsequent chord and, for each of the one or more motions associated with the identified subsequent chord, a meaning of a gesture comprising the identified subsequent chord and the motion.

14. The method of claim 12 further comprising:

identifying a motion of the identified chord performed by the user; and

executing the meaning of the gesture formed by the identified chord and the identified motion.

15. The method of claim 14 further comprising:

providing feedback to the user indicating a gesture formed by the identified chord and the identified motion.

16. The method of claim 15 wherein the feedback includes visual feedback.

17. The method of claim 16 wherein the visual feedback includes highlighting a motion icon associated with the gesture formed by the identified chord and the identified motion.

18. The method of claim 15 wherein the feedback includes audible feedback.

19. The method of claim 18 wherein the audible feedback includes speaking a meaning of the gesture formed by the identified chord and the identified motion.

20. The method of claim 12 wherein the dictionary entry comprises multiple motion icons, each motion icon including a graphical depiction of a motion and a textual description of a corresponding meaning.

21. The method of claim 20 wherein the motion icons are arranged consistently for a plurality of dictionary entries corresponding to a plurality of chords.

22. The method of claim 12 wherein the dictionary entry comprises an animation of the multiple motions.

23. The method of claim 12 wherein the chord further comprises one or more modifier keys.

24. A computer system capable of providing a dictionary of multi-touch gestures, the computer system having a multi-touch interface, a graphical user interface, and a computer memory encoded with executable instructions causing the computer system to:

identify a chord presented to the multi-touch interface by a user to begin a gesture, the identified chord comprising a combination of hand parts associated with the gesture but distinct from motion associated with the identified chord to complete the gesture; and

in response to the identifying of the chord presented to begin the gesture, and prior to identifying motion associated with the identified chord to complete the gesture, display a dictionary entry for the identified chord, the displayed dictionary entry comprising a visual depiction of multiple motions associated with the identified chord and, for each of the multiple motions associated with the identified chord, a meaning of a gesture comprising the identified chord and the corresponding motion.

25. The computer system of claim 24 wherein the computer system is selected from the group consisting of a desktop computer, a tablet computer, and a notebook computer.

26. The computer system of claim 24 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

27. The computer system of claim 24 wherein the dictionary entry comprises multiple motion icons, each motion icon including a graphical depiction of a motion and a textual description of a corresponding meaning.

28. The computer system of claim 27 wherein the motion icons are arranged consistently for a plurality of dictionary entries corresponding to a plurality of chords.

29. The computer system of claim 24 wherein the dictionary entry comprises an animation of the multiple motions.

30. The computer system of claim 24 wherein the executable instructions further cause the computer to:

identify a motion of the identified chord performed by the user; and

provide feedback to the user indicating a gesture formed by the identified chord and the identified motion.

31. The computer system of claim 30 wherein the feedback includes visual feedback.

32. The computer system of claim 31 wherein the visual feedback includes a highlighted motion icon associated with the gesture formed by the identified chord and the identified motion.

33. The computer system of claim 30 wherein the feedback includes audible feedback.

34. The computer system of claim 33 wherein the audible feedback includes a spoken meaning of the gesture formed by the identified chord and the identified motion.

35. The computer system of claim 24 wherein the chord further comprises one or more modifier keys.

36. A computer system capable of providing a dictionary of multi-touch gestures, the computer system having a multi-touch interface, a graphical user interface, and a computer memory encoded with executable instructions causing the computer system to:

identify a chord presented to the multi-touch interface by a user to begin a gesture, the identified chord comprising a combination of hand parts associated with the gesture but distinct from motion associated with the identified chord to complete the gesture, wherein the identified chord is presented during use of a non-dictionary application;

detect an absence of motion associated with the identified chord to complete the gesture for a predetermined period of time after the identifying of the chord presented to begin the gesture; and

in response to the detecting of the absence of motion for the predetermined period of time, display a dictionary entry for the identified chord, the displayed dictionary entry comprising a visual depiction of multiple motions associated with the identified chord and, for each of the multiple motions associated with the identified chord, a meaning of a gesture comprising the identified chord and the corresponding motion.

37. The computer system of claim 36 wherein the computer system is selected from the group of a desktop computer, a tablet computer, and a notebook computer.

38. The computer system of claim 36 wherein the computer system comprises at least one of a handheld computer, a personal digital assistant, a media player, and a mobile telephone.

39. The computer system of claim 36 wherein the dictionary entry comprises multiple motion icons, each motion icon including a graphical depiction of a motion and a textual description of a corresponding meaning.

40. The computer system of claim 39 wherein the motion icons are arranged consistently for a plurality of dictionary entries corresponding to a plurality of chords.

41. The computer system of claim 36 wherein the dictionary entry comprises an animation of the multiple motions.

42. The computer system of claim 36 wherein the executable instructions further cause the computer to:

identify a motion of the identified chord performed by the user; and

execute the meaning of the gesture formed by the identified chord and the identified motion.

43. The computer system of claim 42 wherein the executable instructions further cause the computer to provide feedback to the user indicating a gesture formed by the identified chord and the identified motion.

44. The computer system of claim 43 wherein the feedback includes visual feedback.

45. The computer system of claim 44 wherein the visual feedback includes a highlighted motion icon associated with the gesture formed by the identified chord and the identified motion.

46. The computer system of claim 43 wherein the feedback includes audible feedback.

47. The computer system of claim 46 wherein the audible feedback includes a spoken meaning of the gesture formed by the identified chord and the identified motion.

48. The computer system of claim 36 wherein the chord further comprises one or more modifier keys.

49. A mobile telephone capable of providing a dictionary of multi-touch gestures, the mobile telephone having a multi-touch interface and a graphical user interface, wherein the mobile telephone includes a memory encoded with executable instructions causing the mobile telephone to:

identify a chord presented to the multi-touch interface by a user to begin a gesture, the identified chord comprising a combination of hand parts associated with gesture but distinct from motion associated with the identified chord to complete the gesture; and

in response to the identifying of the chord presented to begin the gesture, and prior to identifying motion associated with the identified chord to complete the gesture, display a dictionary entry for the identified chord, the displayed dictionary entry comprising a visual depiction of multiple motions associated with the identified chord and, for each of the multiple motions associated with the identified chord, a meaning of a gesture comprising the identified chord and the corresponding motion.

50. The mobile telephone of claim 49 wherein the dictionary entry comprises multiple motion icons, each motion icon including a graphical depiction of a motion and a textual description of a corresponding meaning.

51. The mobile telephone of claim 49 wherein the dictionary entry comprises an animation of the multiple motions.

52. The mobile telephone of claim 49 wherein the executable instructions further cause the computer to:

identify a motion of the identified chord performed by the user; and

provide feedback to the user indicating a gesture formed by the identified chord and the identified motion.

53. The mobile telephone of claim 52 wherein the feedback includes visual feedback comprising a highlighted motion icon associated with the gesture formed by the identified chord and the identified motion.

54. The mobile telephone of claim 52 wherein the feedback includes audible feedback comprising a spoken meaning of the gesture formed by the identified chord and the identified motion.

55. The computer system of claim 49 wherein the chord further comprises one or more modifier keys.