Gestures for touch sensitive input devices

• US 20060026521A1
• Filed: 07/30/2004
• Published: 02/02/2006
• Est. Priority Date: 07/30/2004
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Computer Incorporated (Apple Inc.)
• Inventors: Chaudhri, Imran, Ive, Jonathan P., Christie, Greg, Ording, Bas, Kerr, Duncan Robert, Hotelling, Steve, Strickon, Joshua A., Huppi, Brian Q.
• Granted Patent: US 8,479,122 B2
• Time in Patent Office: Days
• US Class Current: 715/702
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Methods and systems for processing touch inputs are disclosed. The invention in one respect includes reading data from a multipoint sensing device such as a multipoint touch screen where the data pertains to touch input with respect to the multipoint sensing device, and identifying at least one multipoint gesture based on the data from the multipoint sensing device.

Claims

1. A computer implemented method for processing touch inputs, said method comprising:

reading data from a touch screen, the data pertaining to touch input with respect to the touch screen, and the touch screen having a multipoint capability; and

identifying at least one multipoint gesture based on the data from the touch screen.

2. A computer implemented method as recited in claim 1 wherein said method further comprises:

performing a compounding operation associated with the at least one gesture.

3. A computer implemented method as recited in claim 1 wherein said method further comprises:

associating a user interface element to the at least one gesture.

4. A computer implemented method as recited in claim 3 wherein said method further comprises:

displaying the user interface element on the touch screen.

5. A computer implemented method as recited in claim 1 wherein said method further comprises:

displaying the user interface element on the touch screen prior to said reading; and

subsequently altering the user interface element being displayed based on the at least one gesture.

6. A computer implemented method as recited in claim 1 wherein said identifying identifies at least two multipoint gestures based on the data from the touch screen.

7. A computer implemented method as recited in claim 6 wherein the at least two gestures being identified are concurrently performed on the touch screen.

8. A computer implemented method as recited in claim 6 wherein the at least two gestures being identified are contiguously performed on the touch screen.

9. A gestural method, comprising:

detecting multiple touches at different points on a touch sensitive surface at the same time; and

segregating the multiple touches into at least two separate gestural inputs occurring simultaneously, each gestural input having a different function.

10. The gestural method as recited in claim 9 wherein touch sensitive surface is associated with a touch screen that overlays a display, and wherein the gestural inputs are selected from zooming, panning and rotating.

11. A gestural method, comprising:

concurrently detecting a plurality of gestures that are concurrently performed with reference to a touch sensing device;

producing different commands for each of the gestures that have been detected.

12. The method as recited in claim 11 wherein the plurality of gestures are implemented with the same group of touches.

13. The method as recited in claim 11 wherein the plurality of gestures are implemented with different groups of touches.

14. The method as recited in claim 11 wherein the gestures include rotate gestures, pang gestures, or zooming gestures.

15. A gestural method, comprising:

displaying a graphical image on a display screen;

detecting a plurality of touches at the same time on a touch sensitive device; and

linking the detected multiple touches to the graphical image presented on the display screen.

16. The method as recited in claim 15 wherein the graphical image is a user interface element.

17. The method as recited in claim 16 wherein the user interface element is displayed when a particular multitouch configuration is detected.

18. The method as recited in claim 17 wherein the user interface element is displayed in proximity to the multiple touches.

19. The method as recited in claim 18 wherein the user interface element is displayed underneath the multiple touches.

20. The method as recited in claim 16 wherein the multiple touches are detected over the displayed user interface element,

21. The method as recited in claim 16 wherein the multiple touches are associated with a single user interface element.

22. The method as recited in claim 16 wherein a first portion of the multiple touches are associated with a first user interface element and a second portion of the multiple touches are associated with a second user interface element that is different than the first user interface element.

23. The method as recited in claim 15 wherein after linking the detected multiple touches to the graphical image, the graphical image changes based on the motion of the linked multiple touches.

24. The method as recited in claim 23 wherein the graphical image is changed in unison with the motion of the linked multiple touches such that the changes in the graphical image continuously follows the motion of the linked multiple touches.

25. The method as recited in claim 23 wherein the size of the graphical image changes when at least one of the touches in the linked multiple touches is translated over the touch sensitive.

26. The method as recited in claim 23 wherein the size of features embedded in the graphical image changes when at least one of the touches in the linked multiple touches is translated over the touch sensitive surface.

27. The method as recited in claim 23 wherein the orientation of the graphical image changes when at least one of the touches in the linked multiple touches is rotated over the touch sensitive surface.

28. The method as recited in claim 23 wherein the position of the graphical image changes when the linked multiple touches are translated together over the touch sensitive surface.

29. A method of invoking a user interface element on a display via a multipoint touch screen of a computing system, said method comprising:

detecting and analyzing the simultaneous presence of two or more objects on contact with said touch screen;

based at least in part on said analyzing, selecting a user interface tool, from a plurality of available tools, to display on a display for interaction by a user of said computing system; and

controlling the interface tool based at least in part on the further movement of said objects in relation to said touch screen.

30. The method as recited in claim 29 wherein said selecting is also based in part on the current mode of the computing device at the time of the simultaneous presence of the two or more objects.

31. The method as recited in claim 29 wherein said tool is displayed at a location on the display proximate to the presence of said two or more objects.

32. A touch-based method, comprising

detecting a user input that occurs over a multipoint sensing device, the user input including one or more inputs, each input having a unique identifier;

during the user input, classifying the user input as a tacking or selecting input when the user input includes one unique identifier or a gesture input when the user input includes at least two unique identifiers;

performing tracking or selecting during the user input when the user input is classified as a tracking or selecting input;

performing one or more control actions during the user input when the user input is classified as a gesturing input.

33. The method as recited in claim 32 wherein scrolling or panning is performed when the at least two unique identifiers move together in substantially the same direction.

34. The method as recited in claim 32 wherein zooming is performed when the at least two unique identifiers linearly move away or towards one another.

35. The method as recited in claim 32 wherein rotation is performed when the at least two unique identifiers rotate relative to each other or relative to a known point.

36. The method as recited in claim 32 wherein multiple control actions are performed simultaneously during the same user input.

37. The method as recited in claim 32 wherein the user input is a continuous stroke, the stroke maintaining continuous contact on the multipoint sensing device.

92. The method as recited in claim 32 wherein the control actions are based at least in part on changes that occur between the at least two unique identifiers.

93. The method as recited in claim 32 wherein the control actions are based at least in part on changes that occur with the at least two unique identifiers.

38. A touch-based method, comprising:

outputting a GUI on a display;

detecting a user input on a touch sensitive device;

analyzing the user input for characteristics indicative of tracking, selecting or a gesturing;

categorizing the user input as a tracking, selecting or gesturing input;

performing tracking or selecting in the GUI when the user input is categorized as a tracking or selecting input;

performing control actions in the GUI when the user input is categorized as a gesturing input, the actions being based on the particular gesturing input.

94. The method as recited in claim 38 wherein the gesturing input is two or more fingers that are translated together in substantially the same direction relative to the touch sensitive device, and wherein the control action is scrolling, the scrolling action being based on one or more characteristics of the translating two or more fingers.

95. The method as recited in claim 38 wherein the gesturing input is two or more fingers that are translated together in substantially the same direction relative to the touch sensitive device, and wherein the control action is panning, the panning action being based on one or more characteristics of the translating two or more fingers.

96. The method as recited in claim 38 wherein the user input is categorized as a tracking or selection input when only one finger is detected over touch sensitive device, and wherein the user input is categorized as a gesturing input when two or more fingers are detected over the touch sensitive device.

97. The method as recited in claim 96 wherein the control action corresponds to scrolling through the GUI when the gesturing input corresponds to two fingers translating together across the touch sensitive device.

98. The method as recited in claim 96 wherein the control action corresponds to panning through the GUI when the gesturing input corresponds to two fingers translating together across the touch sensitive device.

99. The method as recited in claim 96 wherein the control action corresponds to rotating the GUI or a GUI object when the gesturing input corresponds to two fingers rotating over the touch sensitive device.

100. The method as recited in claim 96 wherein the control action corresponds to zooming the GUI or a GUI object when the gesturing input corresponds to two fingers translating relative to each other over the touch sensitive device.

39. A touch-based method, comprising:

capturing an initial touch image;

determining the touch mode based on the touch image;

capturing the next touch image;

determining if the touch mode changed between the initial and next touch images;

if the touch mode changed, setting the next touch image as the initial touch image and determining the touch mode based on the new initial touch image; and

if the touch mode stayed the same, comparing the touch images and performing a control function based on the comparison.

40. The method as recited in claim 39 wherein the control function is selected from tracking, panning, zooming, rotating, scrolling, or enlarging.

41. A computer implemented method for processing touch inputs, said method comprising:

reading data from a touch screen, the data pertaining to touch input with respect to the touch screen, and the touch screen having a multipoint capability;

converting the data to a collection of features;

classifying the features;

grouping the features into one or more feature groups;

calculating key parameters of the feature groups; and

associating the feature groups to user interface elements on a display.

46. The method as recited in claim 41 wherein the step of associating comprises:

receiving group of features;

determining if there is a change in number of features;

if there is a change, calculating initial parameter values;

if there is no change, calculating current parameter values; and

reporting both the initial and current parameter values.

42. The computer implemented method as recited in 41 wherein said method further comprises:

recognizing when at least one of the feature groups indicates performance of a gesture relative to its associated user interface element.

43. The computer implemented method as recited in 41 wherein said method further comprises:

providing user feedback when at least one of the feature groups indicates performance of a gesture relative to its associated user interface element.

44. The computer implemented method as recited in 41 wherein said method further comprises:

implementing an action when at least one of the feature groups indicates performance of a gesture relative to its associated user interface element.

45. The computer implemented method as recited in 44, wherein said method further comprises:

providing user feedback in conjunction with the action.

47. A computer implemented method, comprising:

outputting a graphical image;

receiving a multitouch gesture input over the graphical image; and

changing the graphical image based on and in unison with multitouch gesture input.

48. The method as recited in claim 47 further comprising:

receiving a second multitouch gesture input over the graphical image; and

changing the graphical image based on and in unison with the second multitouch gesture.

49. The method as recited in claim 47 further comprising:

outputting a second graphical image;

receiving a second multitouch gesture input over the second graphical image; and

changing the second graphical image based on and in unison with the second multitouch gesture.

50. A touch based method, comprising:

receiving a gestural input over a first region;

generating a first command when the gestural input is received over the first region;

receiving the same gestural input over a second region; and

generating a second command when the same gestural input is received over the second region, the second command being different than the first command.

51. A method for recognizing multiple gesture inputs, the method comprising:

receiving a multitouch gestural stroke on a touch sensitive surface, the multitouch gestural stroke maintaining continuous contact on the touch sensitive surface;

recognizing a first gesture input during the multitouch gestural stroke; and

recognizing a second gesture input during the multitouch gestural stroke.

52. The method as recited in claim 51 further including recognizing a third gesture input during the multitouch gestural stroke

53. A computer implemented method, comprising:

detecting a plurality of touches on a touch sensing device;

forming one or more touch groups with the plurality of touches;

monitoring the movement of and within each of the touch groups; and

generating control signals when the touches within the touch groups are moved or when the touch groups are moved in their entirety.

54. A method for recognizing a zoom gesture made on a multipoint touch screen computing system, comprising:

detecting the relative locations of a first object and a second object at the same time;

detecting a change in the relative locations of said first and second object;

generating a zoom signal in response to said detected change.

55. The method as recited in claim 54 wherein said zoom signal is a zoom out signal.

56. The method as recited in claim 54 wherein said zoom signal is a zoom in signal.

57. The method as recited in claim 54 further comprising:

associating the first and second objects to an image on a GUI interface;

58. The method as recited in claim 54 further comprising zoom out the image when a zoom out signal is generated.

59. The method as recited in claim 54 further comprising zooming in the image when a zoom in signal is generated.

60. The method as recited in claim 54 further comprising zooming the image when a zoom signal is generated, and wherein the zooming occurs substantially simultaneously with the change in relative locations of the objects.

61. The method as recited in claim 61 wherein the amount of zooming varies according to the distance between the two objects.

62. A method for recognizing a pan gesture made on a multipoint touch screen, comprising:

detecting the presence of at least a first object and a second object at the same time;

monitoring the position of the said at least first and second objects when the objects are moved together across the touch screen; and

generating a pan signal when the position of the said at least first and second objects changes relative to an initial position.

63. The method as recited in claim 62 further comprising:

associating said at least first and second objects to an image on a GUI interface;

panning the image when a pan signal is generated; and

wherein the panning occurs substantially simultaneously with the motion of the objects.

64. A method for recognizing a rotate gesture made on a multipoint touch screen, comprising:

detecting the presence of at least a first object and a second object at the same time;

detecting a rotation of said at least first and second objects; and

generating a rotate signal in response to said detected rotation of said at least first and second objects.

65. The method as recited in claim 64 further comprising:

associating said at least first and second objects to an image on a GUI interface;

rotating the image when a rotate signal is generated; and

wherein the rotating occurs substantially simultaneously with the motion of the objects.

66. A computer implemented method for initiating floating controls via a touch screen, the method comprising:

detecting the presence of an object on the touch screen;

recognizing the object; and

generating a user interface element on the touch screen in the vicinity of the object based on the recognized object.

67. The method as recited in claim 66, wherein the user interface element includes one or more control options that can be selected by a user of the touch screen.

68. The method as recited in claim 67 further comprising generating a control signal when one of the control options is selected by the object.

69. The method as recited in claim 68 wherein the control signal modifies the functionality of the computer.

70. The method as recited in claim 68 wherein the control signal modifies a user interface element.

71. The method as recited in claim 66 wherein the object is a finger or a stylus.

72. A computer implemented method for initiating zooming targets via a touch screen, the method comprising:

displaying an image on a GU;

enlarging the image for a period of time when the presence of an object is detected over the image.

73. The method as recited in claim 72 wherein the image is a user interface element.

74. The method as recited in claim 73 wherein the user interface element is a control box containing one or more control buttons, and wherein said enlarging includes enlarging the control box and each of the control buttons.

75. The method as recited in claim 74 wherein the size of the control buttons correspond to the size of the object or larger so that the control buttons can be easily selected by the object.

76. The method as recited in claim 75 further comprising:

in the enlarged state, generating a control signal associated with the control button when the presence of the object is detected over one of the control buttons.

77. The method as recited in claim 72 wherein the object is a finger.

78. The method as recited in claim 72 further comprising:

implementing a gesture input when the presence of one or more objects are detected over portions of the GUI other than the image.

79. A computer implemented method of initiating a page turn via a touch screen, the method comprising:

displaying a page from a multitude of pages in a GUI presented on the touch screen;

detecting the presence of an object in a predetermined region on the touch screen over the page; and

generating a page turn signal when the object is translated on the touch screen in the predetermined region.

80. The computer implemented method as recited in claim 79 wherein the translation simulates a finger turning the page in a paper bound book.

81. The computer implemented method as recited in claim 79 wherein the translation pertains to a page turn gesture by a user.

82. The computer implemented method as recited in claim 79 wherein said method further comprises:

displaying a next page in response to the page tarn signal.

83. A computer implemented method of initiating inertia, the method comprising:

displaying an image on a GUI;

detecting a stroke on a touch sensitive surface;

noting the speed and direction of the stroke;

moving the image or features embedded in the image in accordance with the speed and direction of the stroke; and

slowing the motion of the image or features embedded in the image in accordance with inertia principals when the stroke is no longer detected.

84. The method as recited in claim 83 further comprising:

detecting an object on the touch sensitive surface when image or features embedded in the image is slowing down because of inertia;

stopping the motion of the image or features embedded in the image when another object is detected, the another object serving as a braking means to the moving image or features embedded in the image

85. A method of simulating a keyboard, comprising:

providing a display and a touch screen positioned over the display;

displaying a keyboard on the display, the keyboard including at least a first and a second key;

detecting the presence of a first object over the first key and a second object over the second key at the same time; and

generating a single control function in response to the detection of said first object over the first key and said second object over said second key.

86. The method as recited in claim 85 wherein the first and second objects are portions of one or more hands.

87. The method as recited in claim 85 wherein the first key is a modifying key that modifies the second key.

88. The method as recited in claim 87 wherein the first key is a shift key, control key or alt key.

89. A computer implemented method comprising:

presenting a virtual wheel;

detecting at least a first finger over the virtual wheel;

associating the finger to the virtual wheel; and

generating control signals when the finger is moved about the virtual wheel.

90. The method as recited in claim 89 further comprising:

rotating the presented virtual wheel when the finger is moved about the virtual wheel.

91. The method as recited in claim 89 wherein the control signals implement scrolling on a display.