Pinch-throw and translation gestures

• US 9,182,884 B2
• Filed: 02/07/2014
• Issued: 11/10/2015
• Est. Priority Date: 06/13/2007
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne Carl, Alten, Brett G.
• Primary Examiner(s): Lee, Gene W
• Application Number:
  US14/175,566
• Publication Number:
  US 20140152604A1
• Time in Patent Office: 641 Days
• Field of Search: 345173-178, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

The detection of finger pinch, rotate, and tap gestures along with a translation and optionally liftoff motion to initiate certain actions is disclosed. To detect both the gesture and the translation, a certain amount of gesture scaling speed can be detected along with a certain amount of translation speed and distance traveled. For a finger pinch gesture, the scaling speed can be computed as the dot product of the velocity vectors of two or more fingers coming together. For a finger rotation gesture, the scaling speed can be computed as a cross product of the velocity vectors of the rotating fingers. The translation speed of a gesture can be computed as the average of the velocity vectors of any fingers involved in the gesture. The amount of gesture scaling speed and translation speed needed to trigger the recognition of a combined gesture with translation can be a predetermined ratio.

Claims

1. A method for recognizing a combined tap and translation gesture on a sensor panel, comprising:

detecting object motion of one or more objects on the sensor panel;

identifying a tap gesture associated with the object motion;

identifying a translation of the one or more objects independent of the object motion associated with the tap gesture following a performance of the tap gesture;

recognizing the tap gesture and the translation as a combined tap and translation gesture distinct from the tap gesture and the translation; and

performing one of a plurality of different operations associated with the combined tap and translation gesture based on a number of a series of one or more taps in the identified tap gesture.

2. The method of claim 1, further comprising:

computing a translation speed of the one or more objects involved in the translation; and

recognizing the tap gesture and the translation as the combined tap and translation gesture distinct from the tap gesture and the translation if the translation speed is greater than a first predetermined threshold.

3. The method of claim 1, further comprising:

identifying the tap gesture by comparing actual images of touch captured over time to expected images of touch for various tap gesture types.

4. The method of claim 3, further comprising:

identifying the tap gesture by determining that the actual images of touch captured over time are substantially similar to one or more object taps in substantially the same location within a predetermined period of time.

5. The method of claim 4, further comprising:

determining a translation distance of the one or more objects involved in the translation; and

recognizing the tap gesture and the translation as the combined tap and translation gesture only if additionally the translation distance is greater than a predetermined translation threshold.

6. The method of claim 4,

wherein the operation is a drag and drop operation capable of moving a graphical user interface (GUI) object located under the tap gesture to a drop location when the translation moves the actual images of touch to the drop location.

7. The method of claim 4, further comprising:

determining whether a liftoff of the one or more objects occurred;

computing a liftoff velocity of the one or more objects at a time of the liftoff; and

recognizing the tap gesture, translation and liftoff as a combined tap, translation and liftoff gesture if the liftoff velocity is greater than a predetermined liftoff velocity threshold.

8. The method of claim 4,

wherein the operation is a trash operation capable of moving a graphical user interface (GUI) object located under the tap gesture to the trash when the translation moves the actual images of touch towards a trash icon.

9. The method of claim 4,

wherein the operation is a move to another desktop operation capable of moving a graphical user interface (GUI) object located under the tap gesture to another desktop when the translation moves the actual images of touch towards the other desktop.

10. The method of claim 4,

wherein the operation is a window minimize operation capable of minimizing a window located under the tap gesture when the translation moves the actual images of touch towards a corner of a desktop.

11. The method of claim 4,

wherein the operation is a print operation capable of printing an image associated with a graphical user interface (GUI) object located under the tap gesture when the translation moves the actual images of touch towards a printer icon.

12. The method of claim 4,

wherein the operation is a scrolling operation capable of scrolling a graphical user interface (GUI) object located under the tap gesture when the translation moves the actual images of touch in a particular direction to be scrolled.

13. The method of claim 12, further comprising:

performing the scrolling operation with a gradual decay in a scrolling speed when a liftoff velocity is greater than a predetermined liftoff velocity threshold.

14. A non-transitory computer-readable storage medium comprising program code for recognizing a combined tap and translation gesture on a sensor panel, the program code for causing performance of a method comprising:

detecting object motion of one or more objects on the sensor panel;

identifying a tap gesture associated with the object motion;

identifying a translation of the one or more objects independent of the object motion associated with the tap gesture following a performance of the tap gesture;

recognizing the tap gesture and the translation as a combined tap and translation gesture distinct from the tap gesture and the translation; and

performing one of a plurality of different operations associated with the combined tap and translation gesture based on a number of a series of one or more taps in the identified tap gesture.

15. The non-transitory computer-readable storage medium of claim 14, the program code further for causing performance of a method comprising:

computing a translation speed of the one or more objects involved in the translation; and

recognizing the tap gesture and the translation as the combined tap and translation gesture distinct from the tap gesture and the translation if the translation speed is greater than a first predetermined threshold.

16. The non-transitory computer-readable storage medium of claim 14, the program code further for causing performance of a method comprising:

identifying the tap gesture by comparing actual images of touch captured over time to expected images of touch for various tap gesture types.

17. The non-transitory computer-readable storage medium of claim 16, the program code further for causing performance of a method comprising:

identifying the tap gesture by determining that the actual images of touch captured over time are substantially similar to one or more object taps in substantially the same location within a predetermined period of time.

18. The non-transitory computer-readable storage medium of claim 17, the program code further for causing performance of a method comprising:

determining a translation distance of the one or more objects involved in the translation; and

recognizing the tap gesture and the translation as the combined tap and translation gesture only if additionally the translation distance is greater than a predetermined translation threshold.

19. The non-transitory computer-readable storage medium of claim 17, the program code further for causing performance of a method comprising:

wherein the operation is a drag and drop operation capable of moving a graphical user interface (GUI) object located under the tap gesture to a drop location when the translation moves the actual images of touch to the drop location.

20. The non-transitory computer-readable storage medium of claim 17, the program code further for causing performance of a method comprising:

determining whether a liftoff of the one or more objects occurred;

computing a liftoff velocity of the one or more objects at a time of the liftoff; and

recognizing the tap gesture, translation and liftoff as a combined tap, translation and liftoff gesture if the liftoff velocity is greater than a predetermined liftoff velocity threshold.

21. The non-transitory computer-readable storage medium of claim 17, the program code further for causing performance of a method comprising:

wherein the operation is a trash operation capable of moving a graphical user interface (GUI) object located under the tap gesture to the trash when the translation moves the actual images of touch towards a trash icon.

22. The non-transitory computer-readable storage medium of claim 17, the program code further for causing performance of a method comprising:

wherein the operation is a move to another desktop operation capable of moving a graphical user interface (GUI) object located under the tap gesture to another desktop when the translation moves the actual images of touch towards the other desktop.

23. The non-transitory computer-readable storage medium of claim 17, the program code further for causing performance of a method comprising:

wherein the operation is a window minimize operation capable of minimizing a window located under the tap gesture when the translation moves the actual images of touch towards a corner of a desktop.