Maintaining activity after contact liftoff or touchdown

• US 7,782,307 B2
• Filed: 11/14/2006
• Issued: 08/24/2010
• Est. Priority Date: 01/26/1998
• Status: Expired due to Fees

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne Carl, Elias, John G.
• Primary Examiner(s): Lefkowitz; Sumati
• Assistant Examiner(s): Amadiz; Rodney
• Application Number:
  US11/559,822
• Publication Number:
  US 20070070052A1
• Time in Patent Office: 1,379 Days
• Field of Search: 345156-178, 178 1801- 1807
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method of processing touch inputs of touch devices that touchdown and liftoff on or near a touch sensitive surface, the surface including an associated plurality of touch sensors, the method comprising:

reading data from the touch sensors to obtain positions of contacts corresponding to the touch devices on or near the surface wherein the data is read to enable simultaneous detection of multiple distinguishable contacts;

tracking the contacts based on the positions;

selecting an activity corresponding to an initial set of at least two of the contacts corresponding to at least two of the touch devices simultaneously on or near the surface, wherein the activity corresponds to one or more input events of a computing device;

maintaining the selection of the activity when the number of contacts on or near the surface changes due to one or more touchdowns and liftoffs of touch devices, so long as at least one touch device remains touched down on or near the surface, wherein a second set includes a contact corresponding to each of the one or more remaining touch devices;

determining a motion based on the tracking, wherein the motion is based on the initial set prior to the change in the number of contacts and is based on the second set after the change in the number of contacts; and

based on the motion, generating at least one input event of the selected activity.

2. The method as recited in claim 1 wherein the one or more input events includes one of the operations of generating symbols, performing commands, and performing manipulation operations.

3. The method as recited in claim 1 wherein the activity is identified based on the contacts in the initial set touching down synchronously upon the surface.

4. The method as recited in claim 1 further comprising associating a graphical object to the input event.

5. The method as recited in claim 4 further comprising displaying the graphical object.

6. The method as recited in claim 1 further comprising displaying a graphical object and subsequently manipulating the graphical object based on the input event.

7. The method as recited in claim 1 wherein two input events of the selected activity are generated concurrently based on the motion.

8. The method as recited in claim 1 wherein a first input event of the selected activity is generated based on a first component of the motion and a second input event of the selected activity is generated based on a second component of the motion.

9. The method as recited in claim 1 wherein the activity is selected based on tracked motions of the contacts in the initial set occurring synchronously.

10. The method as recited in claim 9 wherein the contacts in the initial set touch down asynchronously.

11. The method as recited in claim 1 wherein the second set includes additional one or more contacts that touch down after the initial set is identified.

12. The method as recited in claim 11 wherein the additional one or more contacts access a greater dynamic range of motion than the contacts in the initial set.

13. The method as recited in claim 1 wherein the second set includes a single contact.

14. The method as recited in claim 13 wherein the single contact is not one of the contacts in the initial set.

15. The method as recited in claim 1 further comprising:

identifying a third set of contacts that touch down synchronously while the selected activity corresponding to the initial set is being maintained; and

changing the activity to a second activity corresponding to the third set.

16. The method as recited in claim 1 wherein the at least one input event includes one of a scaling manipulation, a rotation manipulation, and a translation manipulation.

17. The method as recited in claim 1 wherein the initial set is a chord of contacts.

18. A non-transitory computer readable storage medium having stored therein instructions, which when executed by a computing device with a touch surface, cause the computing device to perform a method of processing touch inputs of touch devices that touchdown and liftoff on or near the touch sensitive surface, the method comprising:

reading data from the touch surface to obtain positions of contacts corresponding to the touch devices on or near the surface wherein the data is read to enable simultaneous detection of multiple distinguishable contacts;

tracking the contacts based on the positions;

selecting an activity corresponding to an initial set of at least two of the contacts corresponding to at least two of the touch devices simultaneously on or near the surface, wherein the activity corresponds to one or more input events of the computing device;

maintaining the selection of the activity when the number of contacts on or near the surface changes due to one or more touchdowns and liftoffs of touch devices, so long as at least one touch device remains touched down on or near the surface, wherein a second set includes a contact corresponding to each of the one or more remaining touch devices;

determining a motion based on the tracking, wherein the motion is based on the initial set prior to the change in the number of contacts and is based on the second set after the change in the number of contacts; and

based on the motion, generating at least one input event of the selected activity.

19. The computer readable storage medium as recited in claim 18 wherein the one or more input events includes one of an operation for generating symbols, performing commands, and performing manipulation operations.

20. The computer readable storage medium as recited in claim 18 wherein the method further comprises associating a graphical object to the input event.

21. The computer readable storage medium as recited in claim 20 wherein the method further comprises causing the graphical object to be displayed.

22. The computer readable storage medium as recited in claim 18 wherein the method further comprises causing a graphical object to be displayed and subsequently causing the graphical object to be manipulated based on the input event.

23. The computer readable storage medium as recited in claim 18 wherein two input events of the selected activity are generated based on the motion.

24. The computer readable storage medium as recited in claim 18 wherein a first input event of the selected activity is generated based on a first component of the motion and a second input event of the selected activity is generated based on a second component of the motion.

25. The computer readable storage medium as recited in claim 18 wherein the contacts in the initial set touch down synchronously upon the surface.

26. The computer readable storage medium as recited in claim 18 wherein the activity is selected based on tracked motions of the contacts in the initial set occurring synchronously.

27. The computer readable storage medium as recited in claim 26 wherein the contacts in the initial set touch down asynchronously.

28. The computer readable storage medium as recited in claim 18 wherein the second set includes additional contacts that touch down after the initial set is identified.

29. The computer readable storage medium as recited in claim 28 wherein the additional one or more contacts access a greater dynamic range of motion than the contacts in the initial set.

30. The computer readable storage medium as recited in claim 18 wherein the second set includes a single contact.

31. The computer readable storage medium as recited in claim 30 wherein the single contact is not one of the contacts in the initial set.

32. The computer readable storage medium as recited in claim 18 further comprising:

identifying a third set of contacts that touch down synchronously while the selected activity corresponding to the initial set is being maintained; and

changing the activity to a second activity corresponding to the third set.

33. The computer readable storage medium as recited in claim 18 wherein the at least one input event includes one of a scaling manipulation, a rotation manipulation, and a translation manipulation.

34. The computer readable storage medium as recited in claim 18 wherein the initial set is a chord of contacts.

35. An apparatus including a touch sensitive surface with an associated plurality of touch sensors, the apparatus comprising:

a sensor data reader that obtains position data of contacts corresponding to at least two touch devices on or near the surface wherein the data is read to enable simultaneous detection of multiple distinguishable contacts;

a tracking module that tracks the contacts based on the position data;

an activity selector that selects an activity corresponding to an initial set of at least two of the contacts corresponding to at least two of the touch devices simultaneously on or near the surface, wherein the activity corresponds to one or more input events, and maintains the selection of the activity when the number of contacts on or near the surface changes due to one or more touchdowns and liftoffs of touch devices, so long as at least one touch device remains touched down on or near the surface, wherein a second set includes a contact corresponding to each of the one or more remaining touch devices;

a motion determiner that determines a motion based on the tracking, wherein the motion is based on the initial set prior to the change in the number of contacts and is based on the second set after the change in the number of contacts; and

a touch event recognizer that generates, based on the motion, at least one input event of the selected activity.

36. The apparatus as recited in claim 35 wherein the one or more input events includes one of the operations of generating symbols, performing commands, and performing manipulation operations.

37. The apparatus as recited in claim 35 wherein the activity is selected based on the contacts in the initial set touching down synchronously upon the surface.

38. The apparatus as recited in claim 35 further comprising a display screen for enabling manipulating of graphical objects on the screen in response to the input event.

39. The apparatus as recited in claim 35 further comprising a display screen for enabling displaying a graphical object and subsequently manipulating the graphical object based on the input event.

40. The apparatus as recited in claim 35 wherein two input events of the selected activity are generated concurrently based on the motion.

41. The apparatus as recited in claim 35 wherein the initial set is a chord of contacts.

42. A computing system including a touch sensitive surface with an associated plurality of touch sensors, the system comprising:

means for reading data from the touch sensors to obtain position data of contacts corresponding to at least two touch devices on or near the surface wherein the data is read to enable simultaneous detection of multiple distinguishable contacts;

means for tracking contacts based on the position data;

means for selecting an activity corresponding to an initial set of at least two of the contacts corresponding to at least two of the touch devices simultaneously on or near the touch surface, wherein the activity corresponds to one or more input events, and maintaining the selection of the activity when the number of contacts on or near the surface changes due to one or more touchdowns and liftoffs of touch devices, so long as at least one touch device remains touched down on or near the surface, wherein a second set includes a contact corresponding to each of the one or more remaining touch devices;

means for determining a motion based on the tracking, wherein the motion is based on the initial set prior to the change in the number of contacts and is based on the second set after the change in the number of contacts; and

means for generating, based on the motion, at least one input event of the selected activity.

43. The computing system as recited in claim 42 wherein the initial set is a chord of contacts.