Contact tracking and identification module for touch sensing

• US 8,466,881 B2
• Filed: 04/10/2009
• Issued: 06/18/2013
• Est. Priority Date: 01/26/1998
• Status: Expired due to Fees

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne Carl, Elias, John Greer
• Primary Examiner(s): Sim, Yong H
• Application Number:
  US12/422,212
• Publication Number:
  US 20090251438A1
• Time in Patent Office: 1,530 Days
• Field of Search: 345/156, 345168-179, 178 1801- 1807, 341 22- 26
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method for providing input to a computing device having an associated touch sensitive surface, the method comprising:

obtaining tracking information of a plurality of contacts, each contact corresponding to a touch object being tracked on or near the touch sensitive surface, the tracking information including motion information of individual contacts within the plurality of contacts;

determining a first component of motion of the plurality of contacts, the first component of motion being based on the motion information of each contact in a first set of one or more contacts of the plurality of contacts;

generating a first input of the computing device based on the first component of motion of the plurality of contacts; and

generating a second input of the computing device based on the motion information of each contact of a second set of one or more contacts of the plurality of contacts, wherein at least a portion of the generating of the first input is concurrent with at least a portion of the generating of the second input.

2. The method as recited in claim 1, further comprising:

tracking the plurality of contacts, wherein the tracking is based on a sequence of proximity images obtained by scanning a plurality of capacitive sensors associated with the touch sensitive surface.

3. The method as recited in claim 1, wherein the first input includes a graphical manipulation of an object displayed on a display.

4. The method as recited in claim 3 wherein the graphical manipulation includes one of a scaling manipulation and a rotation manipulation.

5. The method as recited in claim 4, wherein the second input includes a translation manipulation of the object.

6. The method as recited in claim 1, wherein the first set includes a plurality of the contacts, and determining the first component of motion includes performing one of contact translation weighting and contact velocity component nonlinear scaling depending on relative speed between contacts.

7. The method as recited in claim 1, wherein the first and second sets are different.

8. The method as recited in claim 7, wherein the first set does not include any of the one or more contacts of the second set, and the second subset does not include any of the one or more contacts of the first subset.

9. The method as recited in claim 8, wherein the first input includes a graphical manipulation of an object displayed on a display, the graphical manipulation including one of a scaling manipulation and a rotation manipulation, and the second input includes a translation manipulation of the object.

10. The method as recited in claim 9, wherein the first set consists of an innermost contact and an outermost contact, and the second set includes one or more central contacts.

11. The method as recited in claim 7, wherein the first set consists of a single contact.

12. The method as recited in claim 11, wherein the first component of motion is rotational motion, and determining the rotational motion includes extracting the rotational motion from an orientation change of the single contact.

13. The method as recited in claim 12, further comprising:

identifying one of the contacts as a thumb contact; and

selecting the thumb contact as the single contact.

14. The method as recited in claim 12, further comprising:

determining a second component of motion of the plurality of contacts, the second component of motion being based on the motion information of each contact in the second set, wherein the second input is generated based on the second component of motion, and wherein the second component of motion includes one of translational motion and scaling motion.

15. The method as recited in claim 7, wherein one of the first and second sets includes a plurality of contacts.

16. The method as recited in claim 15, further comprising:

identifying an innermost contact and an outermost contact of the plurality of contacts;

identifying one or more central contacts of the plurality of contacts;

selecting the innermost and outermost contacts as the contacts in the first set; and

selecting the one or more central contacts as the contacts in the second set.

17. The method as recited in claim 7, wherein one of the first and second sets includes all of the contacts being tracked.

18. The method as recited in claim 1, further comprising:

determining a second component of motion of the plurality of contacts, the second component of motion being based on the motion information of each contact in the second set, wherein the second input is generated based on the second component of motion.

19. The method as recited in claim 18, wherein the first and second sets are different.

20. The method as recited in claim 19, wherein the first set includes a plurality of contacts, and the second set includes all of the contacts being tracked.

21. The method as recited in claim 20, wherein the first component of motion includes one of rotational motion and scaling motion, and the second component of motion includes translational motion.

22. The method as recited in claim 1, wherein one of the first and second inputs includes one of a cut command, a copy command, a paste command, a cursor motion command, a drag command, and a scrolling command.

23. An apparatus for providing input to a computing device having an associated touch sensitive surface, the apparatus comprising:

a tracking module that obtains tracking information of a plurality of contacts, each contact corresponding to a touch object being tracked on or near the touch sensitive surface, the tracking information including motion information of individual contacts within the plurality of contacts;

a motion component extraction module that determines a first component of motion of the plurality of contacts, the first component of motion being based on the motion information of each contact in a first set of one or more contacts of the plurality of contacts; and

an input generator that

generates a first input of the computing device based on the first component of motion of the plurality of contacts, and

generates a second input of the computing device based on the motion information of each contact of a second set of one or more contacts of the plurality of contacts, wherein at least a portion of the generating of the first input is concurrent with at least a portion of the generating of the second input.

24. The apparatus as recited in claim 23, further comprising:

a contact tracking system that tracks the plurality of contacts, wherein the tracking is based on a sequence of proximity images obtained by scanning a plurality of capacitive sensors associated with the touch sensitive surface.

25. The apparatus as recited in claim 23, wherein the first input includes a graphical manipulation of an object displayed on a display.

26. The apparatus as recited in claim 25 wherein the graphical manipulation includes one of a scaling manipulation and a rotation manipulation.

27. The apparatus as recited in claim 26, wherein the second input includes a translation manipulation of the object.

28. The apparatus as recited in claim 23, wherein the first set includes a plurality of the contacts, and determining the first component of motion includes performing one of contact translation weighting and contact velocity component nonlinear scaling depending on relative speed between contacts.

29. The apparatus as recited in claim 23, wherein the first and second sets are different.

30. The apparatus as recited in claim 29, wherein the first set does not include any of the one or more contacts of the second set, and the second subset does not include any of the one or more contacts of the first subset.

31. The apparatus as recited in claim 30, wherein the first input includes a graphical manipulation of an object displayed on a display, the graphical manipulation including one of a scaling manipulation and a rotation manipulation, and the second input includes a translation manipulation of the object.

32. The apparatus as recited in claim 31, wherein the first set consists of an innermost contact and an outermost contact, and the second set includes on or more central contacts.

33. The apparatus as recited in claim 29, wherein the first set consists of a single contact.

34. The apparatus as recited in claim 33, wherein the first component of motion is rotational motion, and determining the rotational motion includes extracting the rotational motion from an orientation change of the single contact.

35. The apparatus as recited in claim 34, further comprising:

a contact identification module that identifies one of the contacts as a thumb contact, wherein the input generator selects the thumb contact as the single contact.

36. The apparatus as recited in claim 34, wherein the motion component extraction module further determines a second component of motion of the plurality of contacts, the second component of motion being based on the motion information of each contact in the second set, wherein the input generator generates the second input based on the second component of motion, and wherein the second component of motion includes one of translational motion and scaling motion.

37. The apparatus as recited in claim 29, wherein one of the first and second sets includes all of the contacts being tracked.

38. The apparatus as recited in claim 29, wherein one of the first and second sets includes a plurality of contacts.

39. The apparatus as recited in claim 38, further comprising:

a contact identification module that identifies an innermost contact and an outermost contact of the plurality of contacts, and identifies one or more central contacts of the plurality of contacts, wherein the input generator selects the innermost and outermost contacts as the contacts in the first set, and selects the one or more central contacts as the contacts in the second set.

40. The apparatus as recited in claim 23, wherein the motion component extraction module further determines a second component of motion of the plurality of contacts, the second component of motion being based on the motion information of each contact in the second set, wherein the input generator generates the input based on the second component of motion.

41. The apparatus as recited in claim 40, wherein the first and second sets are different.

42. The apparatus as recited in claim 41, wherein the first set includes a plurality of contacts, and the second set includes all of the contacts being tracked.

43. The apparatus as recited in claim 42, wherein the first component of motion includes one of rotational motion and scaling motion, and the second component of motion includes translational motion.

44. The apparatus as recited in claim 23, wherein one of the first and second inputs includes on of a cut command, a copy command, a paste command, a cursor motion command, a drag command, and a scrolling command.

45. A non-transitory computer readable storage medium having instructions stored therein, which when executed by an apparatus, the apparatus including a touch sensitive surface that provides input to a computing system, cause the apparatus to:

obtain tracking information of a plurality of contacts, each contact corresponding to a touch object being tracked on or near the touch sensitive surface, the tracking information including motion information of individual contacts within the plurality of contacts;

determine a first component of motion of the plurality of contacts, the first component of motion being based on the motion information of each contact in a first set of one or more contacts of the plurality of contacts;

generate a first input of the computing system based on the first component of motion of the plurality of contacts; and

generate a second input of the computing system based on the motion information of each contact of a second set of one or more contacts of the plurality of contacts, wherein at least a portion of the generating of the first input is concurrent with at least a portion of the generating of the second input.

46. The non-transitory computer readable storage medium as recited in claim 45 wherein the instructions further cause the apparatus to track the plurality of contacts, wherein the tracking is based on a sequence of proximity images obtained by scanning a plurality of capacitive sensors associated with the touch sensitive surface.

47. The non-transitory computer readable storage medium as recited in claim 45 wherein the first input includes a graphical of an object displayed on a display.

48. The non-transitory computer readable storage medium as recited in claim 47 wherein the graphical manipulation includes one of a scaling manipulation and a rotation manipulation.

49. The non-transitory computer readable storage medium as recited in claim 48, wherein the second input includes a translation manipulation of the object.

50. The non-transitory computer readable storage medium as recited in claim 45, wherein the first set includes a plurality of the contacts, and determining the first component of motion includes performing one of contact translation weighting and contact velocity component nonlinear scaling depending on relative speed between contacts.

51. The non-transitory computer readable storage medium as recited in claim 45, wherein the first and second sets are different.

52. The non-transitory computer readable storage medium as recited in claim 51, wherein the first set does not include any of the one or more contacts of the second set, and the second subset does not include any of the one or more contacts of the first subset.

53. The non-transitory computer readable storage medium as recited in claim 52, wherein the first input includes a graphical manipulation of an object displayed on a display, the graphical manipulation including one of a scaling manipulation and a rotation manipulation, and the second input includes a translation manipulation of the object.

54. The non-transitory computer readable storage medium as recited in claim 53, wherein the first set consists of an innermost contact and an outermost contact, and the second set includes one or more central contacts.

55. The non-transitory computer readable storage medium as recited in claim 51, wherein the first set consists of a single contact.

56. The non-transitory computer readable storage medium as recited in claim 55, wherein the first component of motion is rotational motion, and determining the rotational motion includes extracting the rotational motion from an orientation change of the single contact.

57. The non-transitory computer readable storage medium as recited in claim 56, wherein the instructions further cause the apparatus to:

identify one of the contacts as a thumb contact; and

select the thumb contact as the single contact.

58. The non-transitory computer readable storage medium as recited in claim 56, wherein the instructions further cause the apparatus to:

determine a second component of motion of the plurality of contacts, the second component of motion being based on the motion information of each contact in the second set, wherein the second input is generated based on the second component of motion, and wherein the second component of motion includes one of translational motion and scaling motion.

59. The non-transitory computer readable storage medium as recited in claim 51, wherein one of the first and second sets includes a plurality of contacts.

60. The non-transitory computer readable storage medium as recited in claim 59, wherein the instructions further cause the apparatus to:

identify an innermost contact and an outermost contact of the plurality of contacts;

identify one or more central contacts of the plurality of contacts;

select the innermost and outermost contacts as the contacts in the first set; and

select the one or more central contacts as the contacts in the second set.

61. The non-transitory computer readable storage medium as recited in claim 51, wherein one of the first and second sets includes all of the contacts being tracked.

62. The non-transitory computer readable storage medium as recited in claim 45, wherein the instructions further cause the apparatus to:

determine a second component of motion of the plurality of contacts, the second component of motion being based on the motion information of each contact in the second set, wherein the second input is generated based on the second component of motion.

63. The non-transitory computer readable storage medium as recited in claim 62, wherein the first and second sets are different.

64. The non-transitory computer readable storage medium as recited in claim 63, wherein the first set includes a plurality of contacts, and the second set includes all of the contacts being tracked.

65. The non-transitory computer readable storage medium as recited in claim 64, wherein the first component of motion includes one of rotational motion and scaling motion, and the second component of motion includes translational motion.

66. The non-transitory computer readable storage medium as recited in claim 45, wherein one of the first and second inputs includes one of a cut command, a copy command, a paste command, a cursor motion command, a drag command, and a scrolling command.