Contact tracking and identification module for touch sensing

• US 9,804,701 B2
• Filed: 04/10/2009
• Issued: 10/31/2017
• Est. Priority Date: 01/26/1998
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne, Elias, John G.
• Primary Examiner(s): SIM, MATTHEW Y
• Application Number:
  US12/422,225
• Publication Number:
  US 20090251439A1
• Time in Patent Office: 3,126 Days
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method for providing input to a computing system including a touch sensitive surface, the method comprising:

obtaining a sequence of proximity images of a set of one or more contacts, wherein each of the contacts corresponds to a touch object on or near the surface, the proximity images including proximity information of each contact at different times, wherein each contact appears in each of the proximity images;

determining a first proximity of each contact based on a first proximity image of the sequence of proximity images;

determining a second proximity of each contact based on a second proximity image of the sequence of proximity images, the second proximity image occurring after the first proximity image;

calculating, based on the first and second proximities, a proximity change between the first proximity image and the second proximity image of each of one or more of the contacts, the proximity change including an indication of a motion of a first contact of the set of the one or more contacts; and

providing, based on the one or more proximity changes, input to the computing system, wherein providing the input to the computing system includes modifying the motion of the first contact to generate a modified motion of the first contact, the modified motion of the first contact different than the motion of the first contact indicated by the proximity change.

2. The method of claim 1, further comprising:

generating the sequence of proximity images by scanning a plurality of capacitive sensors associated with the touch sensitive surface.

3. The method of claim 1, wherein calculating the proximity change of each of the one or more contacts includes calculating the proximity change of the first contact, and modifying the motion of the first contact includes modifying the motion of the first contact based on the proximity change of the first contact.

4. The method of claim 1, wherein modifying the motion of the first contact includes downscaling a lateral velocity of the first contact.

5. The method of claim 4, wherein downscaling the lateral velocity of the first contact includes downscaling the lateral velocity in proportion to the proximity change of the first contact.

6. The method of claim 4, further comprising:

generating a cursor input based on the lateral velocity of the first contact, wherein a motion of the cursor is downscaled based on the downscaling of the lateral velocity of the first contact.

7. The method of claim 1, wherein calculating the proximity change of each of the one or more contacts includes calculating the proximity change of one or more contacts in the set other than the first contact, and modifying the motion of the first contact includes modifying the motion of the first contact based on the proximity change of the one or more other contacts.

8. The method of claim 7, wherein calculating the proximity change of each of the one or more contacts includes calculating the proximity changes of all of the contacts in the set, and modifying the motion of the first contact includes downscaling a lateral velocity of the first contact based on a maximum of the proximity changes of the contacts of the set.

9. The method of claim 1, wherein the set consists of a single contact.

10. The method of claim 1, further comprising:

calculating a rate of each of the one or more proximity changes, wherein providing the input to the computing system based on the one or more proximity changes includes providing the input based on the rate of each of the one or more proximity changes.

11. The method of claim 10, further comprising:

determining whether the rate of proximity change of at least one of the contacts is less than a predetermined threshold, wherein providing the input to the computer system includes providing the input based on the determination of whether the rate of proximity change is less than the predetermined threshold.

12. The method of claim 1, wherein providing the input to the computing system includes invoking a process of the computing system.

13. The method of claim 12, wherein invoking the process of the system includes invoking a process that tests a motion of one or more of the contacts.

14. The method of claim 13, wherein the motion of one or more of the contacts includes a lateral motion, and the process that tests the lateral motion includes a process that test the lateral motion for deceleration.

15. A non-transitory computer readable storage medium having instructions stored therein, which when executed by an apparatus, the apparatus including a touch sensitive surface that provides input to a computing system, cause the apparatus to perform a method comprising:

obtaining a sequence of proximity images of a set of one or more contacts, wherein each of the contacts corresponds to a touch object on or near the surface, the proximity images including proximity information of each contact at different times, wherein each contact appears in each of the proximity images;

determining a first proximity of each contact based on a first proximity image of the sequence of proximity images;

determining a second proximity of each contact based on a second proximity image of the sequence of proximity images, the second proximity image occurring after the first initial proximity image;

calculating, based on the first and second proximities, a proximity change between the first proximity image and the second proximity image of each of one or more of the contacts, the proximity change including an indication of a motion of a first contact of the set of the one or more contacts; and

providing, based on the one or more proximity changes, input to the computing system, wherein providing the input to the computing system includes modifying the motion of the first contact to generate a modified motion of the first contact, the modified motion of the first contact different than the motion of the first contact indicated by the proximity change.

16. The non-transitory computer readable storage medium of claim 15, the method further comprising:

generating the sequence of proximity images by scanning a plurality of capacitive sensors associated with the touch sensitive surface.

17. The non-transitory computer readable storage medium of claim 15, wherein calculating the proximity change of each of the one or more contacts includes calculating the proximity change of the first contact, and modifying the motion of the first contact includes modifying the motion of the first contact based on the proximity change of the first contact.

18. The non-transitory computer readable storage medium of claim 15, wherein modifying the motion of the first contact includes downscaling a lateral velocity of the first contact.

19. The non-transitory computer readable storage medium of claim 18, wherein downscaling the lateral velocity of the first contact includes downscaling the lateral velocity in proportion to the proximity change of the first contact.

20. The non-transitory computer readable storage medium of claim 18, the method further comprising:

generating a cursor input based on the lateral velocity of the first contact, wherein a motion of the cursor is downscaled based on the downscaling of the lateral velocity of the first contact.

21. The non-transitory computer readable storage medium of claim 15, wherein calculating the proximity change of each of the one or more contacts includes calculating the proximity change of one or more contacts in the set other than the first contact, and modifying the motion of the first contact includes modifying the motion of the first contact based on the proximity change of the one or more other contacts.

22. The non-transitory computer readable storage medium of claim 21, wherein calculating the proximity change of each of the one or more contacts includes calculating the proximity changes of all of the contacts in the set, and modifying the motion of the first contact includes downscaling a lateral velocity of the first contact based on a maximum of the proximity changes of the contacts of the set.

23. The non-transitory computer readable storage medium of claim 15, wherein the set consists of a single contact.

24. The non-transitory computer readable storage medium of claim 15, the method further comprising:

calculating a rate of each of the one or more proximity changes, wherein providing the input to the computing system based on the one or more proximity changes includes providing the input based on the rate of each of the one or more proximity changes.

25. The non-transitory computer readable storage medium of claim 24, the method further comprising:

determining whether the rate of proximity change of at least one of the contacts is less than a predetermined threshold, wherein providing the input to the computer system includes providing the input based on the determination of whether the rate of proximity change is less than the predetermined threshold.

26. The non-transitory computer readable storage medium of claim 15, wherein providing the input to the computing system includes invoking a process of the computing system.

27. The non-transitory computer readable storage medium of claim 26, wherein invoking the process of the system includes invoking a process that tests a motion of one or more of the contacts.

28. The non-transitory computer readable storage medium of claim 27, wherein the motion of one or more of the contacts includes a lateral motion, and the process that tests the lateral motion includes a process that test the lateral motion for deceleration.

29. An apparatus for providing input to a computing system including a touch sensitive surface, the apparatus comprising:

a processor; and

a non-transitory computer readable storage medium coupled to the processor and storing instructions that, when executed by the processor, cause the processor to;

obtain a sequence of proximity images of a set of one or more contacts, wherein each contact corresponds to a touch object on or near the touch sensitive surface, the proximity images including proximity information of each contact at different times, wherein each contact appears in each of the proximity images,

determine a first proximity of each contact based on a first proximity image of the sequence of proximity images,

determine a second proximity of each contact based on a second proximity image of the sequence of proximity images, the second proximity image occurring after the first proximity image, and

calculate, based on the first and second proximities, a proximity change between the first proximity image and the second proximity image of each of one or more of the contacts, the proximity change including an indication of a motion of a first contact of the set of the one or more contacts; and

provide, based on the one or more proximity changes, input to the computing system, wherein providing the input to the computing system includes modifying the motion of the first contact to generate a modified motion of the first contact, the modified motion of the first contact different than the motion of the first contact indicated by the proximity change.

30. The apparatus of claim 29, further comprising:

a scanning system that generates the proximity images by scanning a plurality of capacitive sensors associated with the touch sensitive surface.

31. The apparatus of claim 29, wherein calculating the proximity change of each of the one or more contacts includes calculating the proximity change of the first contact, and modifying the motion of the first contact includes modifying the motion of the first contact based on the proximity change of the first contact.

32. The apparatus of claim 29, wherein modifying the motion of the first contact includes downscaling a lateral velocity of the first contact.

33. The apparatus of claim 32, wherein downscaling the lateral velocity of the first contact includes downscaling the lateral velocity in proportion to the proximity change of the first contact.

34. The apparatus of claim 32, wherein the processor generates a cursor input based on the lateral velocity of the first contact, wherein a motion of the cursor is downscaled based on the downscaling of the lateral velocity of the first contact.

35. The apparatus of claim 29, wherein calculating the proximity change of each of the one or more contacts includes calculating the proximity change of one or more contacts in the set other than the first contact, and modifying the motion of the first contact includes modifying the motion of the first contact based on the proximity change of the one or more other contacts.

36. The apparatus of claim 35, wherein calculating the proximity change of each of the one or more contacts includes calculating the proximity changes of all of the contacts in the set, and modifying the motion of the first contact includes downscaling a lateral velocity of the first contact based on a maximum of the proximity changes of the contacts of the set.

37. The apparatus of claim 29, wherein the set consists of a single contact.

38. The apparatus of claim 29, wherein the instructions further cause the processor to calculate a rate of each of the one or more proximity changes, wherein providing the input to the computing system based on the one or more proximity changes includes providing the input based on the rate of each of the one or more proximity changes.

39. The apparatus of claim 38, wherein the instructions further cause the processor to determine whether the rate of proximity change of at least one of the contacts is less than a predetermined threshold, wherein providing the input to the computer system includes providing the input based on the determination of whether the rate of proximity change is less than the predetermined threshold.

40. The apparatus of claim 29, wherein providing the input to the computing system includes invoking a process of the computing system.

41. The apparatus of claim 40, wherein invoking the process of the system includes invoking a process that tests a motion of one or more of the contacts.

42. The apparatus of claim 41, wherein the motion of one or more of the contacts includes a lateral motion, and the process that tests the lateral motion includes a process that test the lateral motion for deceleration.

43. A method for providing input to a computing system including a touch sensitive surface, the method comprising:

obtaining a sequence of proximity images of a set of one or more contacts, wherein each of the contacts corresponds to a touch object on or near the surface, the proximity images including proximity information of each contact at different times, wherein each contact appears in each of the proximity images;

determining a first proximity of each contact based on a first proximity image of the sequence of proximity images;

determining a second proximity of each contact based on a second proximity image of the sequence of proximity images, the second proximity image occurring after the first proximity image;

calculating, based on the first and second proximities, a proximity change of each of one or more of the contacts, the proximity change including an indication of a magnitude and direction of a motion of a first contact of the set of the one or more contacts; and

providing, based on the one or more proximity changes, input to the computing system, wherein providing the input to the computing system includes modifying the magnitude and direction of the motion of the first contact to generate a modified magnitude and direction of motion of the first contact, the modified magnitude and direction of motion of the first contact different than the magnitude and direction of motion of the first contact indicated by the proximity change.