Multi-touch contact tracking using predicted paths

• US 8,334,846 B2
• Filed: 11/14/2006
• Issued: 12/18/2012
• Est. Priority Date: 01/26/1998
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne Carl, Elias, John G.
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): Amadiz, Rodney
• Application Number:
  US11/559,736
• Publication Number:
  US 20070081726A1
• Time in Patent Office: 2,226 Days
• Field of Search: 345156-184, 178 1801- 2004
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method for determining which of a plurality of concurrent surface contacts of touch devices on or near a touch sensing surface in a current proximity image corresponds to a surface contact in a prior proximity image, the method comprising:

generating a first predicted path by predicting a continuation of an existing path of the surface contact in the prior proximity image;

generating one or more other predicted paths of one or more other surface contacts in the prior proximity image;

finding a closest surface contact of the plurality of surface contacts in the current proximity image, the closest surface contact being the surface contact closest to the first predicted path; and

pairing the closest surface contact with the first predicted path if the first predicted path is closer to the closest surface contact than each of the one or more other predicted paths.

3. The method of claim 1 or 2 further comprising:

starting new paths for unpaired surface contacts.

4. The method of claim 3 further comprising:

deactivating unpaired predicted paths.

5. The method of claim 1 or 2 further comprising:

deactivating unpaired predicted paths.

6. The method of claim 1 or 2 further comprising:

updating a path parameter for the first predicted path from a measured parameter of the surface contact paired with the first predicted path.

7. The method of claim 6 further comprising:

starting new paths for unpaired surface contacts.

8. The method of claim 7 further comprising:

deactivating unpaired predicted paths.

9. The method of claim 6 further comprising:

deactivating unpaired predicted paths.

10. The method of claim 1 or 2 wherein generating the first predicted path comprises determining a velocity of the surface contact along the existing path.

11. The method of claim 10 further comprising:

starting new paths for unpaired surface contacts.

12. The method of claim 11 further comprising:

deactivating unpaired predicted paths.

13. The method of claim 10 further comprising:

deactivating unpaired predicted paths.

14. The method of claim 10 further comprising:

updating a path parameter for the first predicted path from a measured parameter of the surface contact paired with the first predicted path.

15. The method of claim 14 further comprising:

starting new paths for unpaired surface contacts.

16. The method of claim 15 further comprising:

deactivating unpaired predicted paths.

17. The method of claim 14 further comprising:

deactivating unpaired predicted paths.

2. A method for determining which of a plurality of concurrent surface contacts of touch devices on or near a touch sensing surface in a current proximity image corresponds to a surface contact in a prior proximity image, the method comprising:

generating a first predicted path by predicting a continuation of an existing path of the surface contact in the prior proximity image;

generating one or more other predicted paths of one or more other surface contacts in the prior proximity image;

finding a closest surface contact of the plurality of surface contacts in the current proximity image, the closest surface contact being the surface contact closest to the first predicted path and within a path-dependent tracking radius of the first predicted path; and

pairing the closest surface contact with the first predicted path if the first predicted path is closer to the closest surface contact than each of the one or more other predicted paths, and if the centroid of the closest surface contact is within the path-dependent tracking radius of the first predicted path.

18. A method for determining which of a plurality of concurrent groups of pixels in a current proximity image corresponds to a group of pixels in a prior proximity image, each group of pixels corresponding to a distinguishable hand part or other touch object on or near a surface of a multi-touch apparatus and each proximity image representing a scan of a plurality of proximity sensors of the multi-touch apparatus, the method comprising:

predicting a first predicted path of one of the groups of pixels in the prior proximity image;

predicting one or more other predicted paths of one or more other groups of pixels in the prior proximity image;

finding a closest group of pixels of the plurality of groups of pixels in the current proximity image, the closest group of pixels being the group of pixels closest to the first predicted path; and

pairing the closest group of pixels with the first predicted path if the first predicted path is closer to the closest group of pixels than each of the one or more other predicted paths,

whereby the first predicted path may be used to generate user interface interactions in response to motion of the distinguishable hand part or other touch object from the prior proximity image to the current proximity image.

19. The method of claim 18 further comprising:

starting new paths for unpaired groups of pixels.

20. The method of claim 19 further comprising:

deactivating unpaired predicted paths.

21. The method of claim 18 further comprising:

deactivating unpaired predicted paths.

22. The method of claim 18 wherein at least one of the groups of pixels corresponds to a hand part near but not on the multi-touch surface.

23. The method of claim 22 further comprising:

starting new paths for unpaired groups of pixels, wherein at least one unpaired group of pixels corresponds to a hand part near but not on the multi-touch surface.