Identifying contacts on a touch surface

• US 9,383,855 B2
• Filed: 06/13/2008
• Issued: 07/05/2016
• Est. Priority Date: 01/26/1998
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne, Elias, John G.
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): Rodriguez, Joseph G
• Application Number:
  US12/139,411
• Publication Number:
  US 20090021489A1
• Time in Patent Office: 2,944 Days
• Field of Search: 345/173, 382/172, 382/216
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method for segmenting contacts in a proximity image, the method comprising:

obtaining a proximity image, the proximity image including a plurality of proximity values, each proximity value representing a measurement of touch object proximity at a location within a sensing area of a touch sensing surface, each location being associated with a first coordinate in a first dimension of the sensing surface and a second coordinate in a second dimension of the sensing surface;

locating a local maximum proximity value in the proximity image; and

after locating the local maximum proximity value, determining a two-dimensional boundary of a contact, the boundary surrounding the local maximum proximity value, including a two-dimensional search that applies a first boundary test including comparing a first proximity value to a second proximity value adjacent to the first proximity value in the first dimension, comparing the first proximity value to a third proximity value adjacent to the first proximity value in the second dimension, and comparing the third proximity value to a fourth proximity value adjacent to the third proximity value in the first dimension.

2. The method of claim 1 further comprising forming groups from those proximity values surrounding each local maximum proximity value up to and including the corresponding contact boundary proximity values.

3. The method of claim 2 further comprising combining groups of proximity values that partially overlap.

4. The method of claim 3 further comprising extracting group positions and features by fitting an ellipse to each group of proximity values.

5. The method of claim 2 further comprising extracting group positions and features by fitting an ellipse to each group of proximity values.

6. The method of claim 1 further comprising locating all local maximum proximity values in the proximity image prior to determining the two-dimensional boundary of the contact.

7. An apparatus comprising:

a touch sensing surface; and

an input system that

obtains a proximity image, the proximity image including a plurality of proximity values, each proximity value representing a measurement of touch object proximity at a location within a sensing area of the touch sensing surface, each location being associated with a first coordinate in a first dimension of the sensing surface and a second coordinate in a second dimension of the sensing surface;

locates a local maximum proximity value in the proximity image; and

after locating the local proximity value, determines a two-dimensional boundary of a contact, the boundary surrounding the local maximum proximity value, including a two-dimensional search that applies a first boundary test including comparing a first proximity value to a second proximity value adjacent to the first proximity value in the first dimension, comparing the first proximity value to a third proximity value adjacent to the first proximity value in the second dimension, and comparing the third proximity value to a fourth proximity value adjacent to the third proximity value in the first dimension.

8. The apparatus of claim 7, wherein the input system forms groups from those proximity values surrounding each local maximum proximity value up to and including the corresponding contact boundary proximity values.

9. The apparatus of claim 8, wherein the input system combines groups of proximity values that partially overlap.

10. The apparatus of claim 9, wherein the input system extracts group positions and features by fitting an ellipse to each group of proximity values.

11. The apparatus of claim 8, wherein the input system extracts group positions and features by fitting an ellipse to each group of proximity values.

12. The apparatus of claim 7, wherein the input system locates all local maximum proximity values in the proximity image prior to determining the two-dimensional boundary of the contact.

13. A non-transitory computer-readable storage medium storing computer-readable program instructions executable to perform a method of segmenting contacts in a proximity image, the method comprising:

obtaining a proximity image, the proximity image including a plurality of proximity values, each proximity value representing a measurement of touch object proximity at a location within a sensing area of a touch sensing surface, each location being associated with a first coordinate in a first dimension of the sensing surface and a second coordinate in a second dimension of the sensing surface;

locating a local maximum proximity value in the proximity image; and

after locating the local maximum proximity value, determining a two-dimensional boundary of a contact, the boundary surrounding the local maximum proximity value, including a two-dimensional search that applies a first boundary test including comparing a first proximity value to a second proximity value adjacent to the first proximity value in the first dimension, comparing the first proximity value to a third proximity value adjacent to the first proximity value in the second dimension, and comparing the third proximity value to a fourth proximity value adjacent to the third proximity value in the first dimension.

14. The non-transitory computer-readable storage medium of claim 13, the method further comprising forming groups from those proximity values surrounding each local maximum proximity value up to and including the corresponding contact boundary proximity values.

15. The non-transitory computer-readable storage medium of claim 14, the method further comprising combining groups of proximity values that partially overlap.

16. The non-transitory computer-readable storage medium of claim 15, the method further comprising extracting group positions and features by fitting an ellipse to each group of proximity values.

17. The non-transitory computer-readable storage medium of claim 14, the method further comprising extracting group positions and features by fitting an ellipse to each group of proximity values.

18. The non-transitory computer-readable storage medium of claim 13, the method further comprising locating all local maximum proximity values in the proximity image prior to determining the two-dimensional boundary of the contact.