Contact tracking and identification module for touch sensing

• US 8,730,192 B2
• Filed: 08/07/2012
• Issued: 05/20/2014
• Est. Priority Date: 01/26/1998
• Status: Expired due to Fees

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne, Elias, John Greer
• Primary Examiner(s): Sim, Yong H
• Application Number:
  US13/569,065
• Publication Number:
  US 20120293442A1
• Time in Patent Office: 651 Days
• Field of Search: 345173-184, 178 1801- 1905, 715862-864
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method of processing input from a touch-sensitive surface, the method comprising:

receiving at least one proximity image representing a scan of a plurality of electrodes of the touch-sensitive surface;

segmenting each proximity image into one or more pixel groups that indicate significant proximity, each pixel group representing proximity of a distinguishable hand part or other touch object on or near the touch-sensitive surface; and

determining one of a major axis and a minor axis of an ellipse of at least one of the pixel groups.

2. The method of claim 1, further comprising transmitting one or more ellipse parameters as a control signal to an electronic or electromechanical device.

3. The method of claim 2, wherein the one or more ellipse parameters is selected from the group consisting of position, shape, size, orientation, eccentricity, major axis, minor axis, and any combination thereof.

4. The method of claim 3, wherein the one or more ellipse parameters are used to distinguish a pixel group associated with a fingertip from a pixel group associated with a thumb.

5. The method of claim 1, wherein determining the one of a major axis and a minor axis of at least one of the pixel groups comprises computing one or more eigenvalues and one or more eigenvectors of a covariance matrix associated with the pixel group.

6. The method of claim 1, further comprising:

tracking a path of at least one of the one or more pixel groups through a time-sequenced series of proximity images;

determining the one of a major axis and a minor axis of the at least one of the one or more pixel groups in each of the time-sequenced series of proximity images; and

tracking a change in one or more ellipse parameters through the time-sequenced series of proximity images.

7. The method of claim 6, further comprising transmitting the change in the one or more ellipse parameters as a control signal to an electronic or electromechanical device.

8. The method of claim 7, wherein the change in the one or more ellipse parameters is selected from the group consisting of position, shape, size, orientation, eccentricity, major axis, minor axis, and any combination thereof.

9. The method of claim 6, wherein determining the one of a major axis and a minor axis of the one pixel group comprises computing one or more eigenvalues and one or more eigenvectors of a covariance matrix associated with the pixel group.

23. A non-transitory computer-readable medium having embodied thereon instructions executable by a machine to perform a method according to claim 1.

10. A touch-sensing device comprising:

a substrate;

a plurality of touch-sensing electrodes arranged on the substrate;

electronic scanning hardware adapted to read the plurality of touch-sensing electrodes;

a calibration module operatively coupled to the electronic scanning hardware and adapted to construct a proximity image having a plurality of pixels corresponding to the touch-sensing electrodes; and

a contact tracking and identification module adapted to;

segment the proximity image into one or more pixel groups, each pixel group representing proximity of a distinguishable hand part or other touch object on or near the touch-sensitive surface; and

determine one of a major axis and a minor axis of an ellipse of at least one of the one or more pixel groups.

11. The touch-sensing device of claim 10, further comprising a host communication interface adapted to transmit one or more ellipse parameters as a control signal to an electronic or electromechanical device.

12. The touch-sensing device of claim 11, wherein the touch-sensing device is integral with the electronic or electromechanical device.

13. The touch-sensing device of claim 11, wherein the one or more ellipse parameters comprise one or more parameters selected from the group consisting of position, shape, size, orientation, eccentricity, major axis, minor axis, and any combination thereof.

14. The touch-sensing device of claim 13, wherein the one or more ellipse parameters are used to distinguish a pixel group associated with a fingertip from a pixel group associated with a thumb.

15. The touch-sensing device of claim 10, wherein the contact tracking and identification module is adapted to compute one or more eigenvalues and one or more eigenvectors to determine the one of the major axis and the minor axis.

16. The touch-sensing device of claim 10, wherein the contact tracking and identification module is further adapted to:

track a path of one or more pixel groups through a plurality of time-sequenced proximity images;

determine the one of a major axis and a minor axis of at least one of the one or more pixel groups in a first proximity image of the plurality of time-sequenced proximity images; and

track a change in one or more ellipse parameters through two or more of the time-sequenced proximity images.

17. The touch-sensing device of claim 16, further comprising a host communication interface adapted to transmit the change in at least one of the one or more ellipse parameters as a control signal to an electronic or electromechanical device.

18. The touch-sensing device of claim 17, wherein the touch-sensing device is integral with the electronic or electromechanical device.

19. The touch-sensing device of claim 17, wherein the change in one or more ellipse parameters used as a control input to an electronic or electromechanical device comprises one or more parameters selected from the group consisting of position, shape, size, orientation, eccentricity, major axis, minor axis, and any combination thereof.

20. The touch-sensing device of claim 16, wherein the contact tracking and identification module is adapted to compute one or more eigenvalues and one or more eigenvectors to determine the one of the major axis and the minor axis.

21. The touch-sensing device of claim 10, wherein the touch-sensing device is fabricated on or integrated with a display device.

22. The touch-sensing device of claim 21, wherein the display device comprises a liquid crystal display (LCD) or a light-emitting polymer display (LPD).