MULTI-TOUCH CONTACT MOTION EXTRACTION

• US 20070070050A1
• Filed: 11/14/2006
• Published: 03/29/2007
• Est. Priority Date: 01/26/1998
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: FingerWorks Limited (Apple Inc.)
• Inventors: Westerman, Wayne, Elias, John
• Granted Patent: US 8,330,727 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method for extracting multiple degrees of freedom of hand motion from successive proximity images, the method comprising:

tracking a plurality of contacts associated with a plurality of hand parts across the successive proximity images;

finding an innermost finger contact and an outermost finger contact for a given hand from the plurality of contacts;

computing a scaling velocity component from a change in a distance between the innermost and outermost finger contacts; and

transmitting the computed scaling velocity component as a control signal to an electronic or electromechanical device.

2. The method of claim 1 further comprising:

computing a rotational velocity component from a change in a vector angle between the innermost and outermost finger contacts; and

transmitting the computed rotational velocity component as a control signal to an electronic or electromechanical device.

3. The method of claim 2 further comprising:

computing a translation weighting for each contact associated with a finger;

computing translational velocity components for each contact associated with a finger;

computing a translational velocity average from the computed translational velocity components and the computed translation weightings; and

transmitting the computed translational velocity average as a control signal to an electronic or electromechanical device.

4. The method of claim 1 further comprising:

computing a translation weighting for each contact associated with a finger;

computing translational velocity components for each contact associated with a finger;

computing a translational velocity average from the computed translational velocity components and the computed translation weightings; and

transmitting the computed translational velocity average as a control signal to an electronic or electromechanical device.

8. The method of any one of claims 1-7 wherein transmitting the computed velocity components further comprises filtering the computed velocity components prior to transmission.

9. The method of claim 8, wherein filtering the computed velocity components further comprises:

downscaling each velocity component in proportion to a function of its average speed compared to other average component speeds; and

dead-zone filtering each downscaled velocity component wherein dead-zone width depends on distribution of current component speeds.

10. The method of claim 1-7 further comprising transmitting an orientation of an ellipse fitted to a thumb contact to an electronic or electromechanical device.

11. The method of claim 8 further comprising transmitting an orientation of an ellipse fitted to a thumb contact to an electronic or electromechanical device.

12. The method of claim 9 further comprising transmitting an orientation of an ellipse fitted to a thumb contact to an electronic or electromechanical device.

5. A method for extracting multiple degrees of freedom of hand motion from successive proximity images, the method comprising:

tracking a plurality of contacts associated with a plurality of hand parts across the successive proximity images;

finding an innermost finger contact and an outermost finger contact for a given hand from the plurality of contacts;

computing a rotational velocity component from a change in a vector angle between the innermost and outermost finger contacts; and

transmitting the computed rotational velocity component as a control signal to an electronic or electromechanical device.

6. The method of claim 5 further comprising:

computing a translation weighting for each contact associated with a finger;

computing translational velocity components for each contact associated with a finger;

computing a translational velocity average from the computed translational velocity components and the computed translation weightings; and

transmitting the computed translational velocity average as a control signal to an electronic or electromechanical device.

7. A method for extracting multiple degrees of freedom of hand motion from successive proximity images, the method comprising:

tracking a plurality of contacts associated with a plurality of hand parts across the successive proximity images;

computing a translation weighting for each contact associated with a finger;

computing translational velocity components for each contact associated with a finger;

computing a translational velocity average from the computed translational velocity components and the computed translation weightings and

transmitting the computed translational velocity average as a control signal to an electronic or electromechanical device.

13. A method for extracting multiple degrees of freedom of hand motion from successive proximity images representing successive scans of a plurality of proximity sensors of a multi-touch surface, the method comprising:

tracking, through successive proximity images, a plurality of groups of pixels associated with a plurality of fingers on or near the multi-touch surface;

identifying a group of pixels corresponding to an innermost finger and a group of pixels corresponding to an outermost finger;

computing a scaling velocity from a change in a distance between the innermost and outermost fingers; and

transmitting the computed scaling velocity as a control signal to an electronic or electromechanical device.

14. The method of claim 13 further comprising:

computing a rotational velocity from a change in angle between the innermost and outermost fingers; and

transmitting the computed rotational velocity as a control signal to an electronic or electromechanical device.

15. The method of claim 14 further comprising:

computing a translation weighting for each finger;

computing a translational velocity for each finger;

computing a translational velocity average from the computed translational velocities and the computed translation weightings; and

transmitting the computed translational velocity average as a control signal to an electronic or electromechanical device.

16. The method of claim 13 further comprising:

computing a translation weighting for each finger;

computing a translational velocity for each finger;

computing a translational velocity average from the computed translational veliocities and the computed translation weightings; and

transmitting the computed translational velocity average as a control signal to an electronic or electromechanical device.

20. The method of any one of claims 13-19 wherein transmitting the computed velocities further comprises filtering the computed velocities prior to transmission.

21. The method of claim 13-19 further comprising transmitting an orientation of an ellipse fitted to a thumb contact to an electronic or electromechanical device.

22. The method of claim 20 further comprising transmitting an orientation of an ellipse fitted to a thumb contact to an electronic or electromechanical device.

23. The method of any of claims 13-19 wherein at least one of the fingers is near but not on the multi-touch surface.

24. The method of any one of claims 23 wherein transmitting the computed velocities further comprises filtering the computed velocities prior to transmission.

25. The method of claim 24 further comprising transmitting an orientation of an ellipse fitted to a thumb contact to an electronic or electromechanical device.

26. The method of claim 23 further comprising transmitting an orientation of an ellipse fitted to a thumb contact to an electronic or electromechanical device.

17. A method for extracting multiple degrees of freedom of hand motion from successive proximity images representing successive scans of a plurality of proximity sensors of a multi-touch surface, the method comprising:

tracking, through successive proximity images, a plurality of groups of pixels associated with a plurality of fingers on or near the multi-touch surface;

identifying a group of pixels corresponding to an innermost finger and a group of pixels corresponding to an outermost finger;

computing a rotational velocity from a change in angle between the innermost and outermost fingers; and

transmitting the computed rotational velocity as a control signal to an electronic or electromechanical device.

18. The method of claim 17 further comprising:

computing a translation weighting for each finger;

computing a translational velocity for each finger;

computing a translational velocity average from the computed translational velocities and the computed translation weightings; and

transmitting the computed translational velocity average as a control signal to an electronic or electromechanical device.

19. A method for extracting multiple degrees of freedom of hand motion from successive proximity images representing successive scans of a plurality of proximity sensors of a multi-touch surface, the method comprising:

tracking, through successive proximity images, a plurality of groups of pixels associated with a plurality of fingers on or near the multi-touch surface;

computing a translation weighting for each finger;

computing a translational velocity for each finger;

computing a translational velocity average from the computed translational velocities and the computed translation weightings; and

transmitting the computed translational velocity average as a control signal to an electronic or electromechanical device.