CONTACT TRACKING AND IDENTIFICATION MODULE FOR TOUCH SENSING

• US 20080041639A1
• Filed: 07/30/2007
• Published: 02/21/2008
• Est. Priority Date: 01/26/1998
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne, Elias, John
• Granted Patent: US 8,730,177 B2
• Time in Patent Office: Days
• US Class Current: 178/18.01
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method of interpreting the output of a touch sensor comprising the steps of:

a. receiving a set of contact data from a touch sensor, the touch sensor comprising a touch surface and a plurality of sensors distributed beneath the touch surface, each sensor represented by a sensor identification;

b. associating each sensor with one or more datum regarding the state of the sensor at a particular time;

c. creating groups of sensor identifications, each group representing sensors that correspond to a single hand part;

d. evaluating both sensor data associated with each group and stored data, to estimate path and contact information for each group;

e. associating each group with either a right hand or left hand;

f. associating each group with a specific hand part; and

g. storing a data structure comprising at least a portion of the information about each group.

2. The method of claim 1 wherein contact data is produced with respect to objects within the sensing range of the touch sensor, whether or not objects are touching the touch sensor.

3. The method of claim 1 wherein stored data comprises an information associated with sensors'"'"' states from a plurality of times prior to the particular time.

4. The method of claim 1 wherein hand parts comprise at least two of the following:

index finger, middle finger, ring finger, pinky finger, inner palm heelheel, or outer palm heelheel.

5. The method of claim 1 comprising the further step of estimating the position of each hand.

6. The method of claim 5 wherein the step of creating groups of sensor identifications, each group representing sensors that correspond to a single hand part, incorporates the evaluation of prior hand position information.

7. The method of claim 5 wherein the step of associating each group with either a right hand or left hand, incorporates the evaluation of prior hand position information.

8. The method of claim 1 comprising the further step of associating each group with a data structure comprising information regarding the associated sensors.

9. The method of claim 1 wherein the path is an expression representing the movement of a particular hand part over time.

10. The method of claim 1 wherein the contact information is an indication regarding whether a particular hand part is touching the touch surface.

11. The method of claim 1 wherein the contact information is an indication regarding whether a particular hand part is within sensing range of the touch sensor.

20. The method of claim 1 wherein the path is an expression representing the movement of a particular hand part over time.

12. A method of interpreting the output of a touch sensor comprising the steps of:

h. receiving a first set of contact data from a touch sensor at time t1, the touch sensor comprising a touch surface and a plurality of sensors distributed beneath the touch surface, each sensor represented by a sensor identification;

i. receiving a second set of contact data from the touch sensor at time t2;

j. receiving a third set of contact data from the touch sensor at time t3;

k. for each set of contact data, associating each sensor with one or more datum regarding the state of the sensor;

l. for each set of contact data, creating groups of sensor identifications, each group representing sensors that correspond to a single hand part;

m. for each set of contact data, evaluating both sensor data associated with each group and stored data, to estimate path and contact information for each group, wherein stored data from the third set of contact data, comprises information associated with the first set of contact data and the second set of contact data;

n. for each set of contact data, associating each group with either a right hand or left hand;

o. for each set of contact data, associating each group with a specific hand part; and

p. for each set of contact data, storing a information comprising at least a portion of the information about each group.

13. The method of claim 12 wherein contact data is produced with respect to objects within the sensing range of the touch sensor, whether or not objects are touching the touch sensor.

14. The method of claim 12 wherein hand parts comprise at least two of the following:

index finger, middle finger, ring finger, pinky finger, inner palm heel, or outer palm heel.

15. The method of claim 12 comprising the further step of estimating the position of each hand.

16. The method of claim 15 wherein the step of creating groups of sensor identifications, each group representing sensors that correspond to a single hand part, incorporates the evaluation of prior hand position information.

17. A system for interpreting the output of a touch sensor comprising:

q. a touch sensor comprising a touch surface and a plurality of sensors distributed beneath the touch surface, each sensor represented by a sensor identification;

r. one or more datum associated with the state of the sensor at a particular time;

s. a data structure identifying groups of sensor identifications, each group representing sensors that correspond to a single hand part;

t. program instructions for interpreting the data structure and other information regarding the sensors to estimate path and contact information for each group;

u. program instructions for associating each group with either a right hand or left hand;

v. program instructions for associating each group with a specific hand part.

18. The method of claim 17 wherein hand parts comprise at least two of the following:

index finger, middle finger, ring finger, pinky finger, inner palm heel, or outer palm heel.

19. The method of claim 17 further comprising a feedback unit for making information available to other system module regarding prior processing results.