Contact tracking and identification module for touch sensing

• US 8,482,533 B2
• Filed: 06/05/2009
• Issued: 07/09/2013
• Est. Priority Date: 01/26/1998
• Status: Expired due to Fees

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne Carl, Elias, John Greer
• Primary Examiner(s): Sim, Yong H
• Application Number:
  US12/479,678
• Publication Number:
  US 20090244033A1
• Time in Patent Office: 1,495 Days
• Field of Search: 345/156, 345168-179, 341 22- 26, 178 1801- 1807
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method of generating a control signal of a computing system including a touch sensitive surface, the method comprising:

tracking a path of each contact of a set of one or more contacts, each contact corresponding to a touch object on or near the surface;

determining motion information of the set of contacts based on the tracking;

determining a liftoff from the touch sensitive surface of one or more contacts of the set based on the tracking;

determining a dominant component of motion of the set of contacts based on the motion information prior to the liftoff, the dominant component of motion including one of translation, rotation, and scaling;

determining a first motion of the one or more contacts determined to have lifted off based on the motion information prior to the liftoff, including determining a weighted average velocity of the dominant component of motion prior to the liftoff;

determining whether the first motion is greater than a predetermined threshold; and

generating, based on the determination of whether the first motion is greater than the predetermined threshold, a post-liftoff control signal based on the dominant component of motion.

2. The method of claim 1, further comprising:

determining, prior to the liftoff, a second motion of the set of contacts;

generating a pre-liftoff control signal, wherein the pre-liftoff control signal controls a first input of the computing system based on the second motion prior to the liftoff, and the post-liftoff control signal continues the first input of the computing system after the liftoff.

3. The method of claim 2, wherein the first input includes one of a scroll input, a cursor movement input, a rotation input, and a scaling input.

4. The method of claim 2, wherein the post-liftoff control signal continues the first input based on the first motion.

5. The method of claim 2, wherein the post-liftoff control signal continues the first input based on the weighted average velocity of the dominant component of motion.

6. The method of claim 2, wherein the set of contacts includes a contact that remains on the surface after the liftoff of the one or more contacts, the method further comprising:

determining a third motion of the contact that remains on the surface after the liftoff, wherein the post-liftoff control signal continues the first input based on the third motion.

7. The method of claim 2, wherein the set of contacts includes a contact that remains on the surface after the liftoff of the one or more contacts, the method further comprising:

determining a liftoff of the contact that remains on the surface after the liftoff; and

discontinuing the first input based on the liftoff the remaining contact.

8. The method of claim 2, further comprising:

determining, after the liftoff of the one or more contacts, a touchdown of a contact; and

discontinuing the first input based on the touchdown.

9. The method of claim 1, wherein the first motion includes an acceleration, and the acceleration is one of a translational acceleration, a rotational acceleration, and a scaling acceleration.

10. The method of claim 9, wherein the acceleration includes a percentage acceleration and determining the acceleration includes determining a ratio of a first speed to a second speed, wherein the first speed includes a speed of the lifted off one or more contacts immediately prior to the liftoff, and the second speed includes a speed of the lifted off one or more contacts prior to the first speed.

11. The method of claim 10, wherein the predetermined threshold is a threshold ratio of the first speed to the second speed, and the threshold ratio is approximately equal to one.

12. The method of claim 10, wherein the second speed includes an average speed of the lifted off one or more contacts prior to the first speed, the average speed being computed by one of a moving window average and an autoregressive filter.

13. The method of claim 9, further comprising:

determining whether a proximity of the lifted off one or more contacts is decreasing faster than a predetermined rate, wherein determining the acceleration includes determining the acceleration based on motion information prior to the proximity decreasing faster than the predetermined rate.

14. The method of claim 1, wherein the first control signal includes a one-shot command.

15. An apparatus for generating a control signal of a computing system, the apparatus comprising:

a touch sensitive surface;

a tracking module that tracks a path of each contact of a set of one or more contacts, each contact corresponding to a touch object on or near the surface;

a motion component extraction module that determines motion information of the set of contacts based on the tracking; and

an input generator that determines a liftoff from the touch sensitive surface of one or more contacts of the set based on the tracking, determines a dominant component of motion of the set of contacts based on the motion information prior to the liftoff, the dominant component of motion including one of translation, rotation, and scaling, determines a first motion of the one or more contacts determined to have lifted off based on the motion information prior to the liftoff, including determining a weighted average velocity of the dominant component of motion prior to the liftoff, determines whether the first motion is greater than a predetermined threshold, and generates, based on the determination of whether the first motion is greater than the predetermined threshold, a post-liftoff control signal based on the dominant component of motion.

16. The apparatus of claim 15, wherein the motion component extraction module further determines, prior to the liftoff, a second motion of the set of contacts, and the input generator further generates a pre-liftoff control signal, wherein the pre-liftoff control signal controls a first input of the computing system based on the second motion prior to the liftoff, and the post-liftoff control signal continues the first input of the computing system after the liftoff.

17. The apparatus of claim 16, wherein the first input includes one of a scroll input, a cursor movement input, a rotation input, and a scaling input.

18. The apparatus of claim 16, wherein the post-liftoff control signal continues the first input based on the first motion.

19. The apparatus of claim 16, wherein the post-liftoff control signal continues the first input based on the weighted average velocity of the dominant component of motion.

20. The apparatus of claim 16, wherein the set of contacts includes a contact that remains on the surface after the liftoff of the one or more contacts, and the motion component extraction module further determines a third motion of the contact that remains on the surface after the liftoff, wherein the post-liftoff control signal continues the first input based on the third motion.

21. The apparatus of claim 16, wherein the set of contacts includes a contact that remains on the surface after the liftoff of the one or more contacts, and the input generator further determines a liftoff of the contact that remains on the surface after the liftoff and discontinues the first input based on the liftoff the remaining contact.

22. The apparatus of claim 16, wherein the input generator further determines, after the liftoff of the one or more contacts, a touchdown of a contact, and discontinues the first input based on the touchdown.

23. The apparatus of claim 15, wherein the first motion includes an acceleration, and the acceleration is one of a translational acceleration, a rotational acceleration, and a scaling acceleration.

24. The apparatus of claim 23, wherein the acceleration includes a percentage acceleration and the determining the acceleration by the input generator includes determining a ratio of a first speed to a second speed, wherein the first speed includes a speed of the lifted off one or more contacts immediately prior to the liftoff, and the second speed includes a speed of the lifted off one or more contacts prior to the first speed.

25. The apparatus of claim 24, wherein the predetermined threshold is a threshold ratio of the first speed to the second speed, and the threshold ratio is approximately equal to one.

26. The apparatus of claim 24, wherein the second speed includes an average speed of the lifted off one or more contacts prior to the first speed, the average speed being computed by one of a moving window average and an autoregressive filter.

27. The apparatus of claim 23, wherein the input generator further determines whether a proximity of the lifted off one or more contacts is decreasing faster than a predetermined rate, wherein determining the acceleration includes determining the acceleration based on motion information prior to the proximity decreasing faster than the predetermined rate.

28. The apparatus of claim 15, wherein the first control signal includes a one-shot command.

29. A non-transitory computer readable storage medium having instructions stored therein, which when executed by a computing system including a touch sensitive surface causes the computing system to perform a method of generating a control signal, the method comprising:

tracking a path of each contact of a set of one or more contacts, each contact corresponding to a touch object on or near the surface;

determining motion information of the set of contacts based on the tracking;

determining a liftoff from the touch sensitive surface of one or more contacts of the set based on the tracking;

determining a dominant component of motion of the set of contacts based on the motion information prior to the liftoff, the dominant component of motion including one of translation, rotation, and scaling;

determining a first motion of the one or more contacts determined to have lifted off based on the motion information prior to the liftoff, including determining a weighted average velocity of the dominant component of motion prior to the liftoff;

determining whether the first motion is greater than a predetermined threshold; and

generating, based on the determination of whether the first motion is greater than the predetermined threshold, a post-liftoff control signal based on the dominant component of motion.

30. The non-transitory computer readable storage medium of claim 29, the method further comprising:

determining, prior to the liftoff, a second motion of the set of contacts;

generating a pre-liftoff control signal, wherein the pre-liftoff control signal controls a first input of the computing system based on the second motion prior to the liftoff, wherein the post-liftoff control signal continues the first input of the computing system after the liftoff.

31. The non-transitory computer readable storage medium of claim 30, wherein the first input includes one of a scroll input, a cursor movement input, a rotation input, and a scaling input.

32. The non-transitory computer readable storage medium of claim 30, wherein the post-liftoff control signal continues the first input based on the first motion.

33. The non-transitory computer readable storage medium of claim 30, wherein the post-liftoff control signal continues the first input based on the weighted average velocity of the dominant component of motion.

34. The non-transitory computer readable storage medium of claim 30, wherein the set of contacts includes a contact that remains on the surface after the liftoff of the one or more contacts, the method further comprising:

determining a third motion of the contact that remains on the surface after the liftoff, wherein the post-liftoff control signal continues the first input based on the third motion.

35. The non-transitory computer readable storage medium of claim 30, wherein the set of contacts includes a contact that remains on the surface after the liftoff of the one or more contacts, the method further comprising:

determining a liftoff of the contact that remains on the surface after the liftoff; and

discontinuing the first input based on the liftoff the remaining contact.

36. The non-transitory computer readable storage medium of claim 30, the method further comprising:

determining, after the liftoff of the one or more contacts, a touchdown of a contact; and

discontinuing the first input based on the touchdown.

37. The non-transitory computer readable storage medium of claim 29, wherein the first motion includes an acceleration, and the acceleration is one of a translational acceleration, a rotational acceleration, and a scaling acceleration.

38. The non-transitory computer readable storage medium of claim 37, wherein the acceleration includes a percentage acceleration and determining the acceleration includes determining a ratio of a first speed to a second speed, wherein the first speed includes a speed of the lifted off one or more contacts immediately prior to the liftoff, and the second speed includes a speed of the lifted off one or more contacts prior to the first speed.

39. The non-transitory computer readable storage medium of claim 38, wherein the predetermined threshold is a threshold ratio of the first speed to the second speed and the threshold ratio is approximately equal to one.

40. The non-transitory computer readable storage medium of claim 38, wherein the second speed includes an average speed of the lifted off one or more contacts prior to the first speed, the average speed being computed by one of a moving window average and an autoregressive filter.

41. The non-transitory computer readable storage medium of claim 37, the method further comprising:

determining whether a proximity of the lifted off one or more contacts is decreasing faster than a predetermined rate, wherein determining the acceleration includes determining the acceleration based on motion information prior to the proximity decreasing faster than the predetermined rate.

42. The non-transitory computer readable storage medium of claim 29, wherein the first control signal includes a one-shot command.