Contact tracking and identification module for touch sensing

• US 8,866,752 B2
• Filed: 04/10/2009
• Issued: 10/21/2014
• Est. Priority Date: 01/26/1998
• Status: Expired due to Fees

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Elias, John G., Westerman, Wayne
• Primary Examiner(s): Sim, Yong H
• Application Number:
  US12/422,222
• Publication Number:
  US 20090244031A1
• Time in Patent Office: 2,020 Days
• Field of Search: 345173-184, 178 1801- 1806
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method for a computing system including a touch sensitive surface, the method comprising:

obtaining tracking information of each contact of a set of a plurality of contacts, wherein the contacts correspond to touch objects concurrently on or near the surface, the tracking information including a position of a first contact of the set at a first time, the first contact further having a relative position to a location of each of the plurality of contacts;

determining a motion of the set, wherein the motion of the set is based on the tracking information associated with each contact of the plurality of contacts of the set;

determining a motion of a sub-set of the contacts of the set, wherein the motion of the sub-set is based on the tracking information associated with each of one or more, but not all, of the contacts of the set, wherein the sub-set includes the first contact, and both the motion of the set and the motion of the sub-set are determined based on the position of the first contact at the first time and independent of at least one of the relative positions;

providing the motion of the set as a control signal of a first input of the computing system; and

providing the motion of the sub-set as a control signal of a second input of the computing system.

2. The method of claim 1, further comprising:

tracking the each of the plurality of contacts based on a sequence of proximity images of the contacts.

3. The method of claim 1, further comprising:

modifying the sub-set to obtain a new sub-set, wherein the new sub-set includes one or more contacts not included in the sub-set.

4. The method of claim 3, wherein modifying the sub-set includes modifying the sub-set based on the tracking information of two or more of the plurality of contacts.

5. The method of claim 4, wherein modifying the sub-set based on the tracking information of two or more contacts includes determining a motion of each of two contacts, and determining whether the motions of the two contacts are in opposite directions.

6. The method of claim 1, further comprising:

generating the first input in response to the motion of the set; and

generating the second input in response to the motion of the sub-set.

7. The method of claim 6, wherein the computing system further includes a display, the method further comprising:

displaying a graphical object on the display, wherein generating the first input includes translating the graphical object on the display in response to the motion of the set.

8. The method of claim 6, wherein the second input includes one of rotating and scaling.

9. The method of claim 1, wherein determining the motion of the set includes determining a first component of motion based on the tracking information of each of the contacts of the set.

10. The method of claim 9, wherein the first component of motion includes translation.

11. The method of claim 9, further comprising determining a second component of motion, different from the first component of motion, based on the tracking information of each of the contacts of the sub-set.

12. The method of claim 11, wherein the second component of motion includes one of rotation and scaling.

13. The method of claim 11, wherein the tracking information includes pressure information of each contact, and one of the first and second components of motion includes a pressure component.

14. The method of claim 13, wherein the pressure component includes one of a total pressure and a differential pressure.

15. The method of claim 1, wherein the sub-set includes an outermost contact and an innermost contact.

16. The method of claim 15, further comprising:

associating an anatomical identity with each contact, wherein the outermost and innermost contacts include a thumb contact and a pinky contact.

17. The method of claim 1, wherein determining the motion of the sub-set includes determining that the motion of the sub-set is stationary, such that a zero value of the motion is provided as the control signal of the second input.

18. The method of claim 17, wherein determining that the motion of the sub-set is stationary includes determining a motion of each contact of the sub-set, and determining that the motion of each contact of the sub-set is less than a predetermined threshold.

19. The method of claim 1, wherein the motion of the set and the motion of the sub-set are concurrent.

20. An apparatus for a computing system including a touch sensitive surface, the apparatus comprising:

one or more processing modules that obtain tracking information of each contact of a set of a plurality of contacts, wherein the contacts correspond to touch objects concurrently on or near the surface, the tracking information including a position of a first contact of the set at a first time, the first contact further having a relative position to a location of each of the plurality of contacts, determine a motion of the set, wherein the motion of the set is based on the tracking information associated with each contact of the plurality of contacts of the set, determine a motion of a sub-set of the contacts of the set, wherein the motion of the sub-set is based on the tracking information associated with each of one or more, but not all, of the contacts of the set, wherein the sub-set includes the first contact, and both the motion of the set and the motion of the sub-set are determined based on the position of the first contact at the first time and independent of at least one of the relative positions, provide the motion of the set as a control signal of a first input of the computing system, and provide the motion of the sub-set as a control signal of a second input of the computing system.

21. The apparatus of claim 20, further comprising:

a tracking module that tracks the each of the plurality of contacts of the set based on a sequence of proximity images of the contacts, and generates the tracking information based on the tracking of each chord.

22. The apparatus of claim 20, wherein the one or more processing modules further modify the sub-set to obtain a new sub-set, wherein the new sub-set includes one or more contacts not included in the sub-set.

23. The apparatus of claim 22, wherein modifying the sub-set includes modifying the sub-set based on the tracking information of two or more of the plurality of contacts.

24. The apparatus of claim 23, wherein modifying the sub-set based on the tracking information of two or more contacts includes determining a motion of each of two contacts, and determining whether the motions of the two contacts are in opposite directions.

25. The apparatus of claim 20, wherein the one or more processing modules further generate the first input in response to the motion of the set, and generate the second input in response to the motion of the sub-set.

26. The apparatus of claim 25, wherein the computing system further includes a display, and the one or more processing modules further display a graphical object on the display, wherein generating the first input includes translating the graphical object on the display in response to the motion of the set.

27. The apparatus of claim 25, wherein the second input includes one of rotating and scaling.

28. The apparatus of claim 20, wherein determining the motion of the set includes determining a first component of motion based on the tracking information of each of the contacts of the set.

29. The apparatus of claim 28, wherein the first component of motion includes translation.

30. The apparatus of claim 28, the one or more processing modules further determine a second component of motion, different from the first component of motion, based on the tracking information of each of the contacts of the sub-set.

31. The apparatus of claim 30, wherein the second component of motion includes one of rotation and scaling.

32. The apparatus of claim 30, wherein the tracking information includes pressure information of each contact, and one of the first and second components of motion includes a pressure component.

33. The apparatus of claim 32, wherein the pressure component includes one of a total pressure and a differential pressure.

34. The apparatus of claim 20, wherein the sub-set includes an outermost contact and an innermost contact.

35. The apparatus of claim 34, wherein the one or more processing modules further associate an anatomical identity with each contact, wherein the outermost and innermost contacts include a thumb contact and a pinky contact.

36. The apparatus of claim 20, wherein determining the motion of the sub-set includes determining that the motion of the sub-set is stationary, such that a zero value of the motion is provided as the control signal of the second input.

37. The apparatus of claim 36, wherein determining that the motion of the sub-set is stationary includes determining a motion of each contact of the sub-set, and determining that the motion of each contact of the sub-set is less than a predetermined threshold.

38. The apparatus of claim 20, wherein the motion of the set and the motion of the sub-set are concurrent.

39. A non-transitory computer readable storage medium having computer-executable instructions stored therein, which when executed by an apparatus for a computer system including a touch sensitive surface cause the apparatus to perform a method comprising:

obtaining tracking information of each contact of a set of plurality of contacts, wherein the contacts correspond to touch objects concurrently on or near the surface, the tracking information including a position of a first contact of the set at a first time, the first contact further having a relative position to a location of each of the plurality of contacts;

determining a motion of the set, wherein the motion of the set is based on the tracking information associated with each contact of the plurality of contacts of the set;

determining a motion of a sub-set of the contacts of the set, wherein the motion of the sub-set is based on the tracking information associated with each of one or more, but not all, of the contacts of the set, wherein the sub-set includes the first contact, and both the motion of the set and the motion of the sub-set are determined based on the position of the first contact at the first time and independent of at least one of the relative positions;

providing the motion of the set as a control signal of a first input of the computing system; and

providing the motion of the sub-set as a control signal of a second input of the computing system.

40. The non-transitory computer readable storage medium of claim 39, the method further comprising:

tracking the each of the contacts of the plurality of the set based on a sequence of proximity images of the contacts.

41. The non-transitory computer readable storage medium of claim 39, the method further comprising:

modifying the sub-set to obtain a new sub-set, wherein the new sub-set includes one or more contacts not included in the sub-set.

42. The non-transitory computer readable storage medium of claim 41, wherein modifying the sub-set includes modifying the sub-set based on the tracking information of two or more of the plurality of contacts.

43. The non-transitory computer readable storage medium of claim 42, wherein modifying the sub-set based on the tracking information of two or more contacts includes determining a motion of each of two contacts, and determining whether the motions of the two contacts are in opposite directions.

44. The non-transitory computer readable storage medium of claim 39, the method further comprising:

generating the first input in response to the motion of the set; and

generating the second input in response to the motion of the sub-set.

45. The non-transitory computer readable storage medium of claim 44, wherein the computing system further includes a display, the method further comprising:

displaying a graphical object on the display, wherein generating the first input includes translating the graphical object on the display in response to the motion of the set.

46. The non-transitory computer readable storage medium of claim 44, wherein the second input includes one of rotating and scaling.

47. The non-transitory computer readable storage medium of claim 39, wherein determining the motion of the set includes determining a first component of motion based on the tracking information of each of the contacts of the set.

48. The non-transitory computer readable storage medium of claim 47, wherein the first component of motion includes translation.

49. The non-transitory computer readable storage medium of claim 47, the method further comprising determining a second component of motion, different from the first component of motion, based on the tracking information of each of the contacts of the sub-set.

50. The non-transitory computer readable storage medium of claim 49, wherein the second component of motion includes one of rotation and scaling.

51. The non-transitory computer readable storage medium of claim 49, wherein the tracking information includes pressure information of each contact, and one of the first and second components of motion includes a pressure component.

52. The non-transitory computer readable storage medium of claim 51, wherein the pressure component includes one of a total pressure and a differential pressure.

53. The non-transitory computer readable storage medium of claim 39, wherein the sub-set includes an outermost contact and an innermost contact.

54. The non-transitory computer readable storage medium of claim 53, the method further comprising:

associating an anatomical identity with each contact, wherein the outermost and innermost contacts include a thumb contact and a pinky contact.

55. The non-transitory computer readable storage medium of claim 39, wherein determining the motion of the sub-set includes determining that the motion of the sub-set is stationary, such that a zero value of the motion is provided as the control signal of the second input.

56. The non-transitory computer readable storage medium of claim 55, wherein determining that the motion of the sub-set is stationary includes determining a motion of each contact of the sub-set, and determining that the motion of each contact of the sub-set is less than a predetermined threshold.

57. The non-transitory computer readable storage medium of claim 39, wherein the motion of the set and the motion of the sub-set are concurrent.