Touch sensing architecture

• US 8,629,840 B2
• Filed: 07/30/2007
• Issued: 01/14/2014
• Est. Priority Date: 01/26/1998
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne, Elias, John G.
• Primary Examiner(s): Sim, Yong H
• Application Number:
  US11/830,774
• Publication Number:
  US 20080042986A1
• Time in Patent Office: 2,360 Days
• Field of Search: 345/156, 345168-179, 341 22- 26
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method of selecting input activity of a computing system including a touch sensing surface, the method comprising:

obtaining tracking information of a plurality of contacts corresponding to tracked touch objects concurrently on or near the surface;

repeatedly determining, based on the tracking information of a set of the plurality of contacts, a speed of the set, wherein the set consists of a first number of contacts, and the first number corresponds to a first input activity of the computing system;

monitoring the speed of the set without generating any input based on the speed of the set so long as the speed of the set does not exceed a predetermined threshold speed greater than zero; and

while the set remains touched down, selecting the first input activity based on the speed of the set exceeding the predetermined threshold speed, and generating an input of the first input activity based on the speed of the set.

2. The method of claim 1, wherein initial touch downs of the contacts in the set occurred synchronously.

3. The method of claim 1, wherein initial touch downs of the contacts in the set occurred asynchronously.

4. The method of claim 1, wherein determining the speed of the set of contacts includes determining a mean of speeds of the contacts in the set, and selecting the first input activity includes selecting the first input activity when the mean exceeds the predetermined threshold speed.

5. The method of claim 1, wherein selecting the first input activity includes selecting the first input activity based on a speed of a first contact in the set exceeding the predetermined threshold speed.

6. The method of claim 5, wherein the first contact includes a fastest contact.

7. The method of claim 1, wherein selecting the first input activity includes selecting the first input activity based on a relative speed of the contacts.

8. The method of claim 7, wherein determining the speed of the set of contacts includes determining a range of speeds of the contacts in the set, and selecting the first input activity includes selecting the first input activity only if the range of the speeds is within a predetermined threshold range.

9. The method of claim 8, wherein the predetermined threshold range includes a ratio of a fastest contact speed to a slowest contact speed.

10. The method of claim 1, wherein selecting the first input activity includes selecting from among a plurality of input activities, wherein the input activities correspond to different numbers of contacts.

11. The method of claim 1, wherein determining the speed of the set of contacts includes obtaining one or more components of motion of the set of contacts, and the first input activity includes a plurality of input events that each correspond to a component of motion of the set of contacts, the method further comprising:

generating an input of the first input activity based on the corresponding component of motion.

12. The method of claim 11, wherein the one or more components of motion include one of a translation motion, a rotation motion, and a scaling motion.

13. The method of claim 12, wherein the one or more components of motion further include one of a direction of one of the translation, rotation, and scaling motions, and a range of speed of one of the translation, rotation, and scaling motions.

14. The method of claim 13, wherein the input activity includes text cursor motion, the one or more components of motion include up, down, left, and right translation motion directions, and the one or more inputs include up, down, left, and right cursor movements, respectively.

15. The method of claim 1, wherein the set of contacts includes all contacts being tracked.

16. The method of claim 1, wherein the set of contacts consists of contacts corresponding to fewer than all tracked touch objects concurrently on or near the surface, the method further comprising:

repeatedly determining, based on the tracking information of one or more additional contacts of the plurality of contacts, a speed of the one or more additional contacts.

17. The method of claim 16, further comprising:

generating an input based on the speed of the one or more additional contacts, wherein the input is generated during the monitoring the speed of the set without generating any input based on the speed of the set.

18. The method of claim 17, wherein the input includes an input of a second input activity, different than the first input activity.

19. The method of claim 16, further comprising:

detecting a liftoff of the set; and

detecting a touch down of the set, after the liftoff, wherein selecting the first input activity is further based on detecting the touch down.

20. The method of claim 19, wherein detecting the touch down includes determining whether touch down times of the contacts of the set are synchronous, wherein selecting the first input activity is further based on determining that the touch down times are synchronous.

21. The method of claim 16, further comprising:

generating, based on the speed of the set, an input of the first input activity, wherein the input is generated after the selection of the first input activity.

22. The method of claim 21, wherein generating the input is further based on the speed of the one or more additional contacts.

23. A non-transitory computer-readable storage medium storing computer-executable instructions executable to perform a method of selecting input activity of a computing system including a touch sensing surface, the method comprising:

obtaining tracking information of a plurality of contacts corresponding to tracked touch objects concurrently on or near the surface;

repeatedly determining, based on the tracking information of a set of the plurality of contacts, a speed of the set, wherein the set consists of a first number of contacts, and the first number corresponds to a first input activity of the computing system;

monitoring the speed of the set without generating any input based on the speed of the set so long as the speed of the set does not exceed a predetermined threshold speed greater than zero; and

while the set remains touched down, selecting the first input activity based on the speed of the set exceeding the predetermined threshold speed, and generating an input of the first input activity based on the speed of the set.

24. The non-transitory computer-readable storage medium of claim 23, wherein initial touch downs of the contacts in the set occurred synchronously.

25. The non-transitory computer-readable storage medium of claim 23, wherein initial touch downs of the contacts in the set occurred asynchronously.

26. The non-transitory computer-readable storage medium of claim 23, wherein determining the speed of the set of contacts includes determining a mean of speeds of the contacts in the set, and selecting the first input activity includes selecting the first input activity when the mean exceeds the predetermined threshold speed.

27. The non-transitory computer-readable storage medium of claim 23, wherein selecting the first input activity includes selecting the first input activity based on a speed of a first contact in the set exceeding the predetermined threshold speed.

28. The non-transitory computer-readable storage medium of claim 27, wherein the first contact includes a fastest contact.

29. The non-transitory computer-readable storage medium of claim 23, wherein selecting the first input activity includes selecting the first input activity based on a relative speed of the contacts.

30. The non-transitory computer-readable storage medium of claim 29, wherein determining the speed of the set of contacts includes determining a range of speeds of the contacts in the set, and selecting the first input activity includes selecting the first input activity only if the range of the speeds is within a predetermined threshold range.

31. The non-transitory computer-readable storage medium of claim 30, wherein the predetermined threshold range includes a ratio of a fastest contact speed to a slowest contact speed.

32. The non-transitory computer-readable storage medium of claim 23, wherein selecting the first input activity includes selecting from among a plurality of input activities, wherein the input activities correspond to different numbers of contacts.

33. The non-transitory computer-readable storage medium of claim 23, wherein determining the speed of the set of contacts includes obtaining one or more components of motion of the set of contacts, and the first input activity includes a plurality of input events that each correspond to a component of motion of the set of contacts, the method further comprising:

generating an input of the first input activity based on the corresponding component of motion.

34. The non-transitory computer-readable storage medium of claim 33, wherein the one or more components of motion include one of a translation motion, a rotation motion, and a scaling motion.

35. The non-transitory computer-readable storage medium of claim 34, wherein the one or more components of motion further include one of a direction of one of the translation, rotation, and scaling motions, and a range of speed of one of the translation, rotation, and scaling motions.

36. The non-transitory computer-readable storage medium of claim 35, wherein the input activity includes text cursor motion, the one or more components of motion include up, down, left, and right translation motion directions, and the one or more inputs include up, down, left, and right cursor movements, respectively.

37. The non-transitory computer-readable storage medium of claim 23, wherein the set of contacts includes all contacts being tracked.

38. The non-transitory computer-readable storage medium of claim 23, wherein the set of contacts consists of contacts corresponding to fewer than all tracked touch objects concurrently on or near the surface, the method further comprising:

repeatedly determining, based on the tracking information of one or more additional contacts of the plurality of contacts, a speed of the one or more additional contacts.

39. The non-transitory computer-readable storage medium of claim 38, further comprising:

generating an input based on the speed of the one or more additional contacts, wherein the input is generated during the monitoring the speed of the set without generating any input based on the speed of the set.

40. The non-transitory computer-readable storage medium of claim 39, wherein the input includes an input of a second input activity, different than the first input activity.

41. The non-transitory computer-readable storage medium of claim 38, further comprising:

detecting a liftoff of the set; and

detecting a touch down of the set, after the liftoff, wherein selecting the first input activity is further based on detecting the touch down.

42. The non-transitory computer-readable storage medium of claim 41, wherein detecting the touch down includes determining whether touch down times of the contacts of the set are synchronous, wherein selecting the first input activity is further based on determining that the touch down times are synchronous.

43. The non-transitory computer-readable storage medium of claim 38, further comprising:

generating, based on the speed of the set, an input of the first input activity, wherein the input is generated after the selection of the first input activity.

44. The non-transitory computer-readable storage medium of claim 43, wherein generating the input is further based on the speed of the one or more additional contacts.

45. A computing system comprising:

a touch sensing surface;

one or more touch processing modules that obtain tracking information of a plurality of contacts corresponding to touch objects concurrently on or near the surface, repeatedly determine, based on the tracking information of a set of the plurality of contacts, a speed of the set, wherein the set consists of a first number of contacts, and the first number corresponds to a first input activity of the computing system, monitor the speed of the set without generating any input activity based on the speed of the set so long as the speed of the set does not exceed a predetermined threshold speed greater than zero, and, while the set remains touched down select the first input activity based on the speed of the set exceeding the predetermined threshold speed and generate an input of the first input activity based on the speed of the set.

46. The system of claim 45, wherein initial touch downs of the contacts in the set occurred synchronously.

47. The system of claim 45, wherein initial touch downs of the contacts in the set occurred asynchronously.

48. The system of claim 45, wherein determining the speed of the set of contacts includes determining a mean of speeds of the contacts in the set, and selecting the first input activity includes selecting the first input activity when the mean exceeds the predetermined threshold speed.

49. The system of claim 45, wherein selecting the first input activity includes selecting the first input activity based on a speed of a first contact in the set exceeding the predetermined threshold speed.

50. The system of claim 49, wherein the first contact includes a fastest contact.

51. The system of claim 45, wherein selecting the first input activity includes selecting the first input activity based on a relative speed of the contacts.

52. The system of claim 51, wherein determining the speed of the set of contacts includes determining a range of speeds of the contacts in the set, and selecting the first input activity includes selecting the first input activity only if the range of the speeds is within a predetermined threshold range.

53. The system of claim 52, wherein the predetermined threshold range includes a ratio of a fastest contact speed to a slowest contact speed.

54. The system of claim 45, wherein selecting the first input activity includes selecting from among a plurality of input activities, wherein the input activities correspond to different numbers of contacts.

55. The system of claim 45, wherein determining the speed of the set of contacts includes obtaining one or more components of motion of the set of contacts, and the first input activity includes a plurality of input events that each correspond to a component of motion of the set of contacts, wherein the one or more processing modules further generate an input of the first input activity based on the corresponding component of motion.

56. The system of claim 55, wherein the one or more components of motion include one of a translation motion, a rotation motion, and a scaling motion.

57. The system of claim 56, wherein the one or more components of motion further include one of a direction of one of the translation, rotation, and scaling motions, and a range of speed of one of the translation, rotation, and scaling motions.

58. The system of claim 57, wherein the input activity includes text cursor motion, the one or more components of motion include up, down, left, and right translation motion directions, and the one or more inputs include up, down, left, and right cursor movements, respectively.

59. The system of claim 45, wherein the set of contacts includes contacts corresponding to all tracked touch objects concurrently on or near the surface.

60. The system of claim 45, wherein the set of contacts consists of contacts corresponding to fewer than all tracked touch objects concurrently on or near the surface, wherein the one or more processing modules further repeatedly determine, based on the tracking information of one or more additional contacts of the plurality of contacts, a speed of the one or more additional contacts.

61. The system of claim 60, wherein the one or more processing modules further generate an input based on the speed of the one or more additional contacts, wherein the input is generated during the monitoring the speed of the set without generating any input based on the speed of the set.

62. The system of claim 61, wherein the input includes an input of a second input activity, different than the first input activity.

63. The system of claim 60, wherein the one or more processing modules further detect a liftoff of the set, and detect a touch down of the set, after the liftoff, wherein selecting the first input activity is further based on detecting the touch down.

64. The system of claim 63, wherein detecting the touch down includes determining whether touch down times of the contacts of the set are synchronous, wherein selecting the first input activity is further based on determining that the touch down times are synchronous.

65. The system of claim 60, wherein the one or more processing modules further generate, based on the speed of the set, an input of the first input activity, wherein the input is generated after the selection of the first input activity.

66. The system of claim 65, wherein generating the input is further based on the speed of the one or more additional contacts.