Identifying contacts on a touch surface

• US 8,466,883 B2
• Filed: 05/01/2009
• Issued: 06/18/2013
• Est. Priority Date: 01/26/1998
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne Carl, Elias, John Greer
• Primary Examiner(s): SHENG, TOM V
• Application Number:
  US12/434,439
• Publication Number:
  US 20100149092A1
• Time in Patent Office: 1,509 Days
• Field of Search: 345/156, 345/158, 345/173, 345/174, 345/179, 178/18.03, 715/863, 715/864
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method of input of a computing system including a touch sensing surface, the method comprising:

obtaining touch sensing information of a scan of the touch sensing surface;

determining, based on the touch sensing information of the scan, contact information of a set of one or more contacts, each contact corresponding to a touch object on or near the surface;

determining, based on the contact information, a first orientation of a first contact of the set;

obtaining touch sensing information of another scan of the touch sensing surface;

determining, based on the touch sensing information of the other scan, a second orientation of the first contact;

determining a rotation of the first contact based on the first orientation and the second orientation; and

generating input of the computing system based on the first orientation, including generating the input based on the rotation.

2. The method of claim 1, further comprising:

identifying the first contact, wherein generating the input is further based on the identity of the first contact.

3. The method of claim 2, wherein identifying the first contact includes determining an anatomical identity of the first contact, and generating the input based on the identity includes generating the input based on the anatomical identity of the first contact.

4. The method of claim 3, wherein determining the anatomical identity of the first contact includes identifying the first contact as a thumb contact, and generating the input based on the anatomical identity includes generating the input based on the identification of the first contact as a thumb contact.

5. The method of claim 1, wherein generating the input based on the first orientation includes grouping the first contact together with one or more other contacts as a first group based on the first orientation, and basing the input on the first group.

6. The method of claim 5, wherein grouping the first contact and one or more other contacts as a first group includes weighting a first information based on the first orientation to obtain a weighted first information, wherein the first group is based on the weighted first information.

7. The method of claim 6, wherein the first information includes a distance between the first contact and an attractor point of a template, weighting the first information includes weighting the distance to obtain a weighted distance, and grouping the first contact and one or more other contacts includes determining a cost of the first group based on the weighted distance, and wherein the first group is based on the cost.

8. The method of claim 1, wherein generating the input based on the first orientation includes determining, based on the first orientation, an anatomical identity of the first contact, wherein the first input is based on the anatomical identity.

9. The method of claim 8, further comprising:

determining a third orientation of a second contact of the set, wherein the anatomical identity of the first contact is further determined based on the third orientation.

10. The method of claim 9, further comprising:

evaluating a fuzzy logic expression based on the first and third orientations, wherein the anatomical identity of the first contact is further determined based on the evaluation of the fuzzy logic expression.

11. The method of claim 1, wherein the contact information includes a size of the first contact, and determining the first orientation includes determining the first orientation based on the size of the first contact.

12. The method of claim 11, further comprising:

determining whether the size of the first contact is less than a predetermined threshold size, wherein determining the first orientation includes setting the first orientation to a default orientation if the size of the first contact is less than the predetermined threshold size.

13. The method of claim 1, wherein the set includes a plurality of contacts.

14. The method of claim 1, wherein the contact information includes a shape of the first contact, and determining the first orientation includes determining the first orientation based on the shape of the first contact.

15. The method of claim 14, wherein determining the first orientation based on the shape includes fitting an ellipse to the shape, determining an angle of an axis of the fitted ellipse, and basing the first orientation on the angle.

16. A computing system comprising:

a touch sensing surface;

one or more touch processing modules that obtain touch sensing information of a scan of the touch sensing surface, determine, based on the touch sensing information of the scan, contact information of a set of one or more contacts, each contact corresponding to a touch object on or near the surface, determine, based on the contact information, a first orientation of a first contact of the set, obtain touch sensing information of another scan of the touch sensing surface, determine, based on the touch sensing information of the other scan, a second orientation of the first contact, determine a rotation of the first contact based on the first orientation and the second orientation, and generate input of the computing system based on the first orientation, including generating the input based on the rotation.

17. The system of claim 16, wherein the one or more touch processing modules further identify the first contact, wherein generating the input is further based on the identity of the first contact.

18. The system of claim 17, wherein identifying the first contact includes determining an anatomical identity of the first contact, and generating the input based on the identity includes generating the input based on the anatomical identity of the first contact.

19. The system of claim 18, wherein determining the anatomical identity of the first contact includes identifying the first contact as a thumb contact, and generating the input based on the anatomical identity includes generating the input based on the identification of the first contact as a thumb contact.

20. The system of claim 16, wherein generating the input based on the first orientation includes grouping the first contact together with one or more other contacts as a first group based on the first orientation, and basing the input on the first group.

21. The system of claim 20, wherein grouping the first contact and one or more other contacts as a first group includes weighting a first information based on the first orientation to obtain a weighted first information, wherein the first group is based on the weighted first information.

22. The system of claim 21, wherein the first information includes a distance between the first contact and an attractor point of a template, weighting the first information includes weighting the distance to obtain a weighted distance, and grouping the first contact and one or more other contacts includes determining a cost of the first group based on the weighted distance, and wherein the first group is based on the cost.

23. The system of claim 16, wherein generating the input based on the first orientation includes determining, based on the first orientation, an anatomical identity of the first contact, wherein the first input is based on the anatomical identity.

24. The system of claim 23, wherein the one or more touch processing modules further determine a third orientation of a second contact of the set, wherein the anatomical identity of the first contact is further determined based on the third orientation.

25. The system of claim 24, wherein the one or more touch processing modules further evaluate a fuzzy logic expression based on the first and third orientations, wherein the anatomical identity of the first contact is further determined based on the evaluation of the fuzzy logic expression.

26. The system of claim 16, wherein the contact information includes a size of the first contact, and determining the first orientation includes determining the first orientation based on the size of the first contact.

27. The system of claim 26, wherein the one or more touch processing modules further determine whether the size of the first contact is less than a predetermined threshold size, wherein determining the first orientation includes setting the first orientation to a default orientation if the size of the first contact is less than the predetermined threshold size.

28. The system of claim 16, wherein the set includes a plurality of contacts.

29. The system of claim 16, wherein the contact information includes a shape of the first contact, and determining the first orientation includes determining the first orientation based on the shape of the first contact.

30. The system of claim 29, wherein determining the first orientation based on the shape includes fitting an ellipse to the shape, determining an angle of an axis of the fitted ellipse, and basing the first orientation on the angle.

31. A non-transitory computer-readable storage medium storing computer-executable instructions executable to perform a method of input of a computing system including a touch sensing surface, the method comprising:

obtaining touch sensing information of a scan of the touch sensing surface;

determining, based on the touch sensing information of the scan, contact information of a set of one or more contacts, each contact corresponding to a touch object on or near the surface;

determining, based on the contact information, a first orientation of a first contact of the set;

obtaining touch sensing information of another scan of the touch sensing surface;

determining, based on the touch sensing information of the other scan, a second orientation of the first contact;

determining a rotation of the first contact based on the first orientation and the second orientation; and

generating input of the computing system based on the first orientation, including generating the input based on the rotation.

32. The non-transitory computer-readable storage medium of claim 31, the method further comprising:

identifying the first contact, wherein generating the input is further based on the identity of the first contact.

33. The non-transitory computer-readable storage medium of claim 32, wherein identifying the first contact includes determining an anatomical identity of the first contact, and generating the input based on the identity includes generating the input based on the anatomical identity of the first contact.

34. The non-transitory computer-readable storage medium of claim 33, wherein determining the anatomical identity of the first contact includes identifying the first contact as a thumb contact, and generating the input based on the anatomical identity includes generating the input based on the identification of the first contact as a thumb contact.

35. The non-transitory computer-readable storage medium of claim 31, wherein generating the input based on the first orientation includes grouping the first contact together with one or more other contacts as a first group based on the first orientation, and basing the input on the first group.

36. The non-transitory computer-readable storage medium of claim 35, wherein grouping the first contact and one or more other contacts as a first group includes weighting a first information based on the first orientation to obtain a weighted first information, wherein the first group is based on the weighted first information.

37. The non-transitory computer-readable storage medium of claim 36, wherein the first information includes a distance between the first contact and an attractor point of a template, weighting the first information includes weighting the distance to obtain a weighted distance, and grouping the first contact and one or more other contacts includes determining a cost of the first group based on the weighted distance, and wherein the first group is based on the cost.

38. The non-transitory computer-readable storage medium of claim 31, wherein generating the input based on the first orientation includes determining, based on the first orientation, an anatomical identity of the first contact, wherein the first input is based on the anatomical identity.

39. The non-transitory computer-readable storage medium of claim 38, the method further comprising:

determining a third orientation of a second contact of the set, wherein the anatomical identity of the first contact is further determined based on the third orientation.

40. The non-transitory computer-readable storage medium of claim 39, the method further comprising:

evaluating a fuzzy logic expression based on the first and third orientations, wherein the anatomical identity of the first contact is further determined based on the evaluation of the fuzzy logic expression.

41. The non-transitory computer-readable storage medium of claim 31, wherein the contact information includes a size of the first contact, and determining the first orientation includes determining the first orientation based on the size of the first contact.

42. The non-transitory computer-readable storage medium of claim 41, the method further comprising:

determining whether the size of the first contact is less than a predetermined threshold size, wherein determining the first orientation includes setting the first orientation to a default orientation if the size of the first contact is less than the predetermined threshold size.

43. The non-transitory computer-readable storage medium of claim 31, wherein the set includes a plurality of contacts.

44. The non-transitory computer-readable storage medium of claim 31, wherein the contact information includes a shape of the first contact, and determining the first orientation includes determining the first orientation based on the shape of the first contact.

45. The non-transitory computer-readable storage medium of claim 44, wherein determining the first orientation based on the shape includes fitting an ellipse to the shape, determining an angle of an axis of the fitted ellipse, and basing the first orientation on the angle.