Contact tracking and identification module for touch sensing

• US 8,730,177 B2
• Filed: 07/30/2007
• Issued: 05/20/2014
• Est. Priority Date: 01/26/1998
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Westerman, Wayne, Elias, John G.
• Primary Examiner(s): Sim, Yong H
• Application Number:
  US11/830,766
• Publication Number:
  US 20080041639A1
• Time in Patent Office: 2,486 Days
• Field of Search: 345173-184, 178/18.01, 178/18.06
• US Class Current: 345/173
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0235 using chord techniques G06F3/0234 takes precedence
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/1983 Syntactic or structural pattern recognition, e.g. symbolic string recognition
• CPC Class Codes: G06V 40/107 : Static hand or arm
• CPC Class Codes: H03K 17/9622 : using a plurality of detectors, e.g. keyboard
• CPC Class Codes: H03K 2217/960725 : Charge-transfer

Abstract

Apparatus and methods are disclosed for simultaneously tracking multiple finger and palm contacts as hands approach, touch, and slide across a proximity-sensing, multi-touch surface. Identification and classification of intuitive hand configurations and motions enables unprecedented integration of typing, resting, pointing, scrolling, 3D manipulation, and handwriting into a versatile, ergonomic computer input device.

Claims

1. A method of determining input events of a computing system having a touch sensing surface, the method comprising:

tracking multiple distinguishable contacts, the contacts corresponding to touch devices concurrently on or near the surface, wherein motion of a first contact and motion of a second contact are tracked in a first period during which the first and second contacts remain touched down on the surface;

detecting a liftoff of the first contact from the surface while the second contact remains touched down on the surface, wherein the motion of the second contact continues to be tracked during a second period after the liftoff of the first contact; and

determining at least two input events, including determining a first input event during the first period and determining a second input event during the second period, wherein the first input event corresponds to the tracked motion of the first contact and is not based on the tracked motion of the second contact which is not stationary, and the second input event corresponds to the tracked motion of the second contact.

2. The method of claim 1, wherein the tracking further comprises tracking multiple distinguishable contacts using successive proximity images of the contacts.

3. The method of claim 2, further comprising a scanning a plurality of capacitive sensors associated with the touch sensing surface to generate the proximity images.

10. The method of claim 1, further comprising:

associating the first input event and the second input event with different commands.

11. The method of claim 10, wherein the first input event is associated with an inking command such that the inking command is based on the tracked motion of the first contact, and the second input event is associated with a cursor positioning command such that the cursor positioning command is based on the tracked motion of the second contact.

12. The method of claim 11, wherein determining at least two input events further includes determining a third input event, wherein the third input event corresponds to the detected liftoff of the first contact, the method further comprising associating the third input event with a command instructing a break in the inking command.

13. The method of claim 1, wherein determining at least two input events further includes determining a third input event, the third input event corresponding to the detected liftoff of the first contact.

14. The method of claim 1, further comprising:

transmitting tracked motion to one or more command processing modules of the computing system, including transmitting the first input event to one of the one or more command processing modules in the first period, wherein the command processing module associates the first input event with a first command.

15. The method of claim 14, wherein the tracked motion of the second contact is excluded from transmission to the one or more command processing modules during the first period.

16. The method of claim 15, wherein transmitting tracked motion to the one or more command processing modules further includes transmitting the second input event to a different one of the command processing modules than the command processing module to which the first input event was transmitted, wherein the different one of the command processing modules associates the second input event with a second command, different from the first command.

17. The method of claim 16, wherein the first contact includes at least one inner finger of a pen grip configuration of a hand, the second contact includes at least one of a knuckle and palm of the pen grip configuration, the first input event includes path parameters of the at least one inner finger, the second input event includes at least one of a knuckle motion and a palm motion, the command processing module to which the first input event is transmitted is a handwriting recognizer, and the command processing module to which the second input event is transmitted is a cursor motion module.

18. The method of claim 1, wherein the tracked motion of the second contact is further excluded from determination of input events for a predetermined time after the liftoff of the first contact, such that the second period begins after the predetermined time after the liftoff.

19. The method of claim 18, wherein the predetermined time is approximately a few hundred milliseconds.

20. The method of claim 1, further comprising:

selecting an input mode and maintaining the selected input mode continuously during the first period, the liftoff of the first contact, and the second period; and

associating each of the first and second input events with a command corresponding to the selected input mode.

21. The method of claim 20, wherein the input mode includes a pen grip mode.

4. An apparatus for determining input events of a computing system having a touch sensing surface, the apparatus comprising:

a tracking module that tracks contacts corresponding to touch devices on or near the surface, wherein motion of a first contact and motion of a second contact are tracked in a first period during which the first and second contacts are concurrently touched down on the surface, a liftoff of the first contact from the surface is determined while the second contact remains touched down on the surface, and the motion of the second contact continues to be tracked during a second period after the liftoff of the first contact; and

an input event recognizer that determines at least two input events, including determining a first input event during the first period and determining a second input event during the second period, wherein the first input event corresponds to the tracked motion of the first contact and is not based on the tracked motion of the second contact which is not stationary, and the second input event corresponds to the tracked motion of the second contact.

5. The apparatus of claim 4, wherein the tracking module tracks the contacts using successive proximity images of the contacts.

6. The apparatus of claim 5, further comprising a scanner that successively scans a plurality of capacitive sensors associated with the touch sensing surface to generate the proximity images.

22. The apparatus of claim 4, wherein the input event recognizer further determines a third input event based on the detected liftoff of the first contact.

23. The apparatus of claim 4, further comprising:

one or more command processing modules that associate input events with one or more commands,

wherein the input event recognizer further transmits the first input event to one of the one or more command processing modules in the first period, and the command processing module associates the first input event with a first command.

24. The apparatus of claim 23, wherein the input event recognizer further excludes tracked motion of the second contact from transmission to the one or more command processing modules during the first period.

25. The apparatus of claim 24, wherein the input event recognizer further transmits the second input event to a different one of the command processing modules than the command processing module to which the first input event was transmitted, and the different one of the command processing modules associates the second input event with a second command, different from the first command.

26. The apparatus of claim 25, wherein the first contact includes at least one inner finger of a pen grip configuration of a hand, the second contact includes at least one of a knuckle and palm of the pen grip configuration, the first input event includes path parameters of the at least one inner finger, the second input event includes at least one of a knuckle motion and a palm motion, the command processing module to which the first input event is transmitted is a handwriting recognizer, and the command processing module to which the second input event is transmitted is a cursor motion module.

27. The apparatus of claim 4, wherein the input event recognizer further excludes the tracked motion of the second contact from determination of input events for a predetermined time after the liftoff of the first contact, such that the second period begins after the predetermined time after the liftoff.

28. The apparatus of claim 27, wherein the predetermined time is approximately a few hundred milliseconds.

29. The apparatus of claim 4, further comprising:

an input mode selection module that selects an input mode, maintains the selected input mode continuously during the first period, the liftoff of the first contact, and the second period, and associates each of the first and second input events with a command corresponding to the selected input mode.

30. The apparatus of claim 29, wherein the input mode includes a pen grip mode.

7. A non-transitory computer readable storage medium having stored therein instructions, which when executed by a device with a touch sensing surface, cause the device to perform a method comprising:

tracking multiple distinguishable contacts, the contacts corresponding to touch devices concurrently on or near the surface, wherein motion of a first contact and motion of a second contact are tracked in a first period during which the first and second contacts remain touched down on the surface;

detecting a liftoff of the first contact from the surface while the second contact remains touched down on the surface, wherein the motion of the second contact continues to be tracked during a second period after the liftoff of the first contact; and

determining at least two input events, including determining a first input event during the first period and determining a second input event during the second period, wherein the first input event corresponds to the tracked motion of the first contact and is not based on the tracked motion of the second contact which is not stationary, and the second input event corresponds to the tracked motion of the second contact.

8. The non-transitory computer readable storage medium of claim 7, further comprising instructions to track the multiple distinguishable contacts using successive proximity images of the contacts.

9. The non-transitory computer readable storage medium of claim 7, further comprising instructions to cause the device to successively scan a plurality of capacitive sensors associated with the touch sensing surface to generate the proximity images.

31. The non-transitory computer readable storage medium of claim 7, the method further comprising:

associating the first input event and the second input event with different commands.

32. The non-transitory computer readable storage medium of claim 31, wherein the first input event is associated with an inking command such that the inking command is based on the tracked motion of the first contact, and the second input event is associated with a cursor positioning command such that the cursor positioning command is based on the tracked motion of the second contact.

33. The non-transitory computer readable storage medium of claim 32, wherein determining at least two input events further includes determining a third input event, wherein the third input event corresponds to the detected liftoff of the first contact, the method further comprising associating the third input event with a command instructing a break in the inking command.

34. The non-transitory computer readable storage medium of claim 7, wherein determining at least two input events further includes determining a third input event, the third input event corresponding to the detected liftoff of the first contact.

35. The non-transitory computer readable storage medium of claim 7, the method further comprising:

transmitting tracked motion to one or more command processing modules of the computing system, including transmitting the first input event to one of the one or more command processing modules in the first period, wherein the command processing module associates the first input event with a first command.

36. The non-transitory computer readable storage medium of claim 35, wherein the tracked motion of the second contact is excluded from transmission to the one or more command processing modules during the first period.

37. The non-transitory computer readable storage medium of claim 36, wherein transmitting tracked motion to the one or more command processing modules further includes transmitting the second input event to a different one of the command processing modules than the command processing module to which the first input event was transmitted, wherein the different one of the command processing modules associates the second input event with a second command, different from the first command.

38. The non-transitory computer readable storage medium of claim 37, wherein the first contact includes at least one inner finger of a pen grip configuration of a hand, the second contact includes at least one of a knuckle and palm of the pen grip configuration, the first input event includes path parameters of the at least one inner finger, the second input event includes at least one of a knuckle motion and a palm motion, the command processing module to which the first input event is transmitted is a handwriting recognizer, and the command processing module to which the second input event is transmitted is a cursor motion module.

39. The non-transitory computer readable storage medium of claim 7, wherein the tracked motion of the second contact is further excluded from determination of input events for a predetermined time after the liftoff of the first contact, such that the second period begins after the predetermined time after the liftoff.

40. The non-transitory computer readable storage medium of claim 39, wherein the predetermined time is approximately a few hundred milliseconds.

41. The non-transitory computer readable storage medium of claim 7, the method further comprising:

selecting an input mode and maintaining the selected input mode continuously during the first period, the liftoff of the first contact, and the second period; and

associating each of the first and second input events with a command corresponding to the selected input mode.

42. The non-transitory computer readable storage medium of claim 41, wherein the input mode includes a pen grip mode.