Multi-touch finger registration and its applications

• US 9,104,308 B2
• Filed: 12/16/2011
• Issued: 08/11/2015
• Est. Priority Date: 12/17/2010
• Status: Active Grant

• Current Assignee: Hong Kong University of Science and Technology
• Original Assignee: Hong Kong University of Science and Technology
• Inventors: Au, Kin-Chung, Tai, Chiew-Lan
• Primary Examiner(s): Bolotin, Dmitriy
• Application Number:
  US13/328,549
• Publication Number:
  US 20120154313A1
• Time in Patent Office: 1,334 Days
• Field of Search: 345156-184, 178 1801- 2004
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Described herein are finger registration systems and methods for use with a multi-touch device that can facilitate in place execution of commands or operations, thereby reducing focus switching and minimizing hand movement. The finger registration methods are simple and robust, using only contact points on the touch-based device, without requiring additional hardware. Direct mapping of operations to gestures involving individual fingers or combination of fingers (e.g., finger chords), provides a new and rich set of direct manipulation without focus switching for selecting commands/operations.

Claims

1. A method, comprising:

detecting, by a system comprising a processor, a set of contact points corresponding to at least a thumb, an index finger and a middle finger of a hand on a multi-touch device, wherein the hand is associated with a user identity of a set of user identities;

determining, by the system, whether the hand is a left hand or a right hand based on a set of finger spanning angles comprising a sum of any two angles between three lines connecting at least a first contact point of the set of contact points, a second contact point of the set of contact points, and a third contact point of the set of contact points to a reference point, wherein the first contact point corresponds to the index finger, the second contact point corresponds to the middle finger, and the third contact point corresponds to the thumb;

comparing, by the system, the set of finger spanning angles to a set of registration profile data representing respective finger spanning angles of the set of user identities; and

identifying, by the system, the user identity based on a result of the comparing of the set of finger spanning angles to the registration profile data.

2. The method of claim 1, further comprising:

determining, by the system, a set of contact times for the set of contact points;

comparing, by the system, the set of contact times to a threshold time; and

characterizing, by the system, the first contact point, the second contact point, and the third contact point of the set of contact points as distinct when a respective contact time is determined to be greater than the threshold time.

3. The method of claim 1, further comprising:

determining, by the system, a set of distances between the first contact point and the reference point, the second contact point and the reference point, and the third contact point and the reference point;

comparing, by the system, the set of distances to a threshold distance; and

characterizing, by the system, the first contact point, the second contact point, and the third contact point of the set of contact points as distinct when a respective distance measurement of the set of distances is determined to be greater than the threshold distance.

4. The method of claim 1, further comprising determining, by the system, a center point with respect to the set of contact points as the reference point.

5. The method of claim 1, wherein the sum of the any two angles is the sum of a first angle on a left side of a connecting line between a contact point of the set of contact points and the reference point and a second angle on a right side of the connecting line between the contact point of the set of contact points and the reference point.

6. The method of claim 1, further comprising associating, by the system, the thumb with the third contact point of the set of contact points based on the thumb being determined to have a third finger spanning angle of the set of finger spanning angles that is greater than a first finger spanning angle and a third finger spanning angle respectively, wherein the first finger spanning angle and third finger spanning angle correspond to the index finger and the middle finger, respectively.

7. The method of claim 6, wherein the associating further comprises associating the index finger with the first contact point of the set of contact points, and wherein the associating is based on a determination that a first distance between the third contact point and the first contact point is less than a second distance between the third contact point and the second contact point.

8. The method of claim 6, further comprising rejecting, by the system, an association of the thumb with the third contact point based on the third finger spanning angle being determined to be outside a range of acceptable angles.

9. The method of claim 1, wherein the determining further comprises determining whether the set of contact points corresponds to the left hand or the right hand based on a first position of the thumb relative to a second position of the index finger.

10. The method of claim 9, further comprising associating, by the system, the second contact point of the set of contact points with the middle finger, a fourth contact point of the set of contact points with a ring finger, and a fifth contact point of the set of contact points with a pinkie finger, respectively, based on the determining whether the set of contact points corresponds to the left hand or the right hand.

11. The method of claim 1, further comprising concurrently concurrently detecting, by the system, contact points corresponding to at least a first thumb, a first index finger and a first middle finger of the left hand, and a second thumb, a second index finger, and a second middle finger of the right hand on the multi-touch device.

12. The method of claim 11, wherein the left hand and the right hand correspond to different user identities of the set of user identities.

13. The method of claim 1, further comprising presenting, by the system, a set of reorganized and resized user interface elements customized to the user identity, wherein the set of reorganized and resized user interface elements comprises a button, a menu, and a keyboard layout based on the set of finger spanning angles, the sum of the any two angles, a palm size, a set of finger lengths, and a set of fingertip sizes.

14. A non-transitory computer-readable storage medium having stored thereon instructions, the instructions comprising:

detecting a set of contact points corresponding to at least a thumb, an index finger and a middle finger of one or more hands on a multi-touch device;

comparing a set of information related to the set of contact points to reference information of one or more user identities resulting in a set of compared information, wherein the set of information comprises a sum of any two angles between three lines connecting a first contact point, a second contact point, and a third contact point of the set of contact points to a reference point, wherein the first contact point corresponds to the index finger, the second contact point corresponds to the middle finger, and the third contact point corresponds to the thumb;

identifying a user identity from the one or more user identities based on the set of compared information; and

associating a left hand or a right hand, of the one or more hands, corresponding to the user identity with the index finger, the middle finger, and the thumb based on the set of compared information.

15. The non-transitory computer-readable storage medium of claim 14, wherein the associating the respective fingers with the contact points further comprises:

grouping the set of contact points into a first subset of contact points of the set of contact points and a second subset of contact points of the set of contact points based on the set of compared information, wherein the first subset of contact points and the second subset of contact points correspond to the left hand and the right hand, respectively.

16. The non-transitory computer-readable storage medium of claim 14, the instructions further comprising receiving a touch input corresponding to a function of the user interface.

17. The non-transitory computer-readable storage medium of claim 14, the instructions further comprising hiding the user interface after a defined length of time absent touch input received by the user interface.

18. The non-transitory computer-readable storage medium of claim 14, wherein the adjusting the menu, the button, and the keyboard layout of the user interface further comprises adjusting a location, a scale or a spatial arrangement of elements of the user interface based on locations of the set of contact points.

19. A non-transitory computer-readable storage medium of claim 14, the instructions further comprising adjusting a menu, a button, and a keyboard layout of a user interface in accordance with a set of preferences corresponding to the user identity, wherein the set of preferences are based on the set of compared information and the sum of the any two angles.

20. A system, comprising:

a memory to store executable components; and

a processor, coupled to the memory, that executes or facilitates execution of the executable components, the executable components comprising;

a registration component configured to;

detect a set of contact points on a multi-touch device,

group the set of contact points into a first subset of contact points and a second subset of contact points of the set of contact points, and

associate the first subset of contact points and the second subset of contact points with a left hand and a right hand, respectively, wherein the left hand and the right hand are associated with a user identity of a set of user identities;

an orientation component configured to determine an orientation, a scale, a first set of spanning angles comprising a first sum of any two angles between three lines connecting the first subset of contact points to a first reference point, and a second set of spanning angles comprising a second sum of any two angles connecting the second subset of contact points to a second reference point;

a comparing component configured to compare a set of contact point information comprising the orientation, the scale, the first set of spanning angles, and the second set of spanning angles to a set of user registration profile information comprising orientation information, scale information, and spanning angle information corresponding to the set of user identities; and

an identification component configured to identify the user identity associated with a user registration profile of the set of user registration profile information based on a comparison result output from the comparing component.

21. The system of claim 20, wherein the user interface comprises a pop up user interface.

22. The system of claim 20, wherein the user interface comprises a window, a text box or an image box that exists before an assignment of the layout of the user interface.

23. The system of claim 20, wherein the executable components further comprise an interaction component configured to allow a user associated with the user identity to interact with the user interface according to a touch input.

24. The system of claim 20, wherein the executable components further comprise a hiding component configured to hide the user interface for a determined length of time absent touch input received by the user interface.

25. The system of claim 20, wherein the left hand is associated with a first user identity of the set of user identities of a first person and the right hand is associated with a second user identity of the set of user identities of a second person.

26. The system of claim 20, wherein a left keyboard of the user interface corresponds to the left hand and a right keyboard of the user interface corresponds to the right hand.

27. The system of claim 20, wherein the executable components further comprise a layout component configured to adjust a location, a scale or a spatial arrangement of elements of the user interface based on a set of locations of the set of contact points.

28. The system of claim 20, further comprising an assignment component configured to assign a layout of a user interface to the user registration profile based on the comparison result.