HAND AND FINGER REGISTRATION FOR CONTROL APPLICATIONS

• US 20130207920A1
• Filed: 06/22/2011
• Published: 08/15/2013
• Est. Priority Date: 08/20/2010
• Status: Active Grant

• Current Assignee: University Of Massachusetts
• Original Assignee: University Of Massachusetts
• Inventors: McCann, Eric, Micire, Mark J., Yanco, Holly A., Norton, Adam D.
• Granted Patent: US 9,013,430 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Robot control typically requires many physical joysticks, buttons and switches. We have created a dynamically resizing, ergonomic, and multi-touch (DREAM) controller that enables a software-based operator control unit (SoftOCU) using a multi-touch screen. The DREAM controller can be formed wherever a person places his or her hand on a touch screen. We have also developed an algorithm for accurate hand and finger registration that can be used to create the DREAM controller.

Claims

1. A method of identifying one or more fingers of a hand, the method comprising:

calculating a center position of the hand based at least on points corresponding to a plurality of the fingers;

calculating angles between the points based at least on positions of the points and the center position of the hand; and

identifying, based on the angles, which one or more fingers of the hand correspond to one or more of the points.

2. The method of claim 1, further comprising:

receiving the points, wherein the points represent locations of contact between the fingers and a surface.

3. The method of claim 1, wherein the center position is calculated by determining a center of a bounding box around the points.

4. The method of claim 3, further comprising:

calculating a slope corresponding to non-thumb points of the hand, wherein the bounding box is determined based on the slope and the positions of the points.

5. The method of claim 1, wherein the center position is calculated by determining an average or weighted average of the points.

6. The method of claim 1, further comprising:

determining a largest angle between any two nearest points;

determining a second largest angle between any two nearest points,

wherein the step of identifying comprises identifying a first point at an intersection of the largest and second largest angles as corresponding to a thumb.

7. The method of claim 6, wherein the step of identifying further comprises identifying a second point separated from the first point by the largest angle as corresponding to a little finger.

8. The method of claim 7, wherein the step of identifying further comprises identifying a third point separated from the first point by the second largest angle as corresponding to an index finger.

9. The method of claim 8, wherein the step of identifying further comprises identifying a fourth point nearest the third point as corresponding to a middle finger.

10. The method of claim 8, wherein the step of identifying further comprises identifying a fifth point nearest the second point as corresponding to a ring finger.

11. The method of claim 1, wherein the method is initiated on a repeated basis with a timing independent of times at which the fingers are detected.

12. At least one computer readable storage medium having stored thereon instructions, which, when executed, perform the method of claim 1.

13. A method of determining whether fingers correspond to a right hand or a left hand, the method comprising:

determining a first vector along a first direction between a first point, corresponding to a first finger, and a second point corresponding to a center of a hand;

determining a second vector along a second direction between a third point, corresponding to a second finger, and the second point;

calculating a cross product between the first and second vectors; and

determining whether the fingers correspond to a right hand or a left hand based on the cross product.

14. The method of claim 13, wherein the first point corresponds to detection of the first finger being in contact with a surface and the second point corresponds to detection of the second finger being in contact with the surface, and the method further comprises:

determining a sign of the cross product, wherein the sign of the cross product indicates whether the hand is a right hand or a left hand.

15. A method of determining whether fingers correspond to a right hand or a left hand, the method comprising:

identifying a first finger of a hand as corresponding to a first point;

identifying a second finger of the hand as corresponding to a second point;

determining a third point representing a center position of the hand; and

determining whether the hand is a right hand or a left hand based on the first, second and third points.

16. A method of providing a graphical user interface, the method comprising:

identifying which fingers of a hand correspond to a plurality of points; and

displaying the graphical user interface based on positions of the plurality of points and information identifying which fingers of the hand correspond to the plurality of points.

17. The method of claim 16, wherein displaying the graphical user interface comprises displaying a virtual joystick at a position corresponding to a thumb.

18. The method of claim 16, wherein displaying the graphical user interface comprises displaying a button at a position corresponding to an index finger and/or at a position corresponding to a middle finger.

19. The method of claim 16, further comprising moving a position at which the graphical user interface is displayed to track movement of the hand.

20. The method of claim 16, wherein displaying the graphical user interface comprising displaying at least a portion of a virtual keyboard.

21. At least one computer readable storage medium having stored thereon instructions, which, when executed, perform the method of claim 16.

22. The method of claim 16, further comprising:

determining whether the hand is a right hand or a left hand; and

if the hand is a right hand, displaying one or more controls to the left of a position between the thumb and index finger;

if the hand is a left hand, displaying one or more controls to the right of a position between the thumb and index finger.

23. The method of claim 16, further comprising:

determining whether the hand is a right hand or a left hand; and

if the hand is a right hand, displaying a video display to the left of a position between the thumb and index finger;

if the hand is a left hand, displaying a video display to the right of a position between the thumb and index finger.

24. The method of claim 16, further comprising:

displaying a circle around the hand.

25. A device for identifying fingers, comprising:

an apparatus that detects points corresponding to a plurality of fingers of a hand; and

a microprocessor that determines, based on angles between the points corresponding to different fingers, which one or more fingers of the hand correspond to one or more of the points.

26. The computing device of claim 25, wherein the apparatus comprises a touch-sensitive screen configured to display a graphical user interface under the hand based on positions of the points and the angles between the points.