Methods and apparatus for determining local coordinate frames for a human hand

• US 8,553,001 B2
• Filed: 03/22/2011
• Issued: 10/08/2013
• Est. Priority Date: 03/22/2011
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Krishnaswamy, Aravind
• Primary Examiner(s): Lao, Lun-Yi
• Assistant Examiner(s): McLoone, Peter D
• Application Number:
  US13/069,197
• Publication Number:
  US 20130127733A1
• Time in Patent Office: 931 Days
• Field of Search: 345/104, 345173-179, 715/773, 434/37, 178/18.01, 178/18.03, 178/18.06, 178/19.01, 178/19.03
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Methods and apparatus for providing a local coordinate frame are described. A user may place the tips of all five digits on a multitouch user interface with a natural gesture to provide five touch points. A local coordinate frame may be constructed from the touch points. A local coordinate frame may be computed relative to the entire hand and/or relative to each individual digit. The technique may be agnostic in regard to the orientation of the device and to which hand is used. The user may move or rotate their hand, and the local coordinate frame is adjusted to the new positioning relative to the device. The technique may also infer a relative position of the palm from the digit input points, allowing the display of user interface elements where the elements are visible rather than occluded by the hand.

Claims

1. A method, comprising:

receiving, on a multitouch-enabled device, multitouch gesture input indicating contact of a plurality of digits of a human hand with a multitouch-sensitive screen of the device, the input indicating a plurality of touch points each corresponding to different ones of said plurality of digits on the screen;

calculating straight line distances from each of said plurality of touch points to every other of said plurality of touch points;

for each touch point, determining a mean of the straight line distances to the other touch points;

assigning the touch point with a largest mean straight line distance to the other touch points as a thumb touch point; and

assigning each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points.

2. The method as recited in claim 1, wherein said assigning each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points comprises assigning the touch point with a smallest distance to the thumb touch point as an index finger touch point.

3. The method as recited in claim 1, wherein said assigning each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points comprises:

determining an index finger touch point; and

assigning the touch point with a smallest straight line distance to the index finger touch point that is not the thumb touch point as a middle finger touch point.

4. The method as recited in claim 1, wherein said assigning each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points comprises:

determining an index finger touch point and a middle finger touch point; and

assigning the touch point with a smallest straight line distance to the middle finger touch point that is not the thumb touch point or the index finger touch point as a ring finger touch point.

5. The method as recited in claim 1, wherein said assigning each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points comprises:

determining an index finger touch point, a middle finger touch point, and a ring finger touch point; and

assigning a remaining touch point as a pinky finger touch point.

6. The method as recited in claim 1, wherein said assigning each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points comprises:

assigning the touch point with a smallest distance to the thumb touch point as an index finger touch point;

assigning the touch point with a smallest distance to the index finger touch point that is not the thumb touch point as a middle finger touch point;

assigning the touch point with a smallest distance to the middle finger touch point that is not the thumb touch point or the index finger touch point as a ring finger touch point; and

assigning a remaining touch point as a pinky finger touch point.

7. The method as recited in claim 1, wherein the human hand is either a left hand or a right hand.

8. The method as recited in claim 1, further comprising estimating a location of a palm of the human hand based on the assigned touch points.

9. The method as recited in claim 8, wherein said estimating the location of the palm of the human hand based on the assigned touch points comprises assigning a location of the palm as a midpoint on a line between a pinky touch point and the thumb touch point.

10. The method as recited in claim 1, further comprising determining an up vector for the human hand based on the assigned touch points, the up vector indicating an orientation and a direction of the human hand.

11. The method as recited in claim 10, wherein said determining the up vector for the human hand based on the assigned touch points comprises determining the up vector from orthogonals of vectors connecting two or more touch points.

12. The method as recited in claim 1, further comprising determining an up vector for each of the plurality of digits based on the assigned touch points, the up vector for a respective digit indicating an orientation and a direction of the respective digit.

13. The method as recited in claim 1, wherein the multitouch gesture input does not indicate an ordering of the plurality of touch points relative to the plurality of digits.

14. A multitouch-enabled device, comprising:

at least one processor;

a multitouch-enabled screen; and

a memory comprising program instructions, the program instructions executable by the at least one processor to;

receive multitouch gesture input indicating contact of a plurality of digits of a human hand with the multitouch-sensitive screen, the input indicating a plurality of touch points each corresponding to different ones of said plurality of digits on the screen;

calculate straight line distances from each of said plurality of touch points to every other of said plurality of touch points;

for each touch point, determine a mean of the straight line distances to the other touch points;

assign the touch point with a largest mean straight line distance to the other touch points as a thumb touch point; and

assign each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points.

15. The multitouch-enabled device as recited in claim 14, wherein, to assign each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points, the program instructions are executable by the at least one processor to:

assign the touch point with a smallest straight line distance to the thumb touch point as an index finger touch point;

assign the touch point with a smallest distance to the index finger touch point that is not the thumb touch point as a middle finger touch point;

assign the touch point with a smallest distance to the middle finger touch point that is not the thumb touch point or the index finger touch point as a ring finger touch point; and

assign a remaining touch point as a pinky finger touch point.

16. The multitouch-enabled device as recited in claim 14, wherein the program instructions are executable by the at least one processor to estimate a location of a palm of the human hand based on the assigned touch points.

17. The multitouch-enabled device as recited in claim 14, wherein the program instructions are executable by the at least one processor to:

determine an up vector for the human hand based on the assigned touch points, the up vector indicating an orientation and a direction of the human hand; and

determine additional up vectors for each of the plurality of digits based on the assigned touch points, the additional up vector for a respective digit indicating an orientation and a direction of the respective digit.

18. A computer-readable storage device having program instructions stored thereon that, responsive to execution, perform a method comprising:

receiving multitouch gesture input indicating contact of a plurality of digits of a human hand with a multitouch-enabled device, the input indicating a plurality of touch points each corresponding to different ones of said plurality of digits on the screen;

calculating straight line distances from each of said plurality of touch points to every other of said plurality of touch points;

for each touch point, determine a mean of the straight line distances to the other touch points;

assigning the touch point with a largest mean distance to the other touch points as a thumb touch point; and

assigning each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points.

19. The computer-readable storage device as recited in claim 18, wherein the program instructions, responsive to execution, perform said assigning each of the other plurality of touch points to particular digits based on the thumb touch point and straight line distances between the touch points by:

assigning the touch point with a smallest distance to the thumb touch point as an index finger touch point;

assigning the touch point with a smallest distance to the index finger touch point that is not the thumb touch point as a middle finger touch point;

assigning the touch point with a smallest distance to the middle finger touch point that is not the thumb touch point or the index finger touch point as a ring finger touch point; and

assigning a remaining touch point as a pinky finger touch point.

20. The computer-readable storage device as recited in claim 18, wherein the program instructions, responsive to execution, perform a method further comprising estimating a location of a palm of the human hand based on the assigned touch points.

21. The computer-readable storage device as recited in claim 18, wherein the program instructions, responsive to execution, perform a method further comprising:

determining an up vector for the human hand based on the assigned touch points, the up vector indicating an orientation and a direction of the human hand; and

determining additional up vectors for each of the plurality of digits based on the assigned touch points, the additional up vector for a respective digit indicating an orientation and a direction of the respective digit.