Hand pointing estimation for human computer interaction
First Claim
1. A method for determining a finger pointing target, comprising:
- receiving a plurality of images comprising a sequence of images over time, including images acquired from different perspectives representing a hand having a finger with a finger pointing direction;
tracking a plurality of landmarks on the hand comprising at least two visible landmarks of the finger having the finger pointing direction, a center of the hand, and a position of a wrist proximate to the hand, in a series of images, the respective landmarks being derived from common characteristics of hands from a plurality of different people;
determining a best fit for visible landmarks of the hand in each of the series of images to an active appearance model;
inferring a three dimensional position of each visible landmark in the series of images;
determining the finger pointing directional vector of the finger based on the inferred three dimensional position of each visible landmark in the series of images; and
determining a finger pointing gesture based on at least a change in the determined directional vector over time.
1 Assignment
0 Petitions
Accused Products
Abstract
Hand pointing has been an intuitive gesture for human interaction with computers. A hand pointing estimation system is provided, based on two regular cameras, which includes hand region detection, hand finger estimation, two views'"'"' feature detection, and 3D pointing direction estimation. The technique may employ a polar coordinate system to represent the hand region, and tests show a good result in terms of the robustness to hand orientation variation. To estimate the pointing direction, Active Appearance Models are employed to detect and track, e.g., 14 feature points along the hand contour from a top view and a side view. Combining two views of the hand features, the 3D pointing direction is estimated.
-
Citations
20 Claims
-
1. A method for determining a finger pointing target, comprising:
-
receiving a plurality of images comprising a sequence of images over time, including images acquired from different perspectives representing a hand having a finger with a finger pointing direction; tracking a plurality of landmarks on the hand comprising at least two visible landmarks of the finger having the finger pointing direction, a center of the hand, and a position of a wrist proximate to the hand, in a series of images, the respective landmarks being derived from common characteristics of hands from a plurality of different people; determining a best fit for visible landmarks of the hand in each of the series of images to an active appearance model; inferring a three dimensional position of each visible landmark in the series of images; determining the finger pointing directional vector of the finger based on the inferred three dimensional position of each visible landmark in the series of images; and determining a finger pointing gesture based on at least a change in the determined directional vector over time. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method for determining a hand gesture, comprising
capturing a plurality of images of a hand from different perspectives; -
determining a center of the hand and a position of a wrist associated with the hand in each image; finding a best fit for the hand in each image to an active appearance model; inferring a three dimensional position of a plurality of landmarks on the hand, corresponding to landmarks of human hands in a training image set to which a principal component analysis is applied to formulate a statistical model of the human hand; combining the fit of the hand to the active appearance model in each image, with the inferred three dimensional position of a plurality of landmarks on the hand, to automatically determine at least one directional vector defined by the plurality of landmarks; and determining a gesture based on a change in the at least one directional vector over time. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17)
-
-
18. An apparatus for estimating a finger pointing direction, comprising:
-
a memory configured to store an appearance hand model which tracks a plurality of landmarks on the hand, derived from characteristics of a plurality of different hands; a memory configured to store a plurality of images acquired from different angles representing a hand in a sequence of images over time; at least one automated processor, configured to; detect the hand in each of the plurality of images; locate a center of the hand and a position of a wrist proximate to the hand in each of the plurality of images; apply the active appearance model to find a best fit for visible landmarks of the hand in each of the plurality of images; combine the best fit for the visible landmarks of the hand in each of the plurality of images to the active appearance model, to infer a three dimensional position of each visible landmark in each of the plurality of images; determine a sequence of directional vector over time associated with the inferred three dimensional position of each visible landmark; and determine a gesture based on at least a change in the determined sequence of directional vector over time. - View Dependent Claims (19, 20)
-
Specification