Hand pose recognition
First Claim
Patent Images
1. A hand pose recognition method comprising:
- capturing, with one or more sensors, an image frame of an input hand pose;
determining an initial hand pose estimate for the input hand pose;
performing iterations, each iteration comprising;
identifying a residue for a current hand pose estimate based upon data values for the input hand pose and data values for the current hand pose estimate, wherein the initial hand pose estimate serves as the current hand pose estimate during a first one of the iterations; and
determining a current hand pose estimate based upon the input hand pose and the residue from the previous current hand pose estimate, comprising identifying residue based feature vectors that best match the residue from a general library of feature vectors and adding the identified residue based feature vectors to candidate feature vectors of the current hand pose estimate to form iteration feature vectors;
outputting a hand pose estimate for the image frame of the input hand pose based upon results from the iterations; and
generating one or more control signals, based upon the first image frame hand pose estimate, wherein the input hand pose is transformed to one or more control signals.
1 Assignment
0 Petitions
Accused Products
Abstract
Hand pose recognition comprises determining an initial hand pose estimate for a captured input hand pose and performing iterations based upon hand pose estimates and residues between such estimates and hand pose estimates. One or more control signals are generated based upon the hand pose recognition.
-
Citations
18 Claims
-
1. A hand pose recognition method comprising:
-
capturing, with one or more sensors, an image frame of an input hand pose; determining an initial hand pose estimate for the input hand pose; performing iterations, each iteration comprising; identifying a residue for a current hand pose estimate based upon data values for the input hand pose and data values for the current hand pose estimate, wherein the initial hand pose estimate serves as the current hand pose estimate during a first one of the iterations; and determining a current hand pose estimate based upon the input hand pose and the residue from the previous current hand pose estimate, comprising identifying residue based feature vectors that best match the residue from a general library of feature vectors and adding the identified residue based feature vectors to candidate feature vectors of the current hand pose estimate to form iteration feature vectors; outputting a hand pose estimate for the image frame of the input hand pose based upon results from the iterations; and generating one or more control signals, based upon the first image frame hand pose estimate, wherein the input hand pose is transformed to one or more control signals.
-
-
2. The method of claim 1, wherein determining an initial hand pose estimate for the input hand pose comprises:
-
extracting an input hand pose feature vector from the image frame of the input hand pose; identifying the candidate feature vectors that best match the input hand pose feature vector from the general library of feature vectors for possible hand poses; computing a score for each possible hand pose based upon the candidate feature vectors, wherein the possible hand pose having the highest score is the initial hand pose estimate.
-
-
3. The method of claim 2, wherein identifying candidate feature vectors that best match the input hand pose feature vector from a general library of feature vectors for possible hand poses comprises using a dot product of the input hand pose feature vector and feature vectors of the general library of feature vectors for possible hand poses.
-
4. The method of claim 2, wherein the score for each possible hand pose is computed using a pseudoinverse of the input hand pose feature vector and the candidate feature vectors for each possible hand pose.
-
5. The method of claim 2, wherein determining a new current hand pose estimate based upon the input hand pose and the residue from the previous current hand pose estimate further comprises:
-
identifying best match feature vectors, the best match feature vectors being those iteration feature vectors that best matches the input feature vector; computing a score for each possible hand pose based upon the best match feature vectors, wherein the possible hand pose having the highest score is the new current hand pose estimate.
-
-
6. The method of claim 5 wherein the score for each possible hand pose is computed using a pseudoinverse of the best match feature vectors for each possible hand pose and the input hand pose feature vector.
-
7. The method of claim 5, wherein identifying the best match feature vector comprises:
-
computing a pseudoinverse of the iteration feature vectors; multiplying the pseudoinverse of the iteration feature vectors and the input hand pose feature vector, wherein the best match feature vectors are those iteration feature vectors having the highest scores after their pseudoinverse result is multiplied by the input hand pose feature vector.
-
-
8. The method of claim 1, wherein the initial hand pose estimate has one or more hand pose feature vectors, the method further comprising extracting an input hand pose feature vector from the image frame of the input hand pose.
-
9. The method of claim 1, wherein the image frame hand pose estimate is output in response to a predetermined convergence of the current hand pose estimates from different iterations.
-
10. The method of claim 9, wherein the predetermined convergence comprises the new current hand pose class estimate being same as the previous current hand pose estimate, the residue associated with the new hand pose estimate being less than the residue associated with the previous hand pose estimate and a sum of differences between new current hand pose class estimates and previous current hand pose class estimates for the iterations being less than a predetermined threshold.
-
11. The method of claim 1 further comprising:
-
generating a plurality of image frame hand pose estimates including the first image frame hand pose class estimate; determining a final hand pose estimate based upon the plurality of image frame hand pose estimates, wherein the control signals are generated with the processor based upon the final hand pose estimate.
-
-
12. The method of claim 11 further comprising rejecting a portion of the plurality of image frame hand pose estimates, wherein determining the final hand pose the estimate is based upon the plurality of image frame hand pose estimates excluding the rejected portion of the plurality of image frame hand pose estimates.
-
13. The method of claim 1, wherein the initial hand pose estimate is derived from a hand pose library of possible hand poses, the hand pose library comprising a matrix of one or more hand pose feature vectors for each possible hand pose.
-
14. The method of claim 13 further comprising adding a new possible hand pose to the hand pose library by appending one or more new hand pose feature vectors for the new possible hand pose to the matrix.
-
15. The method of claim 14 further comprising:
-
capturing, with one or more sensors, an image frame of the new possible hand pose; extracting a new possible hand pose feature vector from the image frame of the new possible hand pose; and determining whether the new possible hand pose feature vectors are too close to existing hand pose feature vectors of the library.
-
-
16. The method of claim 1, wherein capturing the first image frame of the input hand pose comprises:
-
capturing a color/grayscale image frame having color/grayscale features of the input hand pose; capturing a depth image frame having depth features of the input hand pose, wherein the depth image frame is captured concurrently with the captured color/grayscale frame; and forming a single input hand pose feature vector based upon the captured depth image frame and the captured color/grayscale image frame.
-
-
17. A hand pose recognition method comprising:
generating a plurality of image frame hand pose estimates for an input hand pose, each image frame hand pose estimate being generated by; capturing, with one or more sensors, an image frame of the input hand pose; determining an initial hand pose estimate for the input hand pose by; extracting an input hand pose feature vector from the image frame of the input hand pose; identifying candidate feature vectors that best match the input hand pose feature vector from a general library of hand pose feature vectors for possible hand poses; computing a score for each possible hand pose based upon the candidate feature vectors, wherein the possible hand pose having the highest score is the initial hand pose estimate; performing iterations, each iteration comprising; identifying a residue for a current hand pose estimate based upon data values for the input hand pose and data values for the current hand pose estimate, wherein the initial hand pose estimate serves as the current hand pose estimate during a first one of the iterations; and determining a new current hand pose estimate based upon the input hand pose and the residue from the previous current hand pose estimate by; determining a new current hand pose estimate based upon the input hand pose and the residue from the previous current hand pose estimate comprises; identifying residue based feature vectors that best match the residue from the general library of feature vectors; adding the identified residue based feature vectors to the candidate feature vectors of the current hand pose estimate to form iteration feature vectors; identifying best match feature vectors, the best match feature vectors being those iteration feature vectors that best matches the input feature vector; and computing a score for each possible hand pose based upon the best match feature vectors, wherein the possible hand pose having the highest score is the new current hand pose estimate; storing the image frame hand pose estimate for the image frame of the input hand pose in response to a predetermined convergence of the current hand pose estimates from different iterations; determining a final hand pose estimate based upon the plurality of image frame hand pose estimates; and generating one more control signals, with a processor, based upon the final hand pose estimate, wherein the input hand pose this transformed to the one or more control signals.
-
18. A hand pose recognition system comprising:
-
one or more sensors configured to capture an image frame of an input hand pose; a memory containing a library of possible hand poses; one or more processors in communication with any of the one or more sensors and the memory, the one or more processors configured to; determine an initial hand pose estimate for the input hand pose; perform iterations, each iteration comprising; identifying a residue for a current hand pose estimate based upon data values for the input hand pose and data values for the current hand pose estimate, wherein the initial hand pose estimate serves as the current hand pose estimate during a first one of the iterations; and determining a new hand pose estimate based upon the input hand pose and the residue from the previous current hand pose estimate; output a hand pose estimate for the image frame of the input hand pose based upon results from the iterations; and generate one more control signals based upon the hand pose estimate, wherein the image frame hand pose estimate is output in response to a predetermined convergence of the current hand pose estimates from different iterations, the predetermined convergence comprises the new current hand pose class estimate being same as the previous current hand pose estimate, the residue associated with the new hand pose estimate being less than the residue associated with the previous hand pose estimate and a sum of differences between new current hand pose class estimates and previous current hand pose class estimates for the iterations being less than a predetermined threshold.
-
Specification