Dynamic hand gesture recognition using depth data
First Claim
1. In a computing environment, a method performed at least in part on at least one processor, the method comprising:
- sensing depth data for a plurality of frames that include hand movement;
for the plurality of frames, processing the depth data, wherein processing the depth data comprises;
segmenting the depth data to isolate a hand represented in the plurality of frames;
determining that a principal direction of the hand is not pointing in a predefined direction in an image plane;
based on the determining, rotating the hand such that the palm of the hand is substantially parallel to the image plane and such that the principal direction of the hand is pointing toward the predefined direction in the image plane;
performing normalization on the rotated hand to provide normalized hand data to compensate for scale and a relative hand size of the hand; and
extracting feature values corresponding to the normalized hand data; and
recognizing the hand movement as a hand gesture based upon the feature values provided to a classifier.
2 Assignments
0 Petitions
Accused Products
Abstract
The subject disclosure is directed towards a technology by which dynamic hand gestures are recognized by processing depth data, including in real-time. In an offline stage, a classifier is trained from feature values extracted from frames of depth data that are associated with intended hand gestures. In an online stage, a feature extractor extracts feature values from sensed depth data that corresponds to an unknown hand gesture. These feature values are input to the classifier as a feature vector to receive a recognition result of the unknown hand gesture. The technology may be used in real time, and may be robust to variations in lighting, hand orientation, and the user'"'"'s gesturing speed and style.
13 Citations
20 Claims
-
1. In a computing environment, a method performed at least in part on at least one processor, the method comprising:
-
sensing depth data for a plurality of frames that include hand movement; for the plurality of frames, processing the depth data, wherein processing the depth data comprises; segmenting the depth data to isolate a hand represented in the plurality of frames; determining that a principal direction of the hand is not pointing in a predefined direction in an image plane; based on the determining, rotating the hand such that the palm of the hand is substantially parallel to the image plane and such that the principal direction of the hand is pointing toward the predefined direction in the image plane; performing normalization on the rotated hand to provide normalized hand data to compensate for scale and a relative hand size of the hand; and extracting feature values corresponding to the normalized hand data; and recognizing the hand movement as a hand gesture based upon the feature values provided to a classifier. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. One or more computer-readable storage devices having computer-executable instructions, which when executed perform operations comprising:
processing sensed depth data for a plurality of frames that include hand movement, wherein processing the sensed depth data comprises; segmenting the depth data to isolate a hand represented in the frames of depth data; determining that a principal direction of the hand is not pointing in a predefined direction in an image plane; based on the determining, rotating the hand such that palm of the hand is substantially parallel to an image plane and such that the principal direction of the hand is pointing toward the predefined direction in the image plane; performing normalization on the rotated hand to provide normalized hand data to compensate for scale and a relative hand size of the hand; and extracting feature values corresponding to the normalized hand data; and recognizing the hand movement as a hand gesture based upon the feature values provided to a classifier. - View Dependent Claims (10, 11)
-
12. A system comprising:
-
a memory; a computing device; and a processor programmed to; sense depth data for a plurality of frames that include hand movement; for the plurality of frames, process the depth data, wherein processing the depth data comprises; segmenting the depth data to isolate a hand represented in the plurality of frames; determine that a principal direction of the hand is not pointing in a predefined direction in an image plane; based on the determining, rotate the hand such that the palm of the hand is substantially parallel to the image plane and such that the principal direction of the hand is pointing toward the predefined direction in the image plane; perform normalization on the rotated hand to provide normalized hand data to compensate for scale and relative hand size of the hand; and extract feature values corresponding to the normalized hand data; and recognize the hand movement as a hand gesture based upon the feature values provided to a classifier. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20)
-
Specification