HAND GESTURE RECOGNITION SYSTEM

• US 20140147035A1
• Filed: 04/11/2011
• Published: 05/29/2014
• Est. Priority Date: 04/11/2011
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Li, Wenlong, Zhang, Yimin, Tong, Xiaofeng, Li, Jianguo, Ding, Dayaong
• Granted Patent: US 8,781,221 B2
• Time in Patent Office: Days
• US Class Current: 382/164
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A cost-effective and computationally efficient hand gesture recognition system for detecting and/or tracking a face region and/or a hand region in a series of images. A skin segmentation model is updated with skin pixel information from the face and iteratively applied to the pixels in the hand region, to more accurately identify the pixels in the hand region given current lighting conditions around the image. Shape features are then extracted from the image, and based on the shape features, a hand gesture is identified in the image. The identified hand gesture may be used to generate a command signal to control the operation of an application or system.

Claims

1. A method for determining a hand gesture in an image, comprising:

detecting, by a face detection module, a facial region in the image;

detecting, by a hand detection module, a hand region including a hand gesture in the image;

segmenting, using a Gaussian skin color model, the facial region into a binary image;

updating, using pixels of the facial region, at least one variable of the Gaussian skin color model;

segmenting, using the Gaussian skin color model with the at least one updated variable, the hand region in the image into a binary image;

determining, using the binary image of the hand region, at least one shape feature associated with the hand gesture; and

determining, using the at least one shape feature, the hand gesture in the binary image.

2. The method of claim 1, wherein:

the at least one variable includes a mean variable or a covariance variable.

3. The method of claim 1, wherein:

the at least one shape feature, as applied to the hand gesture in the image, is selected from the group of eccentricity, compactness, orientation, rectangularity, width center, height center, the number of defects, difference between left and right parts, difference between top and bottom parts, and orientation.

4. The method of claim 1, wherein the image is a red, green, blue (RGB) color image, the method further comprising:

for each RGB image in a series of RGB images;

converting, by a hand tracking module, the RGB image into a hue, saturation, value (HSV) image;

extracting, by the hand tracking module, a color histogram of the hand region in the HSV image; and

tracking, using mean-shifting, the hand region in the series of HSV images.

5. The method of claim 1, wherein determining the hand gesture comprising:

assigning, by a hand gesture recognition module, at least one shape feature to an input node of a multilayer perceptron (MLP) model;

mapping, by a hand gesture recognition module, the input node to an output neuron of the MLP model; and

determining, by a hand gesture recognition module using the MLP model, a probability that the output neuron is one of a predefined hand gesture shape.

6. The method if claim 1, further comprising:

generating a control signal indicative of the determined hand gesture.

7. A system for determining a hand gesture in an image, comprising:

a face detection module configured to detect a facial region in the image;

a hand detection module configured to detect a hand region including a hand gesture in the image;

a skin segmentation module configured to;

segment, using a Gaussian skin color model, the facial region into a binary image;

update, using pixels of the facial region, at least one variable of the Gaussian skin color model; and

segment, using the Gaussian skin color model with the at least one updated variable, the hand region in the image into a binary image;

a shape feature extraction module configured to determine, using the binary image of the hand region, at least one shape feature associated with the hand gesture; and

a hand gesture recognition module configured to determine, using the at least one shape feature, the hand gesture in the binary image.

8. The system of claim 7, wherein:

the at least one variable includes a mean variable or a covariance variable.

9. The method of claim 7, wherein:

the at least one shape feature, as applied to the hand gesture in the image, is selected from the group of eccentricity, compactness, orientation, rectangularity, width center, height center, the number of defects, difference between left and right parts, difference between top and bottom parts, and orientation.

10. The system of claim 7, wherein the image is a red, green, blue (RGB) color image, the system further comprising:

a hand tracking module configured to, for each RGB image in a series of RGB images;

convert the RGB image into a hue, saturation, value (HSV) image;

extract a color histogram of the hand region in the HSV image; and

track, using mean-shifting, the hand region in the series of HSV images.

11. The system of claim 7, wherein the hand gesture recognition module is further configured to:

assign at least one shape feature to an input node of a multilayer perceptron (MLP) model;

map the input node to an output neuron of the MLP model; and

determine, using the MLP model, a probability that the output neuron is one of a predefined hand gesture shape.

12. The system of claim 7, wherein the hand gesture recognition module is further configured to:

generate a control signal indicative of the determined hand gesture.

13. A tangible computer-readable medium including instructions stored thereon which, when executed by one or more processors, cause the computer system to perform operations comprising:

detecting a facial region in an image;

detecting a hand region including a hand gesture in the image;

segmenting, using a Gaussian skin color model, the facial region into a binary image;

updating, using pixels of the facial region, at least one variable of the Gaussian skin color model;

segmenting, using the Gaussian skin color model with the at least one updated variable, the hand region in the image into a binary image;

determining, using the binary image of the hand region, at least one shape feature associated with the hand gesture; and

determining, using the at least one shape feature, the hand gesture in the binary image.

14. The tangible computer-readable medium of claim 13, wherein:

the at least one variable includes a mean variable or a covariance variable.

15. The tangible computer-readable medium of claim 13, wherein:

the at least one shape feature, as applied to the hand gesture in the image, is selected from the group of eccentricity, compactness, orientation, rectangularity, width center, height center, the number of defects, difference between left and right parts, difference between top and bottom parts, and orientation.

16. The tangible computer-readable medium of claim 13 wherein the image is a red, green, blue (RGB) color image;

wherein the instructions that when executed by one or more of the processors result in the following additional operations comprising;

for each RGB image in a series of RGB images;

converting the RGB image into a hue, saturation, value (HSV) image;

extracting a color histogram of the hand region in the HSV image; and

tracking, using mean-shifting, the hand region in the series of HSV images.

17. The tangible computer-readable medium of claim 13, wherein the instructions that when executed by one or more of the processors result in the following additional operations comprising:

assigning at least one shape feature to an input node of a multilayer perceptron (MLP) model;

mapping the input node to an output neuron of the MLP model; and

determining a probability that the output neuron is one of a predefined hand gesture shape.

18. The tangible computer-readable medium of claim 13, wherein the instructions that when executed by one or more of the processors result in the following additional operations comprising:

generating a control signal indicative of the determined hand gesture.