Hand gesture recognition system and method

• US 6,128,003 A
• Filed: 12/22/1997
• Issued: 10/03/2000
• Est. Priority Date: 12/20/1996
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Smith, Anthony Vernon Walker, Mcgrath, Sean, Lemoine, Arnaud, Sutherland, Alistair Ian
• Primary Examiner(s): Cuchlinski, Jr., William A.
• Assistant Examiner(s): Marc-Coleman, Marthe Y.
• Application Number:
  US08/996,147
• Time in Patent Office: 1,016 Days
• Field of Search: 345/158, 345/157, 345/156, 345/358, 345/326, 348/77, 340/825.19
• US Class Current: 345/157
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Noise problems in processing small images or large-granularity images are reduced by representing hand images as rotational vectors calculated using a real-valued centroid. The hand region is therefore sectored independently of pixel quantization. Color segmentation is used to identify hand-color regions, followed by region labelling to filter out noise regions based on region size. Principal component analysis is used to plot gesture models.

Claims

1. A hand gesture recognition system comprising:

input means for receiving a stream of images in real time; and

processing means for representing a hand which is represented in each image as a vector and for processing the vectors to recognize hand gestures,

wherein said processing means comprises;

vector processing means for representing the hand in each image as a rotational vector which is calculated using a real-value centroid of a region representing the hand in which the region is divided into a plurality of sectors through the real-value centroid by calculating the ratio of the number of pixels within each sector representing the color of the hand to the total number of pixels of the color of the hand within the region as an element vector for each section, wherein the element vectors correspond to the rotational vector, and whereby the hand is sectored independently of pixel grid quantization, and

recognition means for analyzing sequences of rotational vectors to recognize hand gestures by performing a principal component analysis in which the rotational vectors are principal components which are plotted against reference gesture models.

2. A hand gesture recognition system as claimed in claim 1, wherein said recognition means comprises a principal component analysis section for performing said principal component analysis in which rotational vectors are principal components which are plotted against reference gesture models.

3. A hand gesture recognition system as claimed in claim 2, wherein said reference gesture models are arranged for searching in a hierarchical structure according to their similarities to each other.

4. A hand gesture recognition system as claimed in claim 1, wherein said processing means further comprises:

color segmentation means for identifying segments in each image having the hand color.

5. A hand gesture recognition system as claimed in claim 4, wherein said color segmentation means identifies segments in each image having the hand color using a three-dimensional color cone reference template.

6. A hand gesture recognition system as claimed in claim 5, wherein said color segmentation means filters pixels initially according to two-dimensional color plot comparisons so as to identify pixels which are in the three-dimensional color cone.

7. A hand gesture recognition system as claimed in claim 5, wherein said color segmentation means comprises:

means for indicating when the image environment is too dark or too bright if an excessive proportion of pixels within the three-dimensional color cone have a value below a lower darkness threshold or above an upper brightness threshold.

8. A hand gesture recognition system as claimed in claim 4 wherein said processing means further comprises:

filter means for filtering noise regions before vector processing.

9. A hand gesture recognition system as claimed in claim 8, wherein said filter means comprises:

means for labeling regions having the hand color and passing to vector processing only the largest such region.

10. A hand gesture recognition system as claimed in claim 1, wherein said vector processing means comprises:

means for computing a real-value centroid of the hand initially and subsequently computing the rotational vector as a function of integer-based vectors computed using origins on pixels adjacent the real-value centroid.

11. A hand gesture recognition system as claimed in claim 10, wherein the function of integer-based vectors is computation of integer-based vectors initially using each or origins on pixels adjacent to the real-value centroid, subsequently summation of the integer-based vectors and then normalization of sum of the integer-based vectors.

12. A hand gesture recognition system as claimed in claim 11, wherein the integer-based vectors are weighted before summation according to the distances of their origins from the real-value centroid.

13. A hand gesture recognition system as claimed in claim 10, wherein the integer-based vectors have origins on four pixels surrounding the real-value centroid.

14. A hand gesture recognition system as claimed in claim 1, wherein said vector processing means comprises:

means for computing the rotational vectors by calculating hand size extending from the real-value centroid directly in each of radial segments, said radial segments being obtained by dividing the image in the real-value centroid as an origin of radial coordinates.

15. A hand gesture recognition system as claimed in claim 14, wherein pixels on sector boundaries are allocated to sectors according to their sub-division between sectors.

16. A hand gesture recognition system as claimed in claim 15, wherein said vector processing means comprises:

means for pre-defining an array of pixels which fall between upper and lower sector boundary limits.

17. A hand gesture recognition method comprising the steps of:

receiving a stream of images in real time;

representing a hand in each image as a rotational vector which is calculated using a real-value centroid of a region representing the hand in which the region is divided into a plurality of sectors through the real-value centroid by calculating the ratio of the number of pixels within each sector representing the color of the hand to the total number of pixels of the color of the hand within the region as an element vector for each section, wherein the element vectors correspond to the rotational vector, and, whereby the hand is sectored independently of pixel grid quantization, and

analyzing sequences of rotational vectors to recognize hand gestures by performing a principal component analysis in which the rotational vectors are principal components which are plotted against reference gesture models.

18. A hand gesture recognition method as claimed in claim 17, wherein said reference gesture models are arranged for searching in a hierarchical structure according to their similarities to each other.