Device and method for recognizing hand shape and position, and recording medium having program for carrying out the method recorded thereon

• US 6,819,782 B1
• Filed: 06/08/2000
• Issued: 11/16/2004
• Est. Priority Date: 06/08/1999
• Status: Expired due to Term

• Current Assignee: National Institute of Information and Communications Technology, Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Communications Research Laboratory of The Ministry of Posts and Telecommunications Japan, Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Lu, Shan, Igi, Seiji, Matsuo, Hideaki, Imagawa, Kazuyuki
• Primary Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/588,948
• Time in Patent Office: 1,622 Days
• Field of Search: 382/103, 382/115, 382/116, 382/190, 382/195, 382/203, 382/209, 382/217, 382/218, 382/224, 382/225, 382/228, 345/157
• US Class Current: 382/115
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An object of the present invention is to provide a device and a method for recognizing hand shape and position even if a hand image to be provided for recognition is rather complicated in shape, and a recording medium having a program for carrying out the method recorded thereon.

A hand image normalization part 11 deletes a wrist region respectively from a plurality of images varied in hand shape and position before subjecting the images to normalization in hand orientation and size to generate hand shape images. An eigenspace calculation part 13 calculates an eigenvalue and an eigenvector respectively from the hand shape images under an analysis based on an eigenspace method. An eigenspace projection part 15 calculates eigenspace projection coordinates by projecting the hand shape images onto an eigenspace having the eigenvectors as a basis. A hand image normalization part 21 deletes a wrist region from an input hand image, and generates an input hand shape image by normalizing the input hand image to be equivalent to the hand shape images. An eigenspace projection part 22 calculates eigenspace projection coordinates for the input hand shape image by projecting the same onto the eigenspace having the eigenvectors as the basis. A hand shape image selection part 23 compares the eigenspace projection coordinates calculated for the input hand shape image with each of the eigenspace projection coordinates calculated for the hand shape images, and then determines which of the hand shape images is closest to the input hand shape image. A shape/position output part 24 outputs shape information and position information on the determined hand shape image.

Claims

1. A device for recognizing hand shape and position of an input hand image, said device comprising:

first hand image normalization means for receiving a plurality of hand images varied in hand shape and position, and after a wrist region is respectively deleted therefrom, subjecting the hand images to normalization in a predetermined manner to generate hand shape images;

hand shape image information storage means for storing so the hand shape images together with shape information and position information about each of the hand shape images;

eigenspace calculation means for calculating an eigenvalue and an eigenvector from each of the hand shape images based on an eigenspace method;

eigenvector storage means for storing the eigenvectors;

first eigenspace projection means for calculating eigenspace projection coordinates respectively for the hand shape images by projecting the hand shape images onto an eigenspace having the eigenvectors as a basis, and storing the eigenspace projection coordinates into said hand shape image information storage means;

second hand image normalization means for receiving the input hand image, and after a wrist region is deleted therefrom, normalizing the input hand image to generate an input hand shape image being equivalent to the hand shape images;

second eigenspace projection means for calculating eigenspace projection coordinates for the input hand shape image by projecting the input hand shape image onto the eigenspace having the eigenvectors as the basis;

hand shape image selection means for comparing the eigenspace projection coordinates calculated by said second eigenspace projection means with the eigenspace projection coordinates stored in said hand shape image information storage means, and determining which of the hand shape images is closest to the input hand shape image; and

shape/position output means for obtaining, for output, the shape information and the position information of the closest hand shape image from said hand shape image information storage means.

6. The device for recognizing hand shape and position as claimed in claim 1, wherein said first hand image normalization means and said second hand image normalization means respectively include:

color distribution storage means for previously storing a color distribution of the hand region to be extracted from the input hand image;

hand region extraction means for extracting the hand region from an input hand image according to the color distribution;

wrist region deletion means for finding which direction a wrist is oriented, and deleting a wrist region from the hand region according to the direction;

region displacement means for displacing the hand region from which the wrist region is deleted to a predetermined location on the image;

rotation angle calculation means for calculating a rotation angle in such a manner that the hand in the hand region is oriented to a predetermined direction;

region rotation means for rotating, according to the rotation angle, the hand region in such a manner that the hand therein is oriented to a direction; and

size normalization means for normalizing the rotated hand region to be in a predetermined size.

8. The device for recognizing hand shape and position as claimed in claim 1, further comprising:

instruction storage means for storing an instruction corresponding respectively to the shape information and the position information; and

instruction output means for receiving the shape information and the position information provided by said shape/position output means, and obtaining, for output, the instruction respectively corresponding to the shape information and the position information from said instruction storage means.

25. The device for recognizing hand shape and position as claimed in claim 1, wherein said first hand image normalization means is operable to subject the hand images to normalization in a predetermined manner based on hand orientation, image size or image contrast.

2. A device for recognizing hand shape and position of an input hand image, said device comprising:

first hand image normalization means for receiving a plurality of hand images varied in hand shape and position, and after a wrist region is respectively deleted therefrom, subjecting the hand images to normalization in a predetermined manner to generate hand shape images;

hand shape image information storage means for storing the hand shape images together with shape information and position information about each of the hand shape images;

eigenspace calculation means for calculating an eigenvalue and an eigenvector from each of the hand shape images under analysis based on an eigenspace method;

eigenvector storage means for storing the eigenvectors;

first eigenspace projection means for calculating eigenspace projection coordinates respectively for the hand shape images by projecting the hand shape images onto an eigenspace having the eigenvectors as a basis, and storing the eigenspace projection coordinates into said hand shape image information storage means;

cluster evaluation means for classifying, into clusters, the eigenspace projection coordinates under cluster evaluation, determining which of the hand shape images belongs to which cluster for storage into said hand shape image information storage means, and obtaining statistical information about each cluster;

cluster information storage means for storing each of the statistical information together with the cluster corresponding thereto;

second hand image normalization means for receiving the input hand image, and after a wrist region is deleted therefrom, normalizing the input hand image to generate an input hand shape image being equivalent to the hand shape images;

second eigenspace projection means for calculating eigenspace projection coordinates for the input hand shape image by projecting the input hand shape image onto the eigenspace having the eigenvectors as the basis;

maximum likelihood cluster judgment means for comparing the eigenspace projection coordinates calculated by said second eigenspace projection means with each of coordinates included in the statistical information stored in said cluster information storage means, and determining which cluster is the closest;

image comparison means for comparing the hand shape images included in the closest cluster with the input hand shape image, and determining which of the hand shape images is most closely analogous to the input hand shape image; and

shape/position output means for obtaining, for output, the shape information and the position information of the most analogous hand shape image from said hand shape image information storage means.

3. The device for recognizing hand shape and position as claimed in claim 2, wherein said image comparison means includes:

identical shape classification means for classifying, according to hand shape, the hand shape images included in the cluster determined by said maximum likelihood cluster judgment means into groups before comparing the hand shape images with the input hand shape image generated by said second hand image normalization means;

shape group statistic calculation means for calculating a statistic representing the groups; and

maximum likelihood shape judgment means for calculating a distance between the input hand shape image and the statistic, and outputting a hand shape included in the closest group.

4. The device for recognizing hand shape and position as claimed in claim 2, wherein said cluster evaluation means obtains the hand shape images and the shape information for each cluster from said hand shape image information storage means, calculates a partial region respectively of the hand shape images for discrimination, and stores the partial regions into said cluster information storage means, and

wherein said image comparison means compares the hand shape images in the cluster determined by said maximum likelihood cluster judgment means with the input hand shape image generated by said second hand image normalization means only in the partial region corresponding to the cluster.

5. The device for recognizing hand shape and position as claimed in claim 2, wherein, when the input hand image is plurally provided by photographing a hand from several directions,

said second hand image normalization means generates the input hand shape image for each of the input hand images,

said second eigenspace projection means calculates the eigenspace projection coordinates in the eigenspace respectively for the input hand shape images generated by said second hand image normalization means,

said maximum likelihood cluster judgment means compares each of the eigenspace projection coordinates calculated by said second eigenspace projection means with the statistical information, and determines which cluster is the closest, and

said image comparison means merges the closest clusters determined by said maximum likelihood cluster judgment means, and estimates hand shape and position consistent with the shape information and the position information about the hand shape images in each of the clusters.

7. The device for recognizing hand shape and position as claimed in claim 2, wherein said first hand image normalization means and said second hand image normalization means respectively include:

color distribution storage means for previously storing a color distribution of the hand region to be extracted from the input hand image;

hand region extraction means for extracting the hand region from an input hand image according to the color distribution;

wrist region deletion means for finding which direction a wrist is oriented, and deleting a wrist region from the hand region according to the direction;

region displacement means for displacing the hand region from which the wrist region is deleted to a predetermined location on the image;

rotation angle calculation means for calculating a rotation angle in such a manner that the hand in the hand region is oriented to a predetermined direction;

region rotation means for rotating, according to the rotation angle, the hand region in such a manner that the hand therein is oriented to a direction; and

size normalization means for normalizing the rotated hand region to be in a predetermined size.

9. A method for recognizing hand shape and position of an input hand image, said method comprising:

receiving a plurality of hand images varied in hand shape and position, and after a wrist region is respectively deleted therefrom, subjecting the hand images to normalization in a predetermined manner to generate hand shape images;

calculating an eigenvalue and an eigenvector from each of the hand shape images based on an eigenspace method;

calculating eigenspace projection coordinates respectively for the hand shape images by projecting the hand shape images onto an eigenspace having the eigenvectors as a basis;

receiving the input hand image, and after a wrist region is deleted therefrom, normalizing the input hand image to generate an input hand shape image being equivalent to the hand shape images;

calculating eigenspace projection coordinates for the input hand shape image by projecting the input hand shape image onto the eigenspace having the eigenvectors as the basis;

comparing the eigenspace projection coordinates calculated for the hand shape images with the eigenspace projection coordinates calculated for the input hand shape image, and determining which of the hand shape images is closest to the input hand shape image; and

outputting the shape information and the position information of the closest hand shape image.

26. The method for recording hand shape and position as claimed in claim 9, wherein said subjecting the hand images to normalization in a predetermined manner comprises subjecting the hand images to normalization in a predetermined manner based on hand orientation, image size or image contrast.

10. A method for recognizing hand shape and position of an input hand image, said method comprising:

receiving a plurality of hand images varied in hand shape and position, and after a wrist region is respectively deleted therefrom, subjecting the hand images to normalization in a predetermined manner to generate hand shape images;

calculating an eigenvalue and an eigenvector from each of the hand shape images based on an eigenspace method;

calculating eigenspace projection coordinates respectively for the hand shape images by projecting the hand shape images onto an eigenspace having the eigenvectors as a basis;

classifying, under cluster evaluation, the eigenspace projection coordinates into clusters, determining which of the hand shape images belongs to which cluster, and obtaining statistical information about each of the clusters;

receiving the input hand image, and after a wrist region is deleted therefrom, normalizing the input hand image to generate an input hand shape image being equivalent to the hand shape images;

calculating eigenspace projection coordinates for the input hand shape image by projecting the input hand shape image onto the eigenspace having the eigenvectors as the basis;

comparing the eigenspace projection coordinates calculated for the input hand shape image with each of the statistical information, and determining the closest cluster;

comparing each of said hand shape images included in the closest cluster with the input hand shape image, and determining which of the hand shape images is most analogous to the input hand shape image, and

outputting the shape information and the position information of the most analogous hand shape image.

11. The method for recognizing hand shape and position as claimed in claim 10, wherein said comparing each of the hand shape images and determining includes:

classifying, into clusters, the hand shape images included in the cluster determined in said comparing the eigenspace projection coordinates and determining before comparing the hand shape images with the input hand shape image generated in said receiving and normalizing of the input hand image;

calculating a statistic representing the clusters; and

calculating a distance between the input hand shape image and the statistic, and outputting a hand shape included in the closest cluster.

12. The method for recognizing hand shape and position as claimed in claim 10,

wherein, in said classifying and obtaining, according to the hand shape images and the shape information, a partial region is calculated respectively for the hand shape images for discrimination, and

wherein in said comparing each of the hand shape images and determining, the hand shape images in the cluster determined in said comparing the eigenspace projection coordinates and determining are compared with the input hand shape image generated in said receiving of the input hand image only in the partial region corresponding to the cluster.

13. The method for recognizing hand shape and position as claimed in claim 10,

wherein, when the input hand image is plurally provided by photographing a hand from several directions,

in said, receiving and normalizing of the input hand image, the input hand shape image is generated for each of the input hand images,

in said calculating of the eigenspace projection coordinates, eigenspace projection coordinates in the eigenspace is calculated respectively for the input hand shape images generated in said receiving and normalizing of the input hand image,

in said comparing the eigenspace projection coordinates and determining, each of the eigenspace projection coordinates calculated in said calculating eigenspace projection coordinates is compared with the statistical information, and the closest cluster is determined, and

in said comparing each of the hand shape images and determining, the closest clusters determined in said comparing the eigenspace projection coordinates and determining are merged, and hand shape and position consistent with the shape information and the position information of the hand shape images in each of the clusters is estimated.

14. The method for recognizing hand shape and position as claimed in claim 10, wherein said receiving a plurality of hand images and subjecting the hand images to normalization and said receiving the input hand images and normalizing the input hand image respectively include:

previously storing a color distribution of the hand region to be extracted from the input hand image;

extracting the hand region from an input hand image according to the color distribution;

finding which direction a wrist is oriented, and deleting a wrist region from the hand region according to the direction;

displacing said the hand region from which the wrist region is deleted to a predetermined location on the image;

calculating a rotation angle in such a manner that the hand in the hand region is oriented to a predetermined direction;

rotating, according to the rotation angle, the hand region in such a manner that the hand therein is oriented to a direction; and

normalizing the rotated hand region to be in a predetermined size.

15. The method for recognizing hand shape and position as claimed in claim 11, wherein said receiving a plurality of hand images and subjecting the hand images to normalization and said receiving the input hand images and normalizing the input hand image respectively include:

previously storing a color distribution of the hand region to be extracted from the input hand image;

extracting the hand region from an input hand image according to the color distribution;

finding which direction a wrist is oriented, and deleting a wrist region from the hand region according to the direction;

displacing the hand region from which the wrist region is deleted to a predetermined location on the image;

calculating a rotation angle in such a manner that the hand in the hand region is oriented to a predetermined direction;

rotating, according to the rotation angle, the hand region in such a manner that the hand therein is oriented to a direction; and

normalizing the rotated hand region to be in a predetermined size.

16. The method for recognizing hand shape and position as claimed in claim 14, further comprising:

storing an instruction corresponding respectively to the shape information and the position information; and

receiving the shape information and the position information outputted in said outputting, and obtaining, for output, the instruction respectively corresponding to the shape information and the position information stored in said storing.

27. The method for recognizing hand shape and position as claimed in claim 10, wherein said subjecting the hand images to normalization in a predetermined manner comprises subjecting the hand images to normalization in a predetermined manner based on hand orientation, image size or image contrast.

17. A recording medium having stored thereon, a computer device executable program for carrying out a method for recognizing hand shape and position of an input hand image the program being for realizing an operational environment on the computer device including:

receiving a plurality of hand images varied in hand shape and position, and after a wrist region is respectively deleted therefrom, subjecting the hand images to normalization in a predetermined manner to generate hand shape images;

calculating an eigenvalue and an eigenvector from each of the hand shape images under analysis based on an eigenspace method;

calculating eigenspace projection coordinates respectively for the hand shape images by projecting the hand shape images onto an eigenspace having the eigenvectors as a basis;

receiving the input hand image, and after a wrist region is deleted therefrom, normalizing the input hand image to generate an input hand shape image being equivalent to the hand shape images;

calculating eigenspace projection coordinates for the input hand shape image by projecting the input hand shape image onto the eigenspace having the eigenvectors as the basis;

comparing the eigenspace projection coordinates calculated for the hand shape images with the eigenspace projection coordinates calculated for the input hand shape image, and determining which of the hand shape images is closest to the input hand shape image; and

outputting the shape information and the position information of the closest hand shape image.

24. The recording medium claimed in claim 17, further comprising:

storing an instruction corresponding respectively to the shape information and the position information; and

receiving the shape information and the position information outputted in said outputting, and obtaining, for output, the instruction respectively corresponding to the shape information and the position information stored in said storing.

28. The recording medium claimed in claim 17, wherein said subjecting the hand images to normalization in a predetermined manner comprises subjecting the hand images to normalization in a predetermined manner based on hand orientation, image size or image contrast.

18. A recording medium having stored thereon, a computer device executable for carrying out a method for recognizing hand shape and position of an input hand image, the program being for realizing an operational environment on the computer device including:

receiving a plurality of hand images varied in hand shape and position, and after a wrist region is respectively deleted therefrom, subjecting the hand images to normalization in a predetermined manner to generate hand shape images;

calculating an eigenvalue and an eigenvector from each of the hand shape images under analysis based on an eigenspace method;

calculating eigenspace projection coordinates respectively for the hand shape images by projecting the hand shape images onto an eigenspace having the eigenvectors as a basis;

classifying, into clusters, the eigenspace projection coordinates under cluster evaluation, determining which of the hand shape images belongs to which cluster, and obtaining statistical information about each cluster;

receiving the input hand image, and after a wrist region is deleted therefrom, normalizing the input hand image to generate an input hand shape image being equivalent to the hand shape images;

calculating eigenspace projection coordinates for the input hand shape image by projecting the input hand shape image onto the eigenspace having the eigenvectors as the basis;

comparing the eigenspace projection coordinates calculated for the input hand shape image with each of coordinates included in the statistical information, and determining which cluster is the closest;

comparing the hand shape images included in the closest cluster with the input hand shape image, and determining which of the hand shape images is most closely analogous to the input hand shape image; and

outputting the shape information and the position information of the most analogous hand shape image.

20. The recording medium as claimed in claim 18,

wherein, in said classifying and obtaining, according to the hand shape images and the shape information, a partial region is calculated respectively for the hand shape images for discrimination, and

wherein in said comparing each of the hand shape images and determining, the hand shape images in the cluster determined in said comparing the eigenspace projection coordinates and determining are compared with the input hand shape image generated in said receiving of the input hand image only in the partial region corresponding to the cluster.

21. The recording medium as claimed in claim 18,

wherein, when the input hand image is plurally provided by photographing a hand from several directions,

in said, receiving and normalizing of the input hand image, the input hand shape image is generated for each of the input hand images,

in said calculating of the eigenspace projection coordinates, eigenspace projection coordinates in the eigenspace is calculated respectively for the input hand shape images generated in said receiving and normalizing of the input hand image,

in said comparing the eigenspace projection coordinates and determining, each of the eigenspace projection coordinates calculated in said calculating eigenspace projection coordinates is compared with the statistical information, and the closest cluster is determined, and

in said comparing each of the hand shape images and determining, the closest clusters determined in said comparing the eigenspace projection coordinates and determining are merged, and hand shape and position consistent with the shape information and the position information of the hand shape images in each of the clusters is estimated.

22. The recording medium as claimed in claim 18, wherein said receiving a plurality of hand images and subjecting the hand images to normalization and said receiving the input hand images and normalizing the input hand image respectively include:

previously storing a color distribution of the hand region to be extracted from the input hand image;

extracting the hand region from an input hand image according to the color distribution;

finding which direction a wrist is oriented, and deleting a wrist region from the hand region according to the direction;

displacing the hand region from which the wrist region is deleted to a predetermined location on the image;

calculating a rotation angle in such a manner that the hand in the hand region is oriented to a predetermined direction;

rotating, according to the rotation angle, the hand region in such a manner that the hand therein is oriented to a direction; and

normalizing the rotated hand region to be in a predetermined size.

19. The recording medium as claimed 18, wherein said comparing each of the hand shape images and determining includes:

classifying, into clusters, the hand shape images included in the cluster determined in said comparing the eigenspace projection coordinates and determining before comparing the hand shape images with the input hand shape image generated in said receiving and normalizing of the input hand image;

calculating a statistic representing the clusters; and

calculating a distance between the input hand shape image and the statistic, and outputting a hand shape included in the closest cluster.

23. The recording medium as claimed in claim 19, wherein said first normalization step and said second normalization step respectively include:

a color storage step of previously storing a color distribution of said hand region to be extracted from the input hand image;

a step of extracting said hand region from an input hand image according to said color distribution;

a step of finding which direction a wrist is oriented, and deleting a wrist region from said hand region according to the direction;

a step of displacing said hand region from which said wrist region is deleted to a predetermined location on the image;

a step of calculating a rotation angle in such a manner that the hand in said hand region is oriented to a predetermined direction;

a step of rotating, according to said rotation angle, said hand region in such a manner that the hand therein is oriented to a direction; and

a step of normalizing said rotated hand region to be in a predetermined size.