METHOD AND SYSTEM OF PERSON IDENTIFICATION BY FACIAL IMAGE

• US 20090141947A1
• Filed: 11/29/2007
• Published: 06/04/2009
• Est. Priority Date: 11/29/2007
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.), Viewdle, Inc.
• Original Assignee: Viewdle, Inc.
• Inventors: Musatenko, Yuriy, Kyiko, Kostiantyn, Matsello, Vyacheslav, Tsariov, Andriy, Schlezinger, Mykhailo, Nedashkivskyi, Oleksii, Kyyko, Volodymyr
• Granted Patent: US 8,064,653 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method to recognize a facial image is described. An input facial image is normalized by scaling and rotation angle using methods of eye pupil centers detection. The input facial image is further normalized by lighting intensity. Template images are obtained either by the processing of certain images taken from different face positions, or by a preliminary reconstruction of a 3D face model based on stereo-pair images. Using the 3D model template, facial images are generated at different rotation angles. Distances between the input facial image and the template image are calculated from the Discrete Cosine Transformation (DCT) features, defined by overlapped blocks of these images. The facial image is recognized based on these distances.

Claims

1. A computer-based method of a person identification by a facial image comprising:

localizing the facial image;

detecting coordinates of eye pupil centers on the facial image;

deriving a normalized facial image based on the facial image;

splitting the normalized facial image into rectangular blocks;

describing the rectangular blocks of the normalized facial image by a DCT (discrete cosine transform);

calculating distances between the normalized facial image and existing templates performed in two stages, the distances being weighted sums of shortest distances between each rectangular block of the normalized facial image and a set of rectangular blocks of the existing templates, wherein a small part of most different blocks are excluded from the weighted sum; and

recognizing the normalized facial image based on the distances calculated.

2. The method of claim 1, where deriving the normalized facial image based on the facial image is performed by scaling the facial image relative to a distance between the eye pupil centers.

3. The method of claim 1, where deriving the normalized facial image based on the facial image is performed by rotating the facial image relative to the coordinates of the eye pupil centers to be in a horizontal orientation.

4. The method of claim 1, where deriving the normalized facial image based on the facial image is performed by adjusting lighting intensity of the facial image.

5. The method of claim 1, wherein the describing of the rectangular blocks comprises, for each block:

calculating a table of DCT coefficients;

extracting the most informative of the DCT coefficients via a zigzag scan of the DCT coefficients of the rectangular block starting from a coefficient of the rectangular block located at the top-left corner; and

replacing each of the first three DCT coefficients by two weighted differences of corresponding coefficients located in neighboring horizontal and vertical rectangular blocks; and

obtaining resultant 18 feature values for each block that has vertical and horizontal neighbors.

6. The method of claim 1, wherein calculating the distances between the normalized facial image and the existing templates comprises determining a sum of absolute values of the distances between corresponding features of the normalized facial image and the existing template divided by the sum of the absolute values of the corresponding features.

7. The method of claim 1, wherein calculating the distances between the normalized facial image and the existing templates utilizes a subset of template rectangular blocks consisting of an image block that has the same coordinates as the image block and 8 neighboring template blocks.

8. The method of claim 1, wherein recognizing the normalized facial image based on the distances calculated results in a rejection of the existing templates when the distance between the facial image and the existing templates obtained in a first stage of two stages is below a predetermined threshold.

9. The method of claim 5, wherein recognizing the normalized facial image is based on the distances calculated during the first stage with the rectangular blocks not overlapping whereas during a second stage each rectangular block overlaps its horizontally and vertically neighboring blocks by a predetermined area.

10. The method of claim 1, wherein calculating the distances between the normalized facial image and the existing templates is based on the existing templates obtained by one or more of the following:

processing of images of the facial image in different orientations; and

reconstruction of a 3D face model based on a stereo-pair image, the 3D face model being utilized to generate facial images at different rotation angles.

11. The method of claim 1, wherein recognizing the normalized facial image is performed when there are similarities between the facial image of one or more of the existing templates associated with one person and no similarities between the existing templates associated with any other person, the similarities being based on the distance between the facial image and the existing templates below a predetermined threshold.

12. The method of claim 1, wherein detecting the coordinates of the eye pupil centers in a video is performed based on the “

AdaBoost”

method.

13. The method of claim 1, wherein the detecting of the coordinates of the eye pupil centers is performed by a preliminary selection of a predetermined set of pixels likely to be located at the centers of the eye pupils, determining location certainty to a predetermined confidence level for each pixel and determining an approximate radius of the eye iris.

14. The method of claim 10, wherein the determining of the location certainty to the predetermined confidence level for each pixel is based on the predetermined confidence level being a sum of confidences and degree of compatibility between two pixels in a pair.

15. The method of claim 10, wherein the detecting is based on a preliminary selection being performed by analysis of pixel lines of the facial image and detecting middle points of segments of the pixel lines, the preliminary selection made based on one or more of the following criteria:

a) segments are symmetrical by brightness relative to the middle point,

b) a light intensity level is below a first predetermined threshold,

c) brightness gradient values at the ends of the segments exceed a second predetermined threshold, and

d) the lengths of the segments belong to a predetermined interval of likely diameters of the eye iris.

16. The method of claim 1, wherein the normalizing of the facial image by the lighting intensity is performed by one or both of the following:

a histogram equalization method and a local contrast calculation method.

17. A machine-readable medium comprising instructions, which when implemented by one or more processors perform the following operations:

localize the facial image;

detect coordinates of eye pupil centers on the facial image;

derive a normalized facial image based on the facial image;

split the normalized facial image into rectangular blocks;

describe the rectangular blocks of the normalized facial image by a DCT (discrete cosine transform);

calculate distances between the normalized facial image and existing templates performed in two stages, the distances being weighted sums of shortest distances between each rectangular block of the normalized facial image and a set of rectangular blocks of the existing templates, wherein a small part of most different blocks are excluded from the weighted sum; and

recognize the normalized facial image based on the distances calculated.

18. A computer-based system of a person identification by a facial image comprising:

means for localizing the facial image;

means for detecting coordinates of eye pupil centers on the facial image;

means for deriving a normalized facial image based on the facial image;

means for splitting the normalized facial image into rectangular blocks;

means for describing the rectangular blocks of the normalized facial image by a DCT (discrete cosine transform);

means for calculating distances between the normalized facial image and existing templates performed in two stages, the distances being weighted sums of shortest distances between each rectangular block of the normalized facial image and a set of rectangular blocks of the existing templates, wherein a small part of most different blocks are excluded from the weighted sum; and

means for recognizing the normalized facial image based on the distances calculated.