Method and system of person identification by facial image

• US 8,064,653 B2
• Filed: 11/29/2007
• Issued: 11/22/2011
• Est. Priority Date: 11/29/2007
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.), Viewdle, Inc.
• Original Assignee: Viewdle, Inc.
• Inventors: Kyiko, Kostiantyn, Tsariov, Andriy, Schlezinger, Mykhailo, Nedashkivskyi, Oleksii, Musatenko, Yuriy, Matsello, Vyacheslav, Kyyko, Volodymyr
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US11/947,588
• Publication Number:
  US 20090141947A1
• Time in Patent Office: 1,454 Days
• Field of Search: 382/106, 382/115, 382117-118, 382/209, 382/254, 382/276, 382/250, 351/208, 351/209
• US Class Current: 382/118
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method to recognize a facial image is described. An input facial image is normalized by scaling and rotation angle using methods of eye pupil centers detection. The input facial image is further normalized by lighting intensity. Template images are obtained either by the processing of certain images taken from different face positions, or by a preliminary reconstruction of a 3D face model based on stereo-pair images. Using the 3D model template, facial images are generated at different rotation angles. Distances between the input facial image and the template image are calculated from the Discrete Cosine Transformation (DCT) features, defined by overlapped blocks of these images. The facial image is recognized based on these distances.

Claims

1. A computer-based method for identification of a person by a facial image, the method comprising:

localizing the facial image;

detecting coordinates of eye pupil centers in the facial image;

responsive to detecting the coordinates of eye pupil centers, deriving a normalized facial image based on the facial image;

splitting the normalized facial image into rectangular blocks;

describing the rectangular blocks of the normalized facial image by a DCT (discrete cosine transform);

calculating distances between the normalized facial image and existing templates, the distances being weighted sums of shortest distances between the rectangular blocks of the normalized facial image and rectangular blocks of the existing templates, wherein one or more most different blocks are filtered from the weighted sum; and

recognizing the normalized facial image based on the distances calculated.

2. The method of claim 1, wherein deriving the normalized facial image based on the facial image comprises scaling the facial image relative to a distance between the eye pupil centers.

3. The method of claim 1, wherein deriving the normalized facial image based on the facial image comprises rotating the facial image relative to the coordinates of the eye pupil centers to orient the facial image in a horizontal orientation.

4. The method of claim 1, wherein deriving the normalized facial image based on the facial image comprises adjusting lighting conditions of the facial image.

5. The method of claim 4, wherein adjusting the lighting conditions of the facial image comprises one or both of the following:

a histogram equalization method and a local contrast calculation method.

6. The method of claim 1, wherein describing the rectangular blocks by a DCT comprises, for each rectangular block:

calculating a table of DCT coefficients;

extracting the most informative of the DCT coefficients via a zigzag scan of the DCT coefficients of the block starting from a coefficient located at the top-left corner of the table of the DCT coefficients; and

replacing one or more of the DCT coefficients by weighted differences of corresponding coefficients located in neighboring horizontal and vertical blocks; and

obtaining resultant feature values for the block.

7. The method of claim 1, wherein calculating the distances between the normalized facial image and the existing templates comprises determining a sum of absolute values of the distances between corresponding features of the normalized facial image and an existing template divided by the sum of the absolute values of the corresponding features.

8. The method of claim 1, wherein calculating the distances between the normalized facial image and the existing templates comprises, for each template:

determining a first distance between a block of the normalized facial image and a first block of an existing template, the first distance corresponding to a block of the normalized facial image and a block of the existing template having the same coordinates; and

calculating one or more additional distances between the block of the normalized facial image and blocks neighboring the first block of the existing template.

9. The method of claim 1, wherein recognizing the normalized facial image based on the distances calculated results in a rejection of one or more existing templates when the distance between the facial image and the one or more existing templates is above a predetermined threshold.

10. The method of claim 1, wherein recognizing the normalized facial image based on the distances calculated comprises:

a first stage wherein the distances calculated between the rectangular blocks of the normalized facial image and the rectangular blocks of the existing templates are calculated for non-overlapping rectangular blocks and compared to a first predetermined threshold, the first predetermined threshold indicating a degree of similarity between the normalized facial image and an existing template; and

responsive to a distance corresponding to an existing template calculated in the first stage being greater than the first predetermined threshold and less than a second predetermined threshold, using a second stage wherein the distances calculated between the rectangular blocks of the normalized facial image and the rectangular blocks of the existing template are calculated for overlapping rectangular blocks, the second predetermined threshold set near the first and indicating that the normalized facial image and an existing template are close to being similar.

11. The method of claim 1, wherein calculating the distances between the normalized facial image and the existing templates is based on the existing templates obtained by one or more of the following:

processing of images of the facial image in different orientations; and

reconstruction of a 3D face model based on a stereo-pair image, the 3D face model being utilized to generate facial images at different rotation angles.

12. The method of claim 1, wherein recognizing the normalized facial image is performed when there are similarities between the normalized facial image and one or more existing templates associated with one person and no similarities between the normalized facial image and existing templates associated with any other person, the similarities being based on the distances between the facial image and the existing templates below a predetermined threshold.

13. The method of claim 1, wherein detecting the coordinates of the eye pupil centers in a video is performed based on the “

AdaBoost”

method.

14. The method of claim 1, wherein detecting the coordinates of the eye pupil centers comprises:

determining a preliminary selection of a predetermined set of pixels likely to be located at the centers of the eye pupils;

determining a location certainty to a predetermined confidence level for each pixel; and

determining an approximate radius of the eye iris.

15. The method of claim 14, wherein determining the location certainty to the predetermined confidence level for each pixel is based on the predetermined confidence level being a sum of confidences and degree of compatibility between two pixels in a pair.

16. The method of claim 14, wherein determining the preliminary selection of the predetermined set of pixels likely to be located at the centers of the eye pupils comprises analysis of pixel lines of the facial image and detecting middle points of segments of the pixel lines, the preliminary selection made based on one or more of the following criteria:

a) segments are symmetrical by brightness relative to the middle point,

b) a light intensity level is below a first predetermined threshold,

c) brightness gradient values at the ends of the segments exceed a second predetermined threshold, and

d) the lengths of the segments belong to a predetermined interval of likely diameters of the eye iris.

17. A non-transitory machine-readable storage medium storing executable computer program instructions for identification of a person by a facial image, the instructions performing steps comprising:

localizing the facial image;

detecting coordinates of eye pupil centers in the facial image;

responsive to detecting the coordinates of the eye pupil centers, deriving a normalized facial image based on the facial image;

splitting the normalized facial image into rectangular blocks;

describing the rectangular blocks of the normalized facial image by a DCT (discrete cosine transform);

calculating distances between the normalized facial image and existing templates, the distances being weighted sums of shortest distances between the rectangular blocks of the normalized facial image and rectangular blocks of the existing templates, wherein one or more most different blocks are filtered from the weighted sum; and

recognizing the normalized facial image based on the distances calculated.

18. The storage medium of claim 17, wherein deriving the normalized facial image based on the facial image comprises scaling the facial image relative to a distance between the eye pupil centers.

19. The storage medium of claim 17, wherein deriving the normalized facial image based on the facial image comprises rotating the facial image relative to the coordinates of the eye pupil centers to orient the facial image in a horizontal orientation.

20. The storage medium of claim 17, wherein deriving the normalized facial image based on the facial image comprises adjusting lighting conditions of the facial image.

21. The storage medium of claim 17, wherein describing the rectangular blocks by a DCT comprises, for each rectangular block:

calculating a table of DCT coefficients;

extracting the most informative of the DCT coefficients via a zigzag scan of the DCT coefficients of the block starting from a coefficient located at the top-left corner of the table of the DCT coefficients; and

replacing one or more of the DCT coefficients by weighted differences of corresponding coefficients located in neighboring horizontal and vertical blocks; and

obtaining resultant feature values for the block.

22. The storage medium of claim 17, wherein recognizing the normalized facial image based on the distances calculated comprises:

a first stage wherein the distances calculated between the rectangular blocks of the normalized facial image and the rectangular blocks of the existing templates are calculated for non-overlapping rectangular blocks and compared to a first predetermined threshold, the first predetermined threshold indicating a degree of similarity between the normalized facial image and an existing template; and

responsive to a distance corresponding to an existing template calculated in the first stage being greater than the first predetermined threshold and less than a second predetermined threshold, using a second stage wherein the distances calculated between the rectangular blocks of the normalized facial image and the rectangular blocks of the existing template are calculated for overlapping rectangular blocks, the second predetermined threshold set near the first and indicating that the normalized facial image and an existing template are close to being similar.

27. The computer-based system of claim 18, wherein describing the rectangular blocks by a DCT comprises, for each rectangular block:

calculating a table of DCT coefficients;

extracting the most informative of the DCT coefficients via a zigzag scan of the DCT coefficients of the block starting from a coefficient located at the top-left corner of the table of the DCT coefficients; and

replacing one or more of the DCT coefficients by weighted differences of corresponding coefficients located in neighboring horizontal and vertical blocks; and

obtaining resultant feature values for the block.

23. A computer-based system for identification of a person by a facial image, the system comprising:

means for localizing the facial image;

means for detecting coordinates of eye pupil centers in the facial image;

means for deriving a normalized facial image based on the facial image responsive to determining the coordinates of eye pupil centers;

means for splitting the normalized facial image into rectangular blocks;

means for describing the rectangular blocks of the normalized facial image by a DCT (discrete cosine transform);

means for calculating distances between the normalized facial image and existing templates, the distances being weighted sums of shortest distances between the rectangular blocks of the normalized facial image and rectangular blocks of the existing templates, wherein one or more most different blocks are filtered from the weighted sum; and

means for recognizing the normalized facial image based on the distances calculated.

24. The computer-based system of claim 23, wherein deriving the normalized facial image based on the facial image comprises scaling the facial image relative to a distance between the eye pupil centers.

25. The computer-based system of claim 23, wherein deriving the normalized facial image based on the facial image comprises rotating the facial image relative to the coordinates of the eye pupil centers to orient the facial image in a horizontal orientation.

26. The computer-based system of claim 23, wherein deriving the normalized facial image based on the facial image comprises adjusting lighting conditions of the facial image.

28. The computer-based system of claim 23, wherein recognizing the normalized facial image based on the distances calculated comprises:

a first stage wherein the distances calculated between the rectangular blocks of the normalized facial image and the rectangular blocks of the existing templates are calculated for non-overlapping rectangular blocks and compared to a first predetermined threshold, the first predetermined threshold indicating a degree of similarity between the normalized facial image and an existing template; and

responsive to a distance corresponding to an existing template calculated in the first stage being greater than the first predetermined threshold and less than a second predetermined threshold, using a second stage wherein the distances calculated between the rectangular blocks of the normalized facial image and the rectangular blocks of the existing template are calculated for overlapping rectangular blocks, the second predetermined threshold set near the first and indicating that the normalized facial image and an existing template are close to being similar.