Method for locating faces in digital color images

• US 7,110,575 B2
• Filed: 08/02/2002
• Issued: 09/19/2006
• Est. Priority Date: 08/02/2002
• Status: Expired due to Fees

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Chen, Shoupu, Ray, Lawrence A.
• Primary Examiner(s): Wu, Jingge
• Assistant Examiner(s): KUHN, JORDAN M
• Application Number:
  US10/211,011
• Publication Number:
  US 20040022423A1
• Time in Patent Office: 1,509 Days
• Field of Search: 382/118, 382/164
• US Class Current: 382/118
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

A digital image processing method for locating faces in a digital color image includes the steps of: generating a mean grid pattern element (MGPe) image from a plurality of sample face images; generating an integral image from the digital color image; and locating faces in the color digital image by using the integral image to perform a correlation between the mean grid pattern element (MGPe) image and the digital color image at a plurality of effective resolutions by reducing the digital color image to grid pattern element images (GPes) at different effective resolutions and correlating the MGPe with the GPes.

Claims

1. A digital image processing method for locating faces in a digital color image, comprising the steps of:

generating an integral image from the digital color image;

generating a grid pattern image from said integral image, said grid pattern image having a grid pattern having a plurality of cells each of said cells having a plurality of pixels;

reducing said grid pattern image to a corresponding grid pattern element image, said grid pattern element image having a number of pixels equal to the number of cells in said grid pattern image; and

performing a correlation test between said grid pattern element image and a predetermined mean grid pattern element image of sample face images.

2. The method claimed in claim 1 wherein said generating an integral image step further comprises:

cropping a sub-image from said digital color image; and

computing said integral image from said sub-image.

3. The method claimed in claim 2 wherein said grid pattern has a first cell size and the method further comprises iterating said generating of said grid pattern image, said reducing, and said performing steps with each of a plurality of additional grid patterns, said grid patterns all having different cell sizes.

4. The method claimed in claim 3 wherein said digital color image has a plurality of different areas, said sub-image is of one of said areas;

and the method further comprises repeating said generating, generating, reducing, performing, and iterating steps on a plurality of different additional sub-images of said digital color image.

5. The method claimed in claim 1, wherein the step of generating the integral image further comprises the steps of:

replacing non-skin-color pixels with black to produce an image C having skin color pixels;

replacing non face shaped clusters with black to produce an image E having skin colored face shaped clusters;

labeling clusters of skin colored face shaped clusters; and

generating the integral image from each cluster of the image E.

6. The method claimed in claim 5, wherein the step of replacing non face shaped clusters comprises the steps of:

clustering skin-color pixels in image C into clusters;

applying morphological opening and closing processes to skin-colored pixel clusters; and

replacing the pixels of a cluster with black if it does not meet a geometrical criterion for a face, resulting in an image E; and

wherein the step of labeling skin colored clusters comprises the step of generating a linked list of sets of parameters including a starting position, width, and height, that defines regions containing a cluster of skin-colored pixels.

7. The method as claimed in claim 5, wherein the step of selecting an irregular grid pattern comprises determining a plurality of different size grid cells that cover major features including eyes, nose, mouth, forehead, and cheek of the mean face image.

8. The method claimed in claim 1 further comprising prior to said generating steps:

generating a mean face image from a collection of sample face images;

generating a mean integral image from the mean face image;

generating a mean grid pattern image from said mean integral image, said mean grid pattern image having a plurality of cells in said grid pattern; and

reducing said mean grid pattern image to a corresponding mean grid pattern element image, said mean grid pattern element image having a number of pixels equal to the number of cells in said mean grid pattern image.

9. The method claimed in claim 8 wherein said grid pattern element images and said mean grid pattern element image are one-dimensional.

10. The method claimed in claim 8, wherein the grid pattern is regular.

11. The method, claimed in claim 10, wherein the step of selecting a regular grid pattern comprises:

computing a distance, e1, between two eye centers of the mean face image;

computing a center position, c, between two eye centers; and

using e1 and c to determine a region (position and size) that contains M by N grid cells with each cell having m by n pixels.

12. The method claimed in claim 8, wherein the grid pattern is irregular.

13. The method claimed in claim 8, wherein the step of generating the mean integral image further comprises the steps of:

replacing non-skin-color pixels with black to produce an image C having skin color pixels;

replacing non face shaped clusters with black to produce an image E having skin colored face shaped clusters;

labeling clusters of skin colored face shaped clusters; and

generating the mean integral image from each cluster of the image E.

14. The method claimed in claim 13, further comprising the steps of:

eliminating faces that contain more than a predetermined percentage of black pixels; and

merging faces that substantially overlap.

15. The method claimed in claim 13, wherein the step of replacing non face shaped clusters comprises the steps of:

clustering skin-color pixels in image C into clusters;

applying morphological opening and closing processes to skin-colored pixel clusters; and

replacing the pixels of a cluster with black if it does not meet a geometrical criterion for a face, resulting in an image E; and

wherein the step of labeling skin colored clusters comprises the step of generating a linked list of sets of parameters including a starting position, width, and height, that defines regions containing a cluster of skin-colored pixels.

16. The method claimed in claim 1, wherein the grid pattern is regular.

17. The method claimed in claim 1, wherein the grid pattern is irregular.

18. A computer program product for performing the method of claim 1, the computer program product comprising computer readable storage medium having a computer program stored thereon.

19. A digital image processing method for locating faces in a digital color image, comprising the steps of:

a) generating a mean arid pattern element (MGPe) image from a plurality of sample face images;

b) generating an integral image from the digital color image; and

c) locating faces in the digital color image by using the integral image to perform a correlation test between the mean grid pattern element (MGPe) image and the digital color image at a plurality of effective resolutions by reducing the digital color image to a plurality of arid pattern element images (GPes) at different effective resolutions and correlating the MGPe with the GPes;

wherein the step of generating a mean arid pattern element (MGPe) image, comprises the steps of;

a1) collecting sample face images;

a2) generating a mean face image from the sample face images;

a3) selecting a arid pattern (GP); and

a4) reducing the resolution of the mean face image to the resolution of the selected arid pattern (GP) by averaging;

wherein the arid pattern is regular; and

wherein the step of selecting a regular grid pattern comprises computing a distance, e1, between two eye centers of the mean face image;

computing a center position, c, between two eye centers;

using e1 and c to determine M and N dimensions and a position of a region wherein said region contains M by N grid cells with each cell having m by n pixels.

20. A digital image processing method for locating faces in a digital color image, comprising the steps of:

a) generating a mean grid pattern element (MGPe) image from a plurality of sample face images;

b) generating an integral image from the digital color image; and

c) locating faces in the digital color image by using the integral image to perform a correlation test between the mean grid pattern element (MGPe) image and the digital color image at a plurality of effective resolutions by reducing the digital color image to a plurality of grid pattern element images (GPes) at different effective resolutions and correlating the MGPe with the GPes;

wherein the step of generating an integral image further comprises the steps of;

b1) replacing non-skin-color pixels with black to produce an image C having skin color pixels;

b2) replacing non face shaped clusters with black to produce an image E having skin colored face shaped clusters;

b3) labeling clusters of skin colored face shaped clusters; and

b4) generating the integral image from each cluster of the image E.

21. The method claimed in claim 20, further comprising the steps of:

d) eliminating faces that contain more than a predetermined percentage of black pixels; and

e) merging faces that substantially overlap.

22. The method claimed in claim 20, wherein the step of replacing non face shaped clusters comprises the steps of:

b2.1) clustering skin-color pixels in image C into clusters;

b2.2) applying morphological opening and closing processes to skin-colored pixel clusters; and

b2.3) replacing the pixels of a cluster with black if it does not meet a geometrical criterion for a face, resulting in an image E; and

wherein the step of labeling skin colored clusters comprises the step of generating a linked list of sets of parameters including a starting position, width, and height, that defines regions containing a cluster of skin-colored pixels.

23. A digital image processing system for locating faces in a digital color image comprising:

means for generating an integral image from the digital color image;

means for generating a grid pattern image from said integral image, said grid pattern image having a grid pattern having a plurality of cells;

means for reducing said grid pattern image to a corresponding grid pattern element image, said grid pattern element image having a number of pixels equal to the number of cells in said grid pattern image; and

means for performing a correlation test between said grid pattern element image and a predetermined mean grid pattern element image of sample face images.