Method for face orientation determination in digital color images

• US 20040151371A1
• Filed: 01/30/2003
• Published: 08/05/2004
• Est. Priority Date: 01/30/2003
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Chen, Shoupu, Ray, Lawrence A.
• Granted Patent: US 7,120,279 B2
• Time in Patent Office: Days
• US Class Current: 382/165
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

A digital image processing method for determining an orientation of a face in a digital color image generates a mean grid pattern element image from a plurality of sample face images and an integral image from the digital color image. A face is located in the color digital image by using the integral image to perform a correlation test between the mean grid pattern element image and the digital color image at a plurality of effective resolutions by reducing the digital color image to a plurality of grid pattern element images at different effective resolutions and correlating the mean grid pattern element image with the plurality of grid pattern element images, whereby either the mean grid pattern element image or the grid pattern element images are provided at a plurality of different orientations. Accordingly, an orientation of the face in the color digital image is determined by using the images with different orientations in the correlation test.

Claims

1. A digital image processing method for determining orientation of faces located in a digital color image, comprising the steps of:

a) generating a mean grid pattern element image from a plurality of sample face images;

b) generating an integral image from the digital color image;

c) locating a face in the color digital image by using the integral image to perform a correlation test between the mean grid pattern element image and the digital color image at a plurality of effective resolutions by reducing the digital color image to a plurality of grid pattern element images at different effective resolutions and correlating the mean grid pattern element image with the plurality of grid pattern element images, whereby either the mean grid pattern element image or the grid pattern element images are provided at a plurality of different orientations; and

d) determining an orientation of the face in the color digital image by using the images with different orientations in the correlation test of step c).

2. The method claimed in claim 1, wherein the step a) of generating a mean grid pattern element image comprises the steps of:

a1) collecting sample face images;

a2) generating a mean face image from the sample face images;

a3) selecting a grid pattern;

a4) reducing the resolution of the mean face image to the resolution of the selected grid pattern by averaging; and

a5) generating mean grid pattern element images with different orientations by rearranging the order of the elements.

3. The method claimed in claim 2, wherein the step a3) comprises selecting a regular grid pattern.

4. The method claimed in claim 2, wherein the step a3) comprises selecting an irregular grid pattern.

5. The method as claimed in claim 4, wherein the step of selecting an irregular grid pattern comprises determining a plurality of different size grid cells that cover major features including at least one of eyes, nose, mouth, forehead, and cheek of the mean face image.

6. The method as claimed in claim 3, wherein the step of selecting a regular grid pattern comprises computing a distance between two eye centers of the mean face image;

computing a center position between the two eye centers; and

using the distance and position to determine a region for containing a predetermined M by N grid cell arrangement with each cell having m by n pixels.

7. The method claimed in claim 1, wherein the step b1) of generating an integral image further comprises the steps of:

b1) replacing non-skin color pixels in the digital color image with black to produce an image having skin color pixels;

b2) replacing non-face shaped clusters of pixels with black to produce an image having skin colored and face shaped clusters;

b3) labeling clusters of skin colored and face shaped clusters as face clusters; and

b4) generating the integral image from each labeled face cluster of the image.

8. The method claimed in claim 7, further comprising the steps of:

eliminating face clusters that contain more than a predetermined percentage of black pixels; and

merging face clusters that substantially overlap.

9. The method claimed in claim 7, wherein the step of replacing non face shaped clusters comprises the steps of:

clustering skin-color pixels in the image into clusters;

applying morphological opening and closing processes to the skin- colored pixel clusters; and

replacing the pixels of a cluster with black if they do not meet a geometrical criterion for a face, thereby resulting in a processed image; and

wherein the step of labeling skin colored clusters comprises the step of generating a linked list of sets of parameters including a starting position, width, and height, that defines regions containing a cluster of skin-colored pixels.

10. The method as claimed in claim 1, wherein the step of determining the image orientation based on face orientations comprises the steps of:

determining the image orientation by applying an order statistics analysis to rank ordered orientation indices associated with the face-like pattern objects; and

determining the image orientation by voting using the a few top orientation indices.

11. A computer program product for performing the method of claim 1.

12. A digital image processing method for locating faces in a digital color image, comprising the steps of:

a) generating a mean grid pattern element image from a plurality of sample face images by obtaining a mean face image from the plurality of sample face images, dividing the mean face image into a grid pattern comprising a plurality of cells, and forming the mean grid pattern element image from a mean value of each cell;

b) generating a grid pattern element image from a region of the digital color image that possibly contains a face; and

c) locating a face in the color digital image by performing a correlation test between the mean grid pattern element image and the grid pattern element image at a plurality of different image orientations, whereby either the mean grid pattern element image or the grid pattern element image is provided at the plurality of different orientations.

13. The method as claimed in claim 12 wherein an orientation indication is obtained for the face by using either the mean grid pattern element image or the grid pattern element images with different orientations in the correlation test of step c).

14. The method as claimed in claim 12 wherein step b) comprises generating an integral image from the digital color image;

and step c) comprises locating a face in the color digital image by using the integral image to perform a correlation test between the mean grid pattern element image and the digital color image at a plurality of effective resolutions by reducing the digital color image to the grid pattern element images at different effective resolutions and correlating the mean grid pattern element image with the grid pattern element images.

15. A computer program product for performing the method of claim 12.

16. A digital image processing method for locating a face in a digital color image, comprising the steps of:

a) generating a mean grid pattern element image from a plurality of sample face images by obtaining a mean face image from the plurality of sample face images, dividing the mean face image into a grid pattern comprising a plurality of cells, and forming the mean grid pattern element image from a mean value of each cell;

b) generating an integral image from the digital color image; and

c) locating a face in the color digital image by using the integral image to perform a correlation test between the mean grid pattern element image and the digital color image at a plurality of effective resolutions by reducing the digital color image to a plurality of grid pattern element images at different effective resolutions and correlating the mean grid pattern element image with the plurality of grid pattern element images, whereby either the mean grid pattern element image or the grid pattern element images are provided at a plurality of different orientations.

17. The method as claimed in claim 16 wherein an orientation indication is obtained for the face by using either the mean grid pattern element image or the grid pattern element images with different orientations in the correlation test of step c).

18. A computer program product for performing the method of claim 16.