Methods and apparatuses for half-face detection

• US 8,605,955 B2
• Filed: 06/28/2010
• Issued: 12/10/2013
• Est. Priority Date: 06/29/2009
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Petrescu, Stefan, Gangea, Mihnea
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US12/825,280
• Publication Number:
  US 20110158547A1
• Time in Patent Office: 1,261 Days
• Field of Search: 382/118, 382/224
• US Class Current: 382/118
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

Classifier chains are used to determine quickly and accurately if a window or sub-window of an image contains a right face, a left face, a full face, or does not contain a face. After acquiring a digital image, an integral image is calculated based on the acquired digital image. Left-face classifiers are applied to the integral image to determine the probability that the window contains a left face. Right-face classifiers are applied to the integral image to determine the probability that the window contains a right face. If the probability of the window containing a right face and a left face are both greater than threshold values, then it is determined that the window contains a full face. Alternatively, if only one of the probabilities exceeds a threshold value, then it may be determined that the window contains only a left face or a right face.

Claims

1. A method comprising steps of:

determining a left-face probability based at least in part on an application of one or more left-face classifiers to an image;

determining a right-face probability based at least in part on an application of one or more right-face classifiers to the image;

selecting a particular characterization of the image, from a plurality of supported characterizations of the image, based at least in part on both the left-face probability and the right-face probability, wherein the plurality of supported characterizations include at least;

(a) the image contains a left face without a visible right face, and

(b) the image contains a right face without a visible left face; and

wherein the steps are performed by an automated device.

2. The method of claim 1, further comprising:

determining that a result of an application of the left-face classifiers and the right-face classifiers to the image contradicts a result of an application of a full-face classifier to the image; and

in response to determining that the result of the application of the left-face classifiers and the right-face classifiers to the image contradicts the result of the application of the full-face classifier to the image, performing additional analysis relative to the image to determine whether the image contains a left face only, a right face only, a full face, or no face.

3. The method of claim 1, further comprising:

calculating at least one of the right-face classifiers based on at least one of the left-face classifiers.

4. The method of claim 1, further comprising:

identifying, within the image, a region that contains at least a specified proportion of skin-toned pixels;

wherein the application of the one or more left-face classifiers to the image consists of an application of the one or more left-face classifiers only to said region; and

wherein the application of the one or more right-face classifiers to the image consists of an application of the one or more right-face classifiers only to said region.

5. The method of claim 4, wherein said image is a first image, and further comprising:

after identifying, within the first image, the region that contains the at least a specified proportion of skin-toned pixels, adjusting parameters of a face-detection algorithm based on luminance characteristics of pixels within the region and applying said face-detection algorithm to a second image that differs from said first image.

6. The method of claim 1, further comprising:

incrementally adjusting a camera'"'"'s exposure over time until application of the left-face classifiers and right-face classifiers results in a detection of at least a portion of a face within the image.

7. The method of claim 1, wherein determining the left-face probability based at least in part on the application of the one or more left-face classifiers to an image comprises:

applying, to the image, a first left-face classifier that is designed to detect a presence of a first feature in the image; and

only if the first left-face classifier returns a result that indicates that the first left-face classifier detected the presence of the first feature in the image, applying, to the image, a second left-face classifier that is designed to detect a presence, in the image, of a second feature that differs from the first feature;

wherein the application of the first left-face classifier is less computationally expensive than the application of the second left-face classifier.

8. The method of claim 1 wherein the plurality of supported characterizations further include at least:

(c) the image contains a full face, and

(d) the image does not contain a face.

9. The method of claim 8, further comprising determining that the image contains a full face by:

determining that the image prospectively contains a full face due to the left-face probability exceeding a first specified threshold and the right-face probability exceeding a second specified threshold; and

verifying that the image contains a full face by receiving a positive result from an application of a full-face classifier to the image.

10. The method of claim 8, further comprising determining that the image contains a left face without a right face by:

determining that the image prospectively contains a left face without a right face due to the left-face probability exceeding a first specified threshold and the right-face probability failing to exceed a second specified threshold; and

verifying that the image does not contain a full face by receiving a negative result from an application of a full-face classifier to the image.

11. The method of claim 8, further comprising determining that the image contains no face by:

determining that the image prospectively contains no face due to the left-face probability failing to exceed a first specified threshold and the right-face probability failing to exceed a second specified threshold; and

verifying that the image does not contain a full face by receiving a negative result from an application of a full-face classifier to the image.

12. A volatile or non-volatile computer-readable storage medium storing instructions which, when executed by one or more processors, cause the processors to perform steps comprising:

determining a left-face probability based at least in part on an application of one or more left-face classifiers to an image;

determining a right-face probability based at least in part on an application of one or more right-face classifiers to the image;

selecting a particular characterization of the image, from a plurality of supported characterizations of the image, based at least in part on both the left-face probability and the right-face probability, wherein the plurality of supported characterizations include at least;

(a) the image contains a left face without a visible right face, and

(b) the image contains a right face without a visible left face.

13. The computer-readable storage medium of claim 12, wherein the steps further comprise:

determining that a result of an application of the left-face classifiers and the right-face classifiers to the image contradicts a result of an application of a full-face classifier to the image; and

in response to determining that the result of the application of the left-face classifiers and the right-face classifiers to the image contradicts the result of the application of the full-face classifier to the image, performing additional analysis relative to the image to determine whether the image contains a left face only, a right face only, a full face, or no face.

14. The computer-readable storage medium of claim 12, wherein the steps further comprise:

calculating at least one of the right-face classifiers based on at least one of the left-face classifiers.

15. The computer-readable storage medium of claim 12, wherein the steps further comprise:

identifying, within the image, a region that contains at least a specified proportion of skin-toned pixels;

wherein the application of the one or more left-face classifiers to the image consists of an application of the one or more left-face classifiers only to said region; and

wherein the application of the one or more right-face classifiers to the image consists of an application of the one or more right-face classifiers only to said region.

16. The computer-readable storage medium of claim 15, wherein said image is a first image, and wherein the steps further comprise:

after identifying, within the first image, the region that contains the at least a specified proportion of skin-toned pixels, adjusting parameters of a face-detection algorithm based on luminance characteristics of pixels within the region and applying said face-detection algorithm to a second image that differs from said first image.

17. The computer-readable storage medium of claim 12, wherein the steps further comprise:

incrementally adjusting a camera'"'"'s exposure over time until application of the left-face classifiers and right-face classifiers results in a detection of at least a portion of a face within the image.

18. The computer-readable storage medium of claim 12, wherein determining the left-face probability based at least in part on the application of the one or more left-face classifiers to an image comprises:

applying, to the image, a first left-face classifier that is designed to detect a presence of a first feature in the image; and

only if the first left-face classifier returns a result that indicates that the first left-face classifier detected the presence of the first feature in the image, applying, to the image, a second left-face classifier that is designed to detect a presence, in the image, of a second feature that differs from the first feature;

wherein the application of the first left-face classifier is less computationally expensive than the application of the second left-face classifier.

19. The computer-readable storage medium of claim 12, wherein the plurality of supported characterizations further include at least:

(c) the image contains a full face, and

(d) the image does not contain a face.

20. The computer-readable storage medium of claim 19, further comprising determining that the image contains a full face by:

determining that the image prospectively contains a full face due to the left-face probability exceeding a first specified threshold and the right-face probability exceeding a second specified threshold; and

verifying that the image contains a full face by receiving a positive result from an application of a full-face classifier to the image.

21. The computer-readable storage medium of claim 19, further comprising determining that the image contains a left face without a right face by:

determining that the image prospectively contains a left face without a right face due to the left-face probability exceeding a first specified threshold and the right-face probability failing to exceed a second specified threshold; and

verifying that the image does not contain a full face by receiving a negative result from an application of a full-face classifier to the image.

22. The computer-readable storage medium of claim 19, further comprising determining that the image contains no face by:

determining that the image prospectively contains no face due to the left-face probability failing to exceed a first specified threshold and the right-face probability failing to exceed a second specified threshold; and

verifying that the image does not contain a full face by receiving a negative result from an application of a full-face classifier to the image.

23. A digital camera comprising:

an image capturing and storing module;

a left-face classifier module that determines a left-face probability based at least in part on an application of one or more left-face classifiers to an image;

a right-face classifier module that determines a right-face probability based at least in part on an application of one or more right-face classifiers to the image; and

an image labeling module that selects a particular characterization of the image, from a plurality of supported characterizations of the image, based at least in part on both the left-face probability and the right-face probability, wherein the plurality of supported characterizations include at least;

(a) the image contains a left face without a visible right face, and

(b) the image contains a right face without a visible left face.

24. The digital camera of claim 23, further comprising:

an analytical modules that (a) determines that a result of an application of the left-face classifiers and the right-face classifiers to the image contradicts a result of an application of a full-face classifier to the image and (b) in response to determining that the result of the application of the left-face classifiers and the right-face classifiers to the image contradicts the result of the application of the full-face classifier to the image, performs additional analysis relative to the image to determine whether the image contains a left face only, a right face only, a full face, or no face.

25. The digital camera of claim 23, further comprising:

a classifier-generating module that calculates at least one of the right-face classifiers based on at least one of the left-face classifiers.

26. The digital camera of claim 23, further comprising:

a tone-identifying module that identifies, within the image, a region that contains at least a specified proportion of skin-toned pixels;

wherein the application of the one or more left-face classifiers to the image consists of an application of the one or more left-face classifiers only to said region; and

wherein the application of the one or more right-face classifiers to the image consists of an application of the one or more right-face classifiers only to said region.

27. The digital camera of claim 26, wherein said image is a first image, and further comprising:

an algorithm-adjustment module which, after identifying, within the first image, the region that contains the at least a specified proportion of skin-toned pixels, adjusts parameters of a face-detection algorithm based on luminance characteristics of pixels within the region and an application of said face-detection algorithm to a second image that differs from said first image.

28. The digital camera of claim 23, further comprising:

an exposure-adjustment module that incrementally adjusts a camera'"'"'s exposure over time until application of the left-face classifiers and right-face classifiers results in a detection of at least a portion of a face within the image.

29. The digital camera of claim 23, wherein determining the left-face probability based at least in part on the application of the one or more left-face classifiers to an image comprises:

applying, to the image, a first left-face classifier that is designed to detect a presence of a first feature in the image; and

only if the first left-face classifier returns a result that indicates that the first left-face classifier detected the presence of the first feature in the image, applying, to the image, a second left-face classifier that is designed to detect a presence, in the image, of a second feature that differs from the first feature;

wherein the application of the first left-face classifier is less computationally expensive than the application of the second left-face classifier.

30. The digital camera of claim 23, wherein the plurality of supported characterizations further include at least:

(c) the image contains a full face, and

(d) the image does not contain a face.

31. The digital camera of claim 30, further comprising determining that the image contains a full face by:

determining that the image prospectively contains a full face due to the left-face probability exceeding a first specified threshold and the right-face probability exceeding a second specified threshold; and

verifying that the image contains a full face by receiving a positive result from an application of a full-face classifier to the image.

32. The digital camera of claim 30, further comprising determining that the image contains a left face by:

determining that the image prospectively contains a left face without a right face due to the left-face probability exceeding a first specified threshold and the right-face probability failing to exceed a second specified threshold; and

verifying that the image does not contain a full face by receiving a negative result from an application of a full-face classifier to the image.

33. The digital camera of claim 30, further comprising determining that the image contains a no face by:

determining that the image prospectively contains no face due to the left-face probability failing to exceed a first specified threshold and the right-face probability failing to exceed a second specified threshold; and

verifying that the image does not contain a full face by receiving a negative result from an application of a full-face classifier to the image.

34. A method comprising steps of:

determining a left-face probability based at least in part on an application of a particular left-face classifier of a plurality of left-face classifiers to an image,

wherein each left-face classifier of the plurality of left-face classifiers includes a left-face classifier threshold,

wherein the particular left-face classifier includes a first left-face classifier threshold that is different than a second left-face classifier threshold;

determining whether the left-face probability is equal to or greater than the first left-face classifier threshold;

determining a right-face probability based at least in part on an application of a particular right-face classifier of a plurality of right-face classifiers to the image,

wherein each right-face classifier of the plurality of right-face classifiers includes a right-face classifier threshold,

wherein the particular right-face classifier includes a first right-face classifier threshold that is different than a second right-face classifier threshold;

determining whether the right-face probability is equal to or greater than the first right-face classifier threshold;

based at least in part whether the left-face probability is equal to or greater than the first left-face classifier threshold and whether the right-face probability is equal to or greater than the first right-face classifier threshold, determining which one of the following is true relative to the image;

(a) the image contains a full face,

(b) the image contains a left face without a right face,

(c) the image contains a right face without a left face, or

(d) the image does not contain a face;

wherein the steps are performed by an automated device.