Face searching and detection in a digital image acquisition device

• US 8,649,604 B2
• Filed: 07/23/2007
• Issued: 02/11/2014
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia, Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia, Inc.)
• Inventors: Steinberg, Eran, Bigioi, Petronel, Corcoran, Peter, Gangea, Mihnea, Costache, Gabriel, Drimbarean, Alexandru, Petrescu, Stefan Mirel, Vasiliu, Andrei
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US12/374,040
• Publication Number:
  US 20100272363A1
• Time in Patent Office: 2,395 Days
• Field of Search: 382/190, 382/228, 382/118
• US Class Current: 382/190
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/164 using holistic features
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method of detecting a face in an image includes performing face detection within a first window of the image at a first location. A confidence level is obtained from the face detection indicating a probability of the image including a face at or in the vicinity of the first location. Face detection is then performed within a second window at a second location, wherein the second location is determined based on the confidence level.

Claims

1. One or more non-transitory processor-readable media having code embedded therein for programming a processor to perform a method of detecting a face in an image, the method comprising:

performing face detection within a first relatively large window of said image at a first location;

obtaining from said face detection a confidence level that is below a threshold indicating a probability that no face is present at or in a vicinity of said first location;

performing second face detection within a second relatively large window of said image at a second location displaced from the first location by a first x-amount in a first direction and/or by a first y-amount in an orthogonal second direction;

obtaining from said second face detection a confidence level that is at or above said threshold indicating a probability that the image at least may include a face in the vicinity of the second location;

applying a sequence of windows, including relatively smaller windows than said second relatively large window, at further locations in the vicinity of said second location, including a third location that is displaced from the second location by a second x-amount that is smaller than the first x-amount in the first direction and/or by a second y-amount that is smaller than the first y-amount in the orthogonal second direction wherein a magnitude of displacement by respective values of x-amount in a first direction and/or by respective values of y-amount in an orthogonal second direction is variable between successive windows of said sequence of windows within said image and wherein said x-amount and/or said y-amount decreases between successive windows when a confidence level indicating that the image at least may include a face increases across one or more further thresholds,

wherein the method further comprising repeating the obtaining of confidence levels until face detection for multiple predetermined sizes of windows has been performed over an entire region of interest of said image.

2. One or more processor-readable media as claimed in claim 1, wherein the method further comprising repeating the obtaining of confidence levels below the threshold for multiple additional windows in different locations indicating probabilities in each case that no face is present at the different locations before said performing said second face detection.

3. One or more processor-readable media as claimed in claim 1 wherein the method further comprising repeating the obtaining of confidence levels for multiple additional windows in different locations until face detection has been performed over an entire region of interest.

4. One or more processor-readable media as claimed in claim 3, wherein dimensions of said one or more additional windows depend on said confidence level.

5. One or more processor-readable media as claimed in claim 1, wherein the method further comprising

determining a set of regions of interest for said image, and

repeating the obtaining of confidence levels for each of said regions of interest of said set.

6. One or more processor-readable media as claimed in claim 1, wherein the method further comprising identifying at least one region of said image likely to contain a face;

and repeating the obtaining of confidence levels until face detection is performed over substantially an entirety of said at least one region.

7. One or more processor-readable media as claimed in claim 1, wherein the method further comprising identifying at least one region of said image likely to contain a face;

and repeating the obtaining of confidence levels until face detection is performed concentrically inside said at least one region.

8. One or more processor-readable media as claimed in claim 1, wherein said face detection comprises applying a chain of classifiers to said windows of said image and wherein said confidence level comprises a function of a number of classifiers successfully applied to said windows.

9. One or more processor-readable media as claimed in claim 1, wherein the face detection window at the third location includes a portion that does not overlap the second window.

10. One or more processor-readable media as claimed in claim 9, wherein a size of the third window relative to a size of said second window is also decreased based on a confidence level of said third face detection increasing across at least one further threshold.

11. One or more processor-readable media as claimed in claim 1, wherein the magnitude of the second x-amount or the magnitude of the second y-amount, or both, are greater than the respective magnitudes of the first x-amount or the second y-amount, or both, for a higher confidence level that the image does not include a face at the first location compared with the second location.

12. One or more processor-readable media as claimed in claim 1, wherein the x-amount and the y-amount displacement magnitudes between at least two successive locations depend separately on face detection confidence levels in the first and second orthogonal directions, respectively.

13. One or more processor-readable media as claimed in claim 1, wherein when there is a confidence level above a threshold value that a face is detected in the vicinity of said first location, then the second location is selected such that the second window overlaps said first window to center on said face, and when there is a confidence level below said threshold, the second location is selected so that the second window does not overlap the first window.

14. One or more processor-readable media as claimed in claim 1, wherein the method further comprising pre-determining one or more regions of interest within the image each as having an enhanced likelihood of including a face, and locating a detection window at a suitable corner of each such region of interest, and applying to each such region of interest a sequence of windows, including applying at said suitable corner a face detection window of relatively larger size than a subsequent face detection window that is applied upon moving a first displacement magnitude amount to a subsequent location upon obtaining a confidence level above at least one of said further thresholds for face detection at a previous location, and displacing a smaller magnitude of displacement distance from the previous location to the subsequent location when said level of confidence that a face may exist at the previous location exceeds said at least one of said further thresholds.

15. One or more processor-readable media as claimed in claim 14, wherein said applying said sequence of windows to said region of interest comprises applying said sequence of windows to two or more regions of interest in time periods with at least some temporal overlap.

16. One or more processor-readable media as claimed in claim 14, wherein said one or more regions of interest comprise one or more regions including a number of skin pixels.

17. One or more processor-readable media as claimed in claim 14, wherein said image comprises is an image in a stream of images and wherein said regions of interest comprise one or more regions in which a face has been detected in a previous image of said stream.

18. One or more processor-readable media as claimed in claim 1, comprising:

responsive to said confidence level indicating a face at or in the vicinity of said first location, performing detection of a frontally aligned face within a second window at a second location wherein said second location is determined based on a confidence level indicating a probability of a candidate region including said face at or in the vicinity of said first location.

19. One or more processor-readable media as claimed in claim 18 wherein the method further comprising:

responsive to detection of a frontally aligned face in any candidate region, selectively applying face recognition to said candidate region.

20. One or more processor-readable media as claimed in claim 19, further comprising applying face recognition in response to any candidate region having a probability of including a face greater than a predetermined threshold.

21. One or more processor-readable media as claimed in claim 18 wherein said performing detection of a frontally aligned face is performed in a candidate region including said face detected at or in the vicinity of said first location.

22. One or more processor-readable media as claimed in claim 21 wherein performing detection of a frontally aligned face is performed in response to said candidate region having a probability of including a face greater than a first threshold.

23. One or more processor-readable media as claimed in claim 21 wherein the method further comprising:

responsive to said first location having a probability of including a face less than a first threshold, extracting an extended region including said first location from said image and performing face detection in said extended region with a relatively fine granularity to provide said candidate region.

24. One or more processor-readable media as claimed in claim 21 wherein the method further comprising:

responsive to a candidate region having a probability of including a face less than a second threshold, rotating an extended region including said face region through one of a sequence of angles and performing face detection in said rotated extended region with a relatively fine granularity to provide another candidate region.

25. One or more processor-readable media as claimed in claim 21 wherein the method further comprising:

responsive to a candidate region having a probability of including a face less than a second threshold, rotating said face within an extended region through one of a sequence of angles and performing face detection in said extended region with a relatively fine granularity to provide another candidate region.

26. A digital camera, comprising:

a processor; and

one or more processor readable media having code embedded therein for programming the processor to perform a method of detecting a face in an image in accordance with any of claims 1, 2-4, 5-8, 9-10, or 11-25.

27. A method of detecting a face in an image in accordance with any of claims 1, 2-4, 5-8, 9-10, or 11-25.