Face searching and detection in a digital image acquisition device

• US 9,224,034 B2
• Filed: 12/22/2014
• Issued: 12/29/2015
• Est. Priority Date: 03/05/2007
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia, Inc.)
• Original Assignee: FotoNation Limited (Adeia, Inc.)
• Inventors: Steinberg, Eran, Bigioi, Petronel, Corcoran, Peter, Gangea, Mihnea, Petrescu, Stefan Mirel, Vasiliu, Andrei, Costache, Gabriel, Drimbarean, Alexandru
• Primary Examiner(s): Li, Ruiping
• Application Number:
  US14/579,773
• Publication Number:
  US 20150169941A1
• Time in Patent Office: 372 Days
• Field of Search: 382/103, 382/190, 382/228, 382/118
• US Class Current: 1/1
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/164 using holistic features
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method of detecting a face in an image includes performing face detection within a first window of the image at a first location. A confidence level is obtained from the face detection indicating a probability of the image including a face at or in the vicinity of the first location. Face detection is then performed within a second window at a second location, wherein the second location is determined based on the confidence level.

Claims

1. A device, comprising:

an image acquiring unit communicatively coupled to a memory unit, and configured to acquire data of a digital image that depicts one or more objects; and

a face detection unit configured to;

perform face detection within a first relatively large window of said image at a first location;

obtain from said face detection a confidence level that is below a threshold indicating a probability that no face is present at or in a vicinity of said first location;

perform second face detection within a second relatively large window of said image at a second location displaced from the first location by a first x-amount in a first direction and/or by a first y-amount in an orthogonal second direction;

obtain from said second face detection a confidence level that is at or above said threshold indicating a probability that the image at least may include a face in the vicinity of the second location;

apply a sequence of windows, including relatively smaller windows than said second relatively large window, at further locations in the vicinity of said second location, including a third location that is displaced from the second location by a second x-amount that is smaller than the first x-amount in the first direction and/or by a second y-amount that is smaller than the first y-amount in the orthogonal second direction wherein a magnitude of displacement by respective values of x-amount in a first direction and/or by respective values of y-amount in an orthogonal second direction is variable between successive windows of said sequence of windows within said image and wherein said x-amount and/or said y-amount decreases between successive windows when a confidence level indicating that the image at least may include a face increases across one or more further thresholds; and

repeat the obtaining of confidence levels until face detection for multiple predetermined sizes of windows has been performed over an entire region of interest of said image.

2. The device of claim 1, wherein the face detection unit is further configured to:

repeat the obtaining of confidence levels below the threshold for multiple additional windows in different locations indicating probabilities in each case that no face is present at the different locations before said performing said second face detection.

3. The device of claim 1, wherein the face detection unit is further configured to:

repeat the obtaining of confidence levels for multiple additional windows in different locations until face detection has been performed over an entire region of interest.

4. The device of claim 3, wherein dimensions of said one or more additional windows depend on said confidence level.

5. The device of claim 1, wherein the face detection unit is further configured to:

determine a set of regions of interest for said image, and

repeat the obtaining of confidence levels for each of all said regions of interest of said set.

6. The device of claim 1, wherein the face detection unit is further configured to:

identify at least one region of said image likely to contain a face; and

repeat the obtaining of confidence levels until face detection is performed over substantially an entirety of said at least one region.

7. The device of claim 1, wherein the face detection unit is further configured to:

identify at least one region of said image likely to contain a face; and

repeat the obtaining of confidence levels until face detection is performed concentrically inside said at least one region.

8. The device of claim 1, wherein the face detection unit is further configured to:

apply a chain of classifiers to said windows of said image and wherein said confidence level comprises a function of a number of classifiers successfully applied to said windows.

9. The device of claim 1, wherein the first relatively large window of said image at a first location includes a portion that does not overlap the second window.

10. The device of claim 1, wherein a size of the first window is decreased based on a confidence level of said face detection.

11. The device of claim 1, wherein the magnitude of the second x-amount or the magnitude of the second y-amount, or both, are greater than the respective magnitudes of the first x-amount or the second y-amount, or both, for a higher confidence level that the image does not include a face at the first location compared with the second location.

12. The device of claim 1, wherein an x-amount displacement magnitude and an y-amount displacement magnitude between at least two successive locations depend separately on face detection confidence levels in a first orthogonal direction and a second orthogonal direction, respectively.

13. The device of claim 1, wherein the confidence level is above a threshold value that a face is detected in the vicinity of said first location.

14. The device of claim 1, wherein the face detection unit is further configured to:

pre-determine one or more regions of interest within the image each as having an enhanced likelihood of including a face,

locate a detection window at a suitable corner of each such region of interest,

apply to each such region of interest a sequence of windows, by applying at said suitable corner a face detection window of relatively larger size than a subsequent face detection window that is applied upon moving a first displacement magnitude amount to a subsequent location upon obtaining a confidence level above at least one of said further thresholds for face detection at the previous location, and

displace a smaller magnitude of displacement distance from the previous location to the subsequent location when a level of confidence that a face may exist at the previous location exceeds said at least one of said further thresholds.

15. The device of claim 1, wherein the face detection unit is further configured to:

apply said sequence of windows to two or more regions of interest in time periods with at least some temporal overlap.

16. The device of claim 1, wherein said one or more regions of interest comprise one or more regions including a number of skin pixels.

17. The device of claim 1, wherein said image comprises is-an image in a stream of images and wherein said regions of interest comprise one or more regions in which a face has been detected in a previous image of said stream.

18. The device of claim 1, wherein the face detection unit is further configured to:

responsive to said confidence level indicating a face at or in the vicinity of said first location, perform detection of a frontally aligned face within the first relatively large window.

19. The device of claim 1, wherein the face detection unit is further configured to:

responsive to detection of a frontally aligned face in any candidate region, selectively apply face recognition to said candidate region.

20. The device of claim 1, wherein the face detection unit is further configured to:

apply face recognition in response to any candidate region having a probability of including a face greater than a predetermined threshold.

21. The device of claim 20, wherein said performing detection is performed in a candidate region including said face detected at or in the vicinity of said first location.

22. The device of claim 1, wherein said performing detection is performed in response to a candidate region having a probability of including a face greater than a first threshold.

23. The device of claim 1, wherein the face detection unit is further configured to:

responsive to said first location having a probability of including a face less than a first threshold, extract an extended region including said first location from said image and perform face detection in said extended region with a relatively fine granularity to provide a candidate region.

24. The device of claim 1, wherein the face detection unit is further configured to:

responsive to a candidate region having a probability of including a face less than a second threshold, rotate an extended region including a face region through one of a sequence of angles and perform face detection in said rotated extended region with a relatively fine granularity to provide another candidate region.

25. The device of claim 1, wherein the face detection unit is further configured to:

responsive to a candidate region having a probability of including a face less than a second threshold, rotate said face within an extended region through one of a sequence of angles and perform face detection in said extended region with a relatively fine granularity to provide another candidate region.

26. A method for improving face searching and detection in a digital image acquisition device, the method comprising:

acquiring, using an image acquiring unit communicatively coupled to a memory unit, data of a digital image that depicts one or more objects;

using a face detection unit performing;

performing face detection within a first relatively large window of said image at a first location;

obtaining from said face detection a confidence level that is below a threshold indicating a probability that no face is present at or in a vicinity of said first location;

performing second face detection within a second relatively large window of said image at a second location displaced from the first location by a first x-amount in a first direction and/or by a first y-amount in an orthogonal second direction;

obtaining from said second face detection a confidence level that is at or above said threshold indicating a probability that the image at least may include a face in the vicinity of the second location;

applying a sequence of windows, including relatively smaller windows than said second relatively large window, at further locations in the vicinity of said second location, including a third location that is displaced from the second location by a second x-amount that is smaller than the first x-amount in the first direction and/or by a second y-amount that is smaller than the first y-amount in the orthogonal second direction wherein a magnitude of displacement by respective values of x-amount in a first direction and/or by respective values of y-amount in an orthogonal second direction is variable between successive windows of said sequence of windows within said image and wherein said x-amount and/or said y-amount decreases between successive windows when a confidence level indicating that the image at least may include a face increases across one or more further thresholds; and

repeating the obtaining of confidence levels until face detection for multiple predetermined sizes of windows has been performed over an entire region of interest of said image.

27. The method of claim 26, further comprising:

repeating the obtaining of confidence levels below the threshold for multiple additional windows in different locations indicating probabilities in each case that no face is present at the different locations before said performing said second face detection.

28. The method of claim 26, further comprising:

repeating the obtaining of confidence levels for multiple additional windows in different locations until face detection has been performed over an entire region of interest.

29. The method of claim 28, wherein dimensions of said one or more additional windows depend on said confidence level.

30. The method of claim 26, further comprising:

determining a set of regions of interest for said image, and

repeating the obtaining of confidence levels for each of all said regions of interest of said set.