Continuous autofocus based on face detection and tracking

• US 8,970,770 B2
• Filed: 12/02/2010
• Issued: 03/03/2015
• Est. Priority Date: 09/28/2010
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Nanu, Florin, Stan, Cosmin Nicolae, Corcoran, Peter
• Primary Examiner(s): Moe, Aung S
• Assistant Examiner(s): Morehead, III, John H
• Application Number:
  US12/959,320
• Publication Number:
  US 20120075492A1
• Time in Patent Office: 1,552 Days
• Field of Search: None
• US Class Current: 348/345
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/672 based on the phase difference signals
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions
• CPC Class Codes: H04N 23/959 by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics

Abstract

An autofocus method includes acquiring an image of a scene that includes one or more out of focus faces and/or partial faces. The method includes detecting one or more of the out of focus faces and/or partial faces within the digital image by applying one or more sets of classifiers trained on faces that are out of focus. One or more sizes of the one of more respective out of focus faces and/or partial faces is/are determined within the digital image. One or more respective depths is/are determined to the one or more out of focus faces and/or partial faces based on the one or more sizes of the one of more faces and/or partial faces within the digital image. One or more respective focus positions of the lens is/are adjusted to focus approximately at the determined one or more respective depths.

Claims

1. An autofocus method for a digital image acquisition device based on face detection, comprising:

using a lens, image sensor and processor of a digital image acquisition device to acquire a digital image of a scene that includes one or more out of focus faces;

detecting the one or more out of focus faces within the digital image by using one or more sets of classifiers trained on faces that are out of focus, wherein the one or more sets of classifiers are reference parameters that are calibrated to reference criteria including at least a set of faces that are out of focus;

determining one or more sizes of the one of more respective out of focus faces within the digital image;

determining one or more respective depths to the one or more out of focus faces based on the one or more sizes of the one of more out of focus faces within the digital image;

adjusting one or more respective focus positions of the lens to focus approximately at the determined one or more respective depths; and

acquiring one or more further images of scenes that include at least one of the one or more out of focus faces with the lens focused at the one or more respectively adjusted focus positions.

2. The method of claim 1, further comprising, upon adjusting the one or more respective focus positions, performing a fine scan, and fine adjusting the one or more respective focus positions based on the fine scan.

3. The method of claim 2, wherein the scene comprises at least one out of focus face not detected by using said one or more sets of face classifiers, and wherein the method further comprises applying a contrast detect scan or a phase detect scan or both for acquiring said at least one out of focus face not detected by using said one or more sets of face classifiers.

4. The method of claim 1, further comprising adjusting at least one of the one or more respective focus positions when at least one of the one or more out of focus faces changes size at least a threshold amount.

5. The method of claim 4, further comprising tracking the at least one of the out of focus faces, and determining the change in size of the at least one of the one or more out of focus faces based on the tracking.

6. The method of claim 1, further comprising increasing depth of field by increasing f-stop number to decrease the aperture of the digital image acquisition device.

7. The method of claim 1, wherein the one or more out of focus faces comprise multiple faces located respectively at multiple different depths approximately determined based on their different determined sizes.

8. The method of claim 1, wherein the step of determining of the one or more depths comprises assigning at least one average depth corresponding to at least one of the one or more determined sizes.

9. The method of claim 1, wherein the step of determining of the one or more depths comprises recognizing a detected face as belonging to a specific individual, calling a known face spatial parameter from a memory that corresponds to the specific face, and determining a depth corresponding to a determined size and the known face spatial parameter.

10. The method of claim 1, wherein the step of adjusting one or more respective focus positions comprises utilizing a MEMS (microelectromechanical system) component.

11. One or more non-transitory processor readable media having code embodied therein for programming a processor to perform an autofocus method for a digital image acquisition device based on face detection, wherein the device has a processor and uses a lens and image sensor to acquire a digital image of a scene that includes one or more out of focus faces, wherein the method comprises:

detecting the one or more out of focus faces within the digital image by using one or more sets of classifiers trained on faces that are out of focus, wherein the one or more sets of classifiers are reference parameters that are calibrated to reference criteria including at least a set of faces that are out of focus;

determining one or more sizes of the one of more respective out of focus faces within the digital image;

determining one or more respective depths to the one or more out of focus faces based on the one or more sizes of the one of more out of focus faces within the digital image;

adjusting one or more respective focus positions of the lens to focus approximately at the determined one or more respective depths; and

acquiring one or more further images of scenes that include at least one of the one or more out of focus faces with the lens focused at the one or more respectively adjusted focus positions.

12. The one or more processor-readable media of claim 11, wherein the method further comprises, upon adjusting the one or more respective focus positions, performing a fine scan, and fine adjusting the one or more respective focus positions based on the fine scan.

13. The one or more processor-readable media of claim 12, wherein the scene comprises at least one out of focus face not detected by using said one or more sets of face classifiers, and wherein the method further comprises applying a contrast detect scan or a phase detect scan or both for acquiring said at least one out of focus face not detected by using said one or more sets of face classifiers.

14. The one or more processor-readable media of claim 11, wherein the method further comprises adjusting at least one of the one or more respective focus positions when at least one of the one or more out of focus faces changes size at least a threshold amount.

15. The one or more processor-readable media of claim 14, wherein the method further comprises tracking the at least one of the out of focus faces, and determining the change in size of the at least one of the one or more out of focus faces based on the tracking.

16. The one or more processor-readable media of claim 11, wherein the method further comprises increasing depth of field by increasing f-stop number to decrease the aperture of the digital image acquisition device.

17. The one or more processor-readable media of claim 11, wherein the one or more out of focus faces comprise multiple faces located respectively at multiple different depths approximately determined based on their different determined sizes.

18. The one or more processor-readable media of claim 11, wherein the step of determining of the one or more depths comprises assigning at least one average depth corresponding to at least one of the one or more determined sizes.

19. The one or more processor-readable media of claim 11, wherein the step of determining of the one or more depths comprises recognizing a detected face as belonging to a specific individual, calling a known face spatial parameter from a memory that corresponds to the specific face, and determining a depth corresponding to a determined size and the known face spatial parameter.

20. The one or more processor-readable media of claim 11, wherein the step of adjusting one or more respective focus positions comprises utilizing a MEMS (microelectromechanical system) component.

21. A digital image acquisition device including a lens, an image sensor, a processor and a memory having code embedded therein for programming the processor to perform an autofocus method based on face detection, wherein the method comprises:

acquiring a digital image of a scene that includes one or more out of focus faces;

detecting the one or more out of focus faces within the digital image by using one or more sets of classifiers trained on faces that are out of focus, wherein the one or more sets of classifiers are reference parameters that are calibrated to reference criteria including at least a set of faces that are out of focus;

determining one or more sizes of the one of more respective out of focus faces within the digital image;

determining one or more respective depths to the one or more out of focus faces based on the one or more sizes of the one of more out of focus faces within the digital image;

adjusting one or more respective focus positions of the lens to focus approximately at the determined one or more respective depths; and

acquiring one or more further images of scenes that include at least one of the one or more out of focus faces with the lens focused at the one or more respectively adjusted focus positions.

22. The device of claim 21, wherein the method further comprises, upon adjusting the one or more respective focus positions, performing a fine scan, and fine adjusting the one or more respective focus positions based on the fine scan.

23. The device of claim 22, wherein the scene comprises at least one out of focus face not detected by using said one or more sets of face classifiers, and wherein the method further comprises applying a contrast detect scan or a phase detect scan or both for acquiring said at least one out of focus face not detected by using said one or more sets of face classifiers.

24. The device of claim 21, wherein the method further comprises adjusting at least one of the one or more respective focus positions when at least one of the one or more out of focus faces changes size at least a threshold amount.

25. The device of claim 24, wherein the method further comprises tracking the at least one of the out of focus faces, and determining the change in size of the at least one of the one or more out of focus faces based on the tracking.

26. The device of claim 21, wherein the method further comprises increasing depth of field by increasing f-stop number to decrease the aperture of the digital image acquisition device.

27. The device of claim 21, wherein the one or more out of focus faces comprise multiple faces located respectively at multiple different depths approximately determined based on their different determined sizes.

28. The device of claim 21, wherein the determining of the one or more depths comprises assigning at least one average depth corresponding to at least one of the one or more determined sizes.

29. The device of claim 21, wherein the determining of the one or more depths comprises recognizing a detected face as belonging to a specific individual, calling a known face spatial parameter from a memory that corresponds to the specific face, and determining a depth corresponding to a determined size and the known face spatial parameter.

30. The device of claim 21, wherein the adjusting one or more respective focus positions comprises utilizing a MEMS (microelectromechanical system) component.