Rapid auto-focus using classifier chains, MEMS and multiple object focusing

• US 8,797,448 B2
• Filed: 02/06/2014
• Issued: 08/05/2014
• Est. Priority Date: 11/11/2010
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Capata, Adrian, Petrescu, Stefan, Bigioi, Petronel, Corcoran, Peter
• Primary Examiner(s): Tran, Nhan T
• Application Number:
  US14/174,821
• Publication Number:
  US 20140193088A1
• Time in Patent Office: 180 Days
• Field of Search: 348/345, 348/346, 348/169, 348/239, 348/349, 348352-354, 382/118, 382/255
• US Class Current: 348/345
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06V 40/166 using acquisition arrangements

Abstract

A smart-focusing technique includes identifying an object of interest, such as a face, in a digital image. A focus-generic classifier chain is applied that is trained to match both focused and unfocused faces and/or data from a face tracking module is accepted. Multiple focus-specific classifier chains are applied, including a first chain trained to match substantially out of focus faces, and a second chain trained to match slightly out of focus faces. Focus position is rapidly adjusted using a MEMS component.

Claims

1. A method comprising:

acquiring data of a digital image which depicts one or more objects;

determining a plurality of in-focus probabilities by applying a plurality of in-focus classifier sets to the data;

wherein each in-focus probability indicates a likelihood that any of the one or more objects is depicted in focus in the digital image;

determining an in-focus cumulative probability based on the plurality of in-focus probabilities;

determining whether the in-focus cumulative probability is below an in-focus threshold;

in response to determining that the in-focus cumulative probability is below the in-focus threshold, performing the steps of;

determining a plurality of slightly-out-of-focus probabilities by applying a plurality of slightly-out-of-focus classifier sets to the data;

wherein each slightly-out-of-focus probability indicates a likelihood that any of the one or more objects is depicted slightly-out-of focus in the digital image;

determining slightly-out-of-focus cumulative probability based on the plurality of slightly-out-of-focus probabilities; and

in response to determining that the slightly-out-of-focus cumulative probability is equal to, or above, a slightly-out-of-focus threshold, determining a particular object, of the one or more objects, that is depicted in the digital image slightly-out-of-focus; and

wherein the method is performed by one or more computing devices.

2. The method of claim 1,

wherein an in-focus classifier set of the plurality of in-focus classifier sets comprises an in-focus-program configured to determine an in-focus probability that any of the one or more objects is depicted in-focus in the digital image;

wherein a slightly-out-of-focus classifier set of the one or more slightly-out-of-focus classifier sets comprises a slightly-out-of-focus-program configured to determine a slightly-out-of-focus probability that any of the one or more objects is depicted slightly-out-of-focus in the digital image; and

wherein the slightly-out-of-focus cumulative probability is computed from one or more slightly-out-of-focus probabilities determined by the one or more slightly-out-of-focus classifier sets.

3. The method of claim 2, further comprising:

determining whether the slightly-out-of-focus cumulative probability is below a slightly-out-of-focus threshold; and

in response to determining that the slightly-out-of-focus cumulative probability is below, the slightly-out-of-focus threshold, applying one or more significantly-out-of-focus classifier sets to determine a significantly-out-of-focus cumulative probability.

4. The method of claim 3,

wherein a significantly-out-of-focus classifier set of the one or more significantly-out-of-focus classifier sets comprises a significantly-out-of-focus-program configured to determine a significantly-out-of-focus probability that any of the one or more objects is depicted significantly-out-of-focus in the digital image;

and

wherein the significantly-out-of-focus cumulative probability is computed from one or more significantly-out-of-focus probabilities determined by the one or more significantly-out-of-focus classifier sets.

5. The method of claim 4, further comprising:

determining whether the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold, determining the particular object, of the one or more objects, that is depicted in the digital image significantly-out-of-focus.

6. The method of claim 5, further comprising:

determining whether the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold, determining that none of the one or more objects is depicted in the digital image.

7. The method of claim 6, further comprising:

in response to determining that the in-focus cumulative probability is above, or equal to, the in-focus threshold, determining the particular object, of the one or more objects, that is depicted in-focus face in the digital image.

8. A non-transitory computer-readable storage medium, storing one or more computer instructions which, when executed by one or more processors, cause the one or more processors to perform:

acquiring data of a digital image which depicts one or more objects;

determining a plurality of in-focus probabilities by applying a plurality of in-focus classifier sets to the data;

wherein each in-focus probability indicates a likelihood that any of the one or more objects is depicted in focus in the digital image;

determining an in-focus cumulative probability based on the plurality of in-focus probabilities;

determining whether the in-focus cumulative probability is below an in-focus threshold;

in response to determining that the in-focus cumulative probability is below the in-focus threshold, performing the steps of;

determining a plurality of slightly-out-of-focus probabilities by applying a plurality of slightly-out-of-focus classifier sets to the data;

wherein each slightly-out-of-focus probability indicates a likelihood that any of the one or more objects is depicted slightly-out-of focus in the digital image;

determining slightly-out-of-focus cumulative probability based on the plurality of slightly-out-of-focus probabilities; and

in response to determining that the slightly-out-of-focus cumulative probability is equal to, or above, a slightly-out-of-focus threshold, determining a particular object, of the one or more objects, that is depicted in the digital image slightly-out-of-focus.

9. The non-transitory computer-readable storage medium of claim 8,

wherein an in-focus classifier set of the plurality of in-focus classifier sets comprises an in-focus-program configured to determine an in-focus probability that any of the one or more objects is depicted in-focus in the digital image;

wherein a slightly-out-of-focus classifier set of the one or more slightly-out-of-focus classifier sets comprises a slightly-out-of-focus-program configured to determine a slightly-out-of-focus probability that any of the one or more objects is depicted slightly-out-of-focus in the digital image; and

wherein the slightly-out-of-focus cumulative probability is computed from one or more slightly-out-of-focus probabilities determined by the one or more slightly-out-of-focus classifier sets.

10. The non-transitory computer-readable storage medium of claim 9, storing additional instructions which, when executed, cause the one or more processors to perform:

determining whether the slightly-out-of-focus cumulative probability is below a slightly-out-of-focus threshold; and

in response to determining that the slightly-out-of-focus cumulative probability is below, the slightly-out-of-focus threshold, applying one or more significantly-out-of-focus classifier sets to determine a significantly-out-of-focus cumulative probability.

11. The non-transitory computer-readable storage medium of claim 10,

wherein a significantly-out-of-focus classifier set of the one or more significantly-out-of-focus classifier sets comprises a significantly-out-of-focus-program configured to determine a significantly-out-of-focus probability that any of the one or more objects is depicted significantly-out-of-focus in the digital image;

and

wherein the significantly-out-of-focus cumulative probability is computed from one or more significantly-out-of-focus probabilities determined by the one or more significantly-out-of-focus classifier sets.

12. The non-transitory computer-readable storage medium of claim 11, storing additional instructions which, when executed, cause the one or more processors to perform:

determining whether the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold, determining the particular object, of the one or more objects, that is depicted in the digital image significantly-out-of-focus.

13. The non-transitory computer-readable storage medium of claim 12, storing additional instructions which, when executed, cause the one or more processors to perform:

determining whether the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold, determining that none of the one or more objects is depicted in the digital image.

14. The non-transitory computer-readable storage medium of claim 13, storing additional instructions which, when executed, cause the one or more processors to perform:

in response to determining that the in-focus cumulative probability is above, or equal to, the in-focus threshold, determining the particular object, of the one or more objects, that is depicted in-focus face in the digital image.

15. A device, comprising:

an image retrieval unit coupled to one or more memory units and configured to acquire data of a digital image which depicts one or more objects; and

an image analysis unit configured to;

determine a plurality of in-focus probabilities by applying a plurality of in-focus classifier sets to the data;

wherein each in-focus probability indicates a likelihood that any of the one or more objects is depicted in focus in the digital image;

determine an in-focus cumulative probability based on the plurality of in-focus probabilities;

determine whether the in-focus cumulative probability is below an in-focus threshold;

in response to determining that the in-focus cumulative probability is below the in-focus threshold, performing the steps of;

determine a plurality of slightly-out-of-focus probabilities by applying a plurality of slightly-out-of-focus classifier sets to the data;

wherein each slightly-out-of-focus probability indicates a likelihood that any of the one or more objects is depicted slightly-out-of focus in the digital image;

determine slightly-out-of-focus cumulative probability based on the plurality of slightly-out-of-focus probabilities; and

in response to determining that the slightly-out-of-focus cumulative probability is equal to, or above, a slightly-out-of-focus threshold, determine a particular object, of the one or more objects, that is depicted in the digital image slightly-out-of-focus.

16. The device of claim 15,

wherein an in-focus classifier set of the plurality of in-focus classifier sets comprises an in-focus-program configured to determine an in-focus probability that any of the one or more objects is depicted in-focus in the digital image;

wherein a slightly-out-of-focus classifier set of the one or more slightly-out-of-focus classifier sets comprises a slightly-out-of-focus-program configured to determine a slightly-out-of-focus probability that any of the one or more objects is depicted slightly-out-of-focus in the digital image; and

wherein the slightly-out-of-focus cumulative probability is computed from one or more slightly-out-of-focus probabilities determined by the one or more slightly-out-of-focus classifier sets.

17. The device of claim 16, wherein the image analysis unit is further configured to:

determine whether the slightly-out-of-focus cumulative probability is below a slightly-out-of-focus threshold; and

in response to determining that the slightly-out-of-focus cumulative probability is below, the slightly-out-of-focus threshold, apply one or more significantly-out-of-focus classifier sets to determine a significantly-out-of-focus cumulative probability.

18. The device of claim 17,

wherein a significantly-out-of-focus classifier set of the one or more significantly-out-of-focus classifier sets comprises a significantly-out-of-focus-program configured to determine a significantly-out-of-focus probability that any of the one or more objects is depicted significantly-out-of-focus in the digital image;

and

wherein the significantly-out-of-focus cumulative probability is computed from one or snore significantly-out-of-focus probabilities determined by the one or more significantly-out-of-focus classifier sets.

19. The device of claim 18, wherein the image analysis unit is further configured to:

determine whether the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold, determine the particular object, of the one or more objects, that is depicted in the digital image significantly-out-of-focus.

20. The device of claim 19, wherein the image analysis unit is further configured to:

determine whether the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold, determine that none of the one or more objects is depicted in the digital image.