Rapid auto-focus using classifier chains, MEMS and/or multiple object focusing

• US 8,648,959 B2
• Filed: 11/11/2010
• Issued: 02/11/2014
• Est. Priority Date: 11/11/2010
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Capata, Adrian, Petrescu, Stefan, Bigioi, Petronel, Corcoran, Peter
• Primary Examiner(s): Tran, Nhan T
• Application Number:
  US12/944,701
• Publication Number:
  US 20120120283A1
• Time in Patent Office: 1,188 Days
• Field of Search: 348/345, 348/346, 348/169, 348/239, 348/349, 348352-354, 382/118, 382/255
• US Class Current: 348/345
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06V 40/166 using acquisition arrangements

Abstract

A smart-focusing technique includes identifying an object of interest, such as a face, in a digital image. A focus-generic classifier chain is applied that is trained to match both focused and unfocused faces and/or data from a face tracking module is accepted. Multiple focus-specific classifier chains are applied, including a first chain trained to match substantially out of focus faces, and a second chain trained to match slightly out of focus faces. Focus position is rapidly adjusted using a MEMS component.

Claims

1. An object-based auto-focus method, comprising:

(a) acquiring a digital image including data corresponding to one or more faces or other specific objects of interest, or both (hereinafter an “

object” and

an “

object data”

) that appears to be out of focus;

(b) applying one or more focus condition classifier programs to the object data, each of the one or more focus condition classifier programs being tuned to a different non-zero degree of out of focus blur, wherein said out of focus classifier programs each comprises one or more slightly out of focus classifiers or one or more significantly out of focus classifiers;

(c) identifying the object data as corresponding to said object within the digital image;

(d) determining an out of focus condition for the object also as a result of the applying of the one or more focus condition classifier programs, wherein said out of focus condition corresponds to a particular degree of out of focus blur at which one of the programs is tuned;

(e) correcting the out of focus condition of the object based on the determining, to thereby generate a corrected object image appearing to be sharply focused; and

(f) electronically capturing, combining with another image, storing, transmitting, applying an object recognition program to, editing, or displaying the corrected object image, or combinations thereof.

2. The method of claim 1, further comprising applying an object recognition program to the corrected object image.

3. The method of claim 1, wherein the identifying the object data and the determining of the out of focus condition of the object are performed simultaneously.

4. The method of claim 1, further comprising applying a set of feature detector programs to reject non-object data from being identified as the object data.

5. The method of claim 1, further comprising applying a sharply focused classifier program to the object data.

6. The method of claim 5, further comprising determining a focus condition based on acceptance of the object data by one of the classifier programs.

7. The method of claim 6, wherein the digital image is one of multiple images in a series that include said object, and wherein said correcting is applied to a different image in the series than said digital image within which the focus condition is determined.

8. The method of claim 1, further comprising applying a sharply focused classifier program to the object data.

9. The method of claim 1, wherein the applying comprises applying at least two full classifier sets after determining that no single focus condition applies and that the object data is not rejected as being a specific object.

10. The method of claim 1, wherein the object comprises a face.

11. An object detection method, comprising:

(a) acquiring a digital image;

(b) extracting a sub-window from said image;

(c) applying two or more shortened face detection classifier cascades, each trained to be selectively sensitive to a different focus condition of one or more faces or other specific objects of interest, or both (hereinafter “

object”

), wherein each different focus condition corresponds to a different degree of out of focus blur of the object;

(d) based on the applying, determining a probability that a specific object with a certain focus condition, corresponding to a certain degree of out of focus blur of the is present within the sub-window;

(e) based on the determining, applying an extended object detection classifier cascade trained for sensitivity to said certain focus condition, wherein the certain focus condition is determined by applying one or more out of focus classifier cascades which each comprises one or more slightly out of focus classifiers or one or more significantly out of focus classifiers;

(f) providing a final determination that the specific object exists within the image sub-window; and

(g) repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further focus conditions, or both.

12. The method of claim 11, further comprising applying a sharply focused classifier cascade.

13. The method of claim 12, further comprising determining a focus condition of the specific object within a sub-window based on acceptance by one of the classifier cascades.

14. The method of claim 13, wherein the digital image is one of multiple images in a series that include the specific object, and the method further comprises correcting an out of focus condition of the specific object within a different image in the series than said digital image within which the focus condition is determined.

15. The method of claim 11, wherein the object comprises a face.

16. One or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform an auto-focus method, wherein the method comprises:

(a) acquiring a digital image including data corresponding to one or more faces or other specific objects of interest, or both (hereinafter an “

object” and

an “

object data”

) that appears to be out of focus;

(b) applying one or more focus condition classifier programs to the object data, each of the one or more focus condition classifier programs being tuned to a different non-zero degree of out of focus blur, wherein said out of focus classifier programs each comprises one or more slightly out of focus classifiers or one or more significantly out of focus classifiers;

(c) identifying the object data as corresponding to said object within the digital image;

(d) determining an out of focus condition for the object also as a result of the applying of the one or more focus condition classifier programs, wherein said out of focus condition corresponds to a particular degree of out of focus blur at which one of the programs is tuned;

(e) correcting the out of focus condition of the object based on the determining, to thereby generate a corrected object image appearing to be sharply focused; and

(f) electronically capturing, combining with another image, storing, transmitting, applying an object recognition program to, editing, or displaying the corrected object image, or combinations thereof.

17. The one or more non-transitory processor-readable media of claim 16, wherein the method further comprises applying an object recognition program to the corrected object image.

18. The one or more non-transitory processor-readable media of claim 16, wherein the identifying the object data and the determining of the out of focus condition of the object are performed simultaneously.

19. The one or more non-transitory processor-readable media of claim 16, wherein the method further comprises applying a set of feature detector programs to reject non-object data from being identified as the object data.

20. The one or more non-transitory processor-readable media of claim 16, wherein the method further comprises applying a sharply focused classifier program to the object data.

21. The one or more non-transitory processor-readable media of claim 20, wherein the method further comprises determining a focus condition based on acceptance of the object data by one of the classifier programs.

22. The one or more non-transitory processor-readable media of claim 21, wherein the digital image is one of multiple images in a series that include said object, and wherein said correcting is applied to a different image in the series than said digital image within which the focus condition is determined.

23. The one or more non-transitory processor-readable media of claim 16, wherein the method further comprises applying a sharply focused classifier program to the object data.

24. The one or more non-transitory processor-readable media of claim 16, wherein the applying comprises applying at least two full classifier sets after determining that no single focus condition applies and that the object data is not rejected as being a specific object.

25. The one or more non-transitory processor-readable media of claim 16, wherein the object comprises a face.

26. One or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform an object detection method, wherein the method comprises:

(a) acquiring a digital image;

(b) extracting a sub-window from said image;

(c) applying two or more shortened face detection classifier cascades, each trained to be selectively sensitive to a different focus condition of one or more faces or other specific objects of interest, or both (hereinafter “

object”

), wherein each different focus condition corresponds to a different degree of out of focus blur of the object;

(d) based on the applying, determining a probability that a specific object with a certain focus condition, corresponding to a certain degree of out of focus blur of the is present within the sub-window;

(e) based on the determining, applying an extended object detection classifier cascade trained for sensitivity to said certain focus condition, wherein the certain focus condition is determined by applying one or more out of focus classifier cascades which each comprises one or more slightly out of focus classifiers or one or more significantly out of focus classifiers;

(f) providing a final determination that the specific object exists within the image sub-window; and

(g) repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further focus conditions, or both.

27. The one or more non-transitory processor-readable media of claim 26, wherein the method further comprises applying a sharply focused classifier cascade.

28. The one or more non-transitory processor-readable media of claim 27, wherein the method further comprises determining a focus condition of the specific object within a sub-window based on acceptance by one of the classifier cascades.

29. The one or more non-transitory processor-readable media of claim 28, wherein the digital image is one of multiple images in a series that include the specific object, and the method further comprises correcting an out of focus condition of the specific object within a different image in the series than said digital image within which the focus condition is determined.

30. The one or more non-transitory processor-readable media of claim 26, wherein the object comprises a face.

31. A digital image acquisition device, comprising:

a lens and image sensor for acquiring digital images;

a processor; and

one or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform an auto-focus method, wherein the method comprises;

(a) acquiring a digital image including data corresponding to one or more faces or other specific objects of interest, or both (hereinafter an “

object” and

an “

object data”

) that appears to be out of focus;

(b) applying one or more focus condition classifier programs to the object data, each of the one or more focus condition classifier programs being tuned to a different non-zero degree of out of focus blur, wherein said out of focus classifier programs each comprises one or more slightly out of focus classifiers or one or more significantly out of focus classifiers;

(c) identifying the object data as corresponding to said object within the digital image;

(d) determining an out of focus condition for the object also as a result of the applying of the one or more focus condition classifier programs, wherein said out of focus condition corresponds to a particular degree of out of focus blur at which one of the programs is tuned;

(e) correcting the out of focus condition of the object based on the determining, to thereby generate a corrected object image appearing to be sharply focused; and

(f) electronically capturing, combining with another image, storing, transmitting, applying an object recognition program to, editing, or displaying the corrected object image, or combinations thereof.

32. The device of claim 31, wherein the method further comprises applying an object recognition program to the corrected object image.

33. The device of claim 31, wherein the identifying the object data and the determining of the out of focus condition of the object are performed simultaneously.

34. The device of claim 31, wherein the method further comprises applying a set of feature detector programs to reject non-object data from being identified as the object data.

35. The device of claim 31, wherein the method further comprises applying a sharply focused classifier program to the object data.

36. The device of claim 35, wherein the method further comprises determining a focus condition based on acceptance of the object data by one of the classifier programs.

37. The device of claim 36, wherein the digital image is one of multiple images in a series that include said object, and wherein said correcting is applied to a different image in the series than said digital image within which the focus condition is determined.

38. The device of claim 31, wherein the method further comprises applying a sharply focused classifier program to the object data.

39. The device of claim 31, wherein the applying comprises applying at least two full classifier sets after determining that no single focus condition applies and that the object data is not rejected as being a specific object.

40. The device of claim 31, wherein the object comprises a face.

41. A digital image acquisition device, comprising:

a lens and image sensor for acquiring digital images;

a processor; and

one or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform an object detection method, wherein the method comprises;

(a) acquiring a digital image;

(b) extracting a sub-window from said image;

(c) applying two or more shortened face detection classifier cascades, each trained to be selectively sensitive to a different focus condition of one or more faces or other specific objects of interest, or both (hereinafter “

object”

), wherein each different focus condition corresponds to a different degree of out of focus blur of the object;

(d) based on the applying, determining a probability that a specific object with a certain focus condition, corresponding to a certain degree of out of focus blur of the object, is present within the sub-window;

(e) based on the determining, applying an extended object detection classifier cascade trained for sensitivity to said certain focus condition, wherein the certain focus condition is determined by applying one or more out of focus classifier cascades which each comprises one or more slightly out of focus classifiers or one or more significantly out of focus classifiers;

(f) providing a final determination that the specific object exists within the image sub-window; and

(g) repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further focus conditions, or both.

42. The device of claim 41, wherein the method further comprises applying a sharply focused classifier cascade.

43. The device of claim 42, wherein the method further comprises determining a focus condition of the specific object within a sub-window based on acceptance by one of the classifier cascades.

44. The device of claim 43, wherein the digital image is one of multiple images in a series that include the specific object, and the method further comprises correcting an out of focus condition of the specific object within a different image in the series than said digital image within which the focus condition is determined.

45. The device of claim 41, wherein the object comprises a face.