RAPID AUTO-FOCUS USING CLASSIFIER CHAINS, MEMS AND/OR MULTIPLE OBJECT FOCUSING

• US 20120120283A1
• Filed: 11/11/2010
• Published: 05/17/2012
• Est. Priority Date: 11/11/2010
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Capata, Adrian, Petrescu, Stefan, Bigioi, Petronel, Corcoran, Peter
• Granted Patent: US 8,648,959 B2
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06V 40/166 using acquisition arrangements

Abstract

A smart-focusing technique includes identifying an object of interest, such as a face, in a digital image. A focus-generic classifier chain is applied that is trained to match both focused and unfocused faces and/or data from a face tracking module is accepted. Multiple focus-specific classifier chains are applied, including a first chain trained to match substantially out of focus faces, and a second chain trained to match slightly out of focus faces. Focus position is rapidly adjusted using a MEMS component.

Claims

1. A object-based auto-focus method, comprising:

(a) acquiring a digital image including data corresponding to one or more faces or other specific objects of interest, or both (hereinafter “

object” and

“

object data”

) that appears to be out of focus;

(b) applying one or more focus condition classifier programs to the object data;

(c) identifying the object data as corresponding to said object within the digital image;

(d) determining an out of focus condition for the object also as a result of the applying of the one or more focus condition classifier programs;

(e) correcting the out of focus condition of the object based on the determining, to thereby generate a corrected object image appearing to be sharply focused; and

(f) electronically capturing, combining with another image, storing, transmitting, applying an object recognition program to, editing, or displaying the corrected object image, or combinations thereof.

2. The method of claim 1, further comprising applying an object recognition program to the corrected object image.

3. The method of claim 1, wherein the detecting of the object and the determining of the out of focus condition of the object are performed simultaneously.

4. The method of claim 1, further comprising applying a set of feature detector programs to reject non-object data from being identified as object data.

5. The method of claim 1, further comprising applying a sharply focused classifier program to the object data.

6. The method of claim 5, further comprising determining a focus condition based on acceptance of the object data by one of the classifier programs.

7. The method of claim 6, wherein the digital image is one of multiple images in a series that include said object, and wherein said correcting is applied to a different image in the series than said digital image within which the focus condition is determined.

8. The method of claim 1, wherein said out of focus classifier programs comprise one or more slightly out of focus classifiers and one or more significantly out of focus classifiers.

9. The method of claim 8, further comprising applying a sharply focused classifier program to the object data.

10. The method of claim 1, wherein the applying comprises applying at least two full classifier sets after determining that no single focus condition applies and that the object data is not rejected as being a specific object.

11. The method of claim 1, wherein the object comprises a face.

12. An object detection method, comprising:

acquiring a digital image;

extracting a sub-window from said image;

applying two or more shortened face detection classifier cascades, trained to be selectively sensitive to a focus condition of one or more faces or other specific objects of interest, or both (hereinafter “

object”

);

based on the applying, determining a probability that a specific object with a certain focus condition is present within the sub-window;

based on the determining, applying an extended object detection classifier cascade trained for sensitivity to said certain focus condition;

providing a final determination that the specific object exists within the image sub-window; and

repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further focus conditions, or both.

13. The method of claim 12, further comprising determining a focus condition by applying one or more out of focus classifier cascades.

14. The method of claim 13, further comprising applying a sharply focused classifier cascade.

15. The method of claim 14, further comprising determining a focus condition of the specific object within a sub-window based on acceptance by one of the classifier cascades.

16. The method of claim 15, wherein the digital image is one of multiple images in a series that include the specific object, and the method further comprises correcting an out of focus condition of the specific object within a different image in the series than said digital image within which the focus condition is determined.

17. The method of claim 13, wherein said out of focus classifier cascades comprise one or more slightly out of focus classifiers and one or more significantly out of focus classifiers.

18. The method of claim 12, wherein the object comprises a face.

19. One or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform an auto-focus method, wherein the method comprises:

(a) acquiring a digital image including data corresponding to one or more faces or other specific objects of interest, or both (hereinafter “

object” and

“

object data”

) that appears to be out of focus;

(b) applying one or more focus condition classifier programs to the object data;

(c) identifying the object data as corresponding to said object within the digital image;

(d) determining an out of focus condition for the object also as a result of the applying of the one or more focus condition classifier programs;

(e) correcting the out of focus condition of the object based on the determining, to thereby generate a corrected object image appearing to be sharply focused; and

(f) electronically capturing, combining with another image, storing, transmitting, applying an object recognition program to, editing, or displaying the corrected object image, or combinations thereof.

20. The one or more non-transitory processor-readable media of claim 19, wherein the method further comprises applying an object recognition program to the corrected object image.

21. The one or more non-transitory processor-readable media of claim 19, wherein the detecting of the object and the determining of the out of focus condition of the object are performed simultaneously.

22. The one or more non-transitory processor-readable media of claim 19, wherein the method further comprises applying a set of feature detector programs to reject non-object data from being identified as object data.

23. The one or more non-transitory processor-readable media of claim 19, wherein the method further comprises applying a sharply focused classifier program to the object data.

24. The one or more non-transitory processor-readable media of claim 23, wherein the method further comprises determining a focus condition based on acceptance of the object data by one of the classifier programs.

25. The one or more non-transitory processor-readable media of claim 24, wherein the digital image is one of multiple images in a series that include said object, and wherein said correcting is applied to a different image in the series than said digital image within which the focus condition is determined.

26. The one or more non-transitory processor-readable media of claim 19, wherein said out of focus classifier programs comprise one or more slightly out of focus classifiers and one or more significantly out of focus classifiers.

27. The one or more non-transitory processor-readable media of claim 26, wherein the method further comprises applying a sharply focused classifier program to the object data.

28. The one or more non-transitory processor-readable media of claim 19, wherein the applying comprises applying at least two full classifier sets after determining that no single focus condition applies and that the object data is not rejected as being a specific object.

29. The one or more non-transitory processor-readable media of claim 19, wherein the object comprises a face.

30. One or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform an object detection method, wherein the method comprises:

(a) acquiring a digital image

(b) extracting a sub-window from said image

(c) applying two or more shortened face detection classifier cascades, trained to be selectively sensitive to a focus condition of one or more faces or other specific objects of interest, or both (hereinafter “

object”

),

(d) based on the applying, determining a probability that a specific object with a certain focus condition is present within the sub-window;

(e) based on the determining, applying an extended object detection classifier cascade trained for sensitivity to said certain focus condition;

(f) providing a final determination that the specific object exists within the image sub-window; and

(g) repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further focus conditions, or both.

31. The one or more non-transitory processor-readable media of claim 30, wherein the method further comprises determining a focus condition by applying one or more out of focus classifier cascades.

32. The one or more non-transitory processor-readable media of claim 31, wherein the method further comprises applying a sharply focused classifier cascade.

33. The one or more non-transitory processor-readable media of claim 32, wherein the method further comprises determining a focus condition of the specific object within a sub-window based on acceptance by one of the classifier cascades.

34. The one or more non-transitory processor-readable media of claim 33, wherein the digital image is one of multiple images in a series that include the specific object, and the method further comprises correcting an out of focus condition of the specific object within a different image in the series than said digital image within which the focus condition is determined.

35. The one or more non-transitory processor-readable media of claim 31, wherein said out of focus classifier cascades comprise one or more slightly out of focus classifiers and one or more significantly out of focus classifiers.

36. The one or more non-transitory processor-readable media of claim 30, wherein the object comprises a face.

37. A digital image acquisition device, comprising:

a lens and image sensor for acquiring digital images;

a processor; and

one or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform an auto-focus method, wherein the method comprises;

(a) acquiring a digital image including data corresponding to one or more faces or other specific objects of interest, or both (hereinafter “

object” and

“

object data”

) that appears to be out of focus;

(b) applying one or more focus condition classifier programs to the object data;

(c) identifying the object data as corresponding to said object within the digital image;

(d) determining an out of focus condition for the object also as a result of the applying of the one or more focus condition classifier programs;

(e) correcting the out of focus condition of the object based on the determining, to thereby generate a corrected object image appearing to be sharply focused; and

(f) electronically capturing, combining with another image, storing, transmitting, applying an object recognition program to, editing, or displaying the corrected object image, or combinations thereof.

38. The device of claim 37, wherein the method further comprises applying an object recognition program to the corrected object image.

39. The device of claim 37, wherein the detecting of the object and the determining of the out of focus condition of the object are performed simultaneously.

40. The device of claim 37, wherein the method further comprises applying a set of feature detector programs to reject non-object data from being identified as object data.

41. The device of claim 37, wherein the method further comprises applying a sharply focused classifier program to the object data.

42. The device of claim 41, wherein the method further comprises determining a focus condition based on acceptance of the object data by one of the classifier programs.

43. The device of claim 42, wherein the digital image is one of multiple images in a series that include said object, and wherein said correcting is applied to a different image in the series than said digital image within which the focus condition is determined.

44. The device of claim 37, wherein said out of focus classifier programs comprise one or more slightly out of focus classifiers and one or more significantly out of focus classifiers.

45. The device of claim 44, wherein the method further comprises applying a sharply focused classifier program to the object data.

46. The device of claim 37, wherein the applying comprises applying at least two full classifier sets after determining that no single focus condition applies and that the object data is not rejected as being a specific object.

47. The device of claim 37, wherein the object comprises a face.

48. A digital image acquisition device, comprising:

a lens and image sensor for acquiring digital images;

a processor; and

one or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform an object detection method, wherein the method comprises;

acquiring a digital image;

extracting a sub-window from said image;

applying two or more shortened face detection classifier cascades, trained to be selectively sensitive to a focus condition of one or more faces or other specific objects of interest, or both (hereinafter “

object”

);

based on the applying, determining a probability that a specific object with a certain focus condition is present within the sub-window;

based on the determining, applying an extended object detection classifier cascade trained for sensitivity to said certain focus condition;

providing a final determination that the specific object exists within the image sub-window; and

repeating steps (b)-(e) one or more times for one or more further sub-windows from the image or one or more further focus conditions, or both.

49. The device of claim 48, wherein the method further comprises determining a focus condition by applying one or more out of focus classifier cascades.

50. The device of claim 49, wherein the method further comprises applying a sharply focused classifier cascade.

51. The device of claim 50, wherein the method further comprises determining a focus condition of the specific object within a sub-window based on acceptance by one of the classifier cascades.

52. The device of claim 51, wherein the digital image is one of multiple images in a series that include the specific object, and the method further comprises correcting an out of focus condition of the specific object within a different image in the series than said digital image within which the focus condition is determined.

53. The device of claim 49, wherein said out of focus classifier cascades comprise one or more slightly out of focus classifiers and one or more significantly out of focus classifiers.

54. The device of claim 48, wherein the object comprises a face.