Automatic Focusing Apparatus and Method for Digital Images Using Automatic Filter Switching

• US 20100188558A1
• Filed: 05/05/2009
• Published: 07/29/2010
• Est. Priority Date: 01/28/2009
• Status: Active Grant

• Current Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Original Assignee: Board of Regents of the University of Texas System (University of Texas System)
• Inventors: Gamadia, Mark N., Kehtarnavaz, Nasser
• Granted Patent: US 8,049,811 B2
• Time in Patent Office: Days
• US Class Current: 348/345
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method

Abstract

An apparatus and method for automatically focusing a lens of a digital imaging device, such as a digital still or video camera or camera phone, that reduces the auto focus time while not compromising image sharpness quality and simultaneously reducing device power consumption due to focus actuator movement, all without tuning or adjustment of thresholds or parameters is disclosed. The present invention automatically computes a set of digital band-pass filters matched to the imaging device specifications along with corresponding step-size magnitudes. A filter-switching search for the in-focus focus actuator position is formulated such that filters are switched and step-sizes are reduced as the search gets closer to the in-focus position by utilizing local estimates of the first and second-order differential information of the focus value surface.

Claims

1. An automatic focusing apparatus comprising:

a focusing lens;

a focus actuator connected to the focusing lens to move the focusing lens;

an image sensor aligned with the focusing lens to detect an image through the focusing lens;

j digital band-pass filters connected to the image sensor to extract a set of sharpness samples from a noise-free, blurred, Bayer-pattern sampled image within a focus window from the detected image wherein j>

=2;

a memory storing a set of device parameters; and

a processor connected to the focus actuator, the digital band-pass filters and the memory, wherein the processor (a) generates M sets of spatial frequency parameters and M focus actuator step sizes using the set of device parameters wherein M is a number of segments of a spatial frequency axis, and with i=1 (b) loads the i to i+j sets of spatial frequency parameters into the digital band-pass filters (c) obtains a matrix of focus value samples from the digital band-pass filters, (d) controls the focus actuator to sample a local surface behavior using the i focus actuator step size, (e) repeats steps (b) to (d) with i=i+1 whenever a switch condition is satisfied, and (f) estimates a final in-focus position for the image and controls the focusing lens to move to the final in-focus position whenever a stop condition is satisfied.

2. The automatic focusing apparatus as recited in claim 1, wherein the set of device parameters comprises a F-number of the focusing lens (F_#), a pixel sampling period of the image sensor (T_1,2), and a focus actuator sampling period (T₃).

3. The automatic focusing apparatus as recite in claim 1, wherein the processor generates the M sets of spatial frequency parameters and the M focus actuator step sizes by (a) producing a set of spatial frequencies along the spatial frequency axis using the set of device parameters, (b) determines an effective width in focus actuator steps for each spatial frequency using the set of device parameters, (c) segments the spatial frequency axis into a set of candidate band edges using the effective width for each spatial frequency, (d) constructs the M sets of spatial frequency parameters from the set of candidate band edges, and (e) computes the M focus actuator step sizes using the spatial frequency parameters and the set of device parameters.

4. The automatic focusing apparatus as recite in claim 3, wherein the processor further (i) converts the set of spatial frequencies from cycles per pixel to cycles per mm using the set of device parameters, (ii) periodically samples the effective width to create a balance between an auto-focus search speed and an auto-focus accuracy, (iii) rounds the focus actuator steps to a nearest integer, or (iv) merges the candidate band edges to eliminate any narrow bands.

5. The automatic focusing apparatus as recited in claim 1, wherein each set of spatial frequency parameters comprises a lower frequency band edge, an upper frequency band edge, and a center frequency.

6. The automatic focusing apparatus as recite in claim 1, wherein the processor generates the M sets of spatial frequency parameters and the M focus actuator step sizes by:

(a) periodically sampling the spatial frequency axis in an interval T₄≦

f′

_sp≦

½

with a spacing of T₄ cycles per pixel between successive samples to produce a set of spatial frequencies f′

_sp=n₄T₄, for n₄ varying from 1 to N₄−

1, where N₄=(2T₄)<sup>−

1</sup>+1 and T₄=T₃;

(b) converting the set of spatial frequencies f′

_sp in cycles per pixel to cycles per mm via

7. The automatic focusing apparatus as recited in claim 1, wherein steps (c) to (e) comprise the following steps for each iteration k:

(a) determine whether the stop condition is satisfied;

(b) update the sample state matrix Φ

^(k) with the kth focus value samples as Φ

^(k)=Φ

<sup>(k−

1)</sup>L₃+[0[F_BPF(x₁^(k)) F_BPF(x₀^(k))]^T] where Φ

^(k) is a 2×

3 matrix of samples from a focus value surface, L₃ is a 3×

3 left-shift matrix with ones on the lower diagonal, 0 is a 2×

2 matrix of zeros, F_BPF(x₀^(k)) and F_BPF(x₁^(k)) are the latest focus values computed at step-position n₃^(k) using the digital band-pass filter associated with the center spatial frequencies f₀^(k) and f₁^(k);

(c) normalize the samples in the state matrix Φ

^(k) for comparison of the two focus values via Φ

′

^(k)=diag((F_BPF(x₁^(k)))⁻

, (F_BPF(x₀^(k)))<sup>−

1</sup>Φ

^(k), where diag(.) indicates a 2×

2 diagonal matrix operator.

(d) determine whether a switch condition is not satisfied and repeat steps (a)-(d) for the next iteration k whenever the switch condition is not satisfied;

(e) compute local estimates of the second partial derivatives of the focus value surface expressed as

8. The automatic focusing apparatus as recited in claim 1, wherein the switch condition comprises an indication of a slope sign change.

9. The automatic focusing apparatus as recited in claim 1, wherein the switch condition comprises:

(a) if Φ

′

^(k) is fully populated, then compute an estimate of the following first partial derivative along the focus actuator direction for the filter slot determined by s_j^T,

10. The automatic focusing apparatus as recited in claim 1, wherein the stop condition comprises either an end of a search range has been reached or the search moved to a position beyond the end of the search range.

11. The automatic focusing apparatus as recited in claim 1, wherein the stop condition comprises:

12. The automatic focusing apparatus as recited in claim 1, wherein the final in-focus position is estimated via an inverse quadratic interpolation using the focus value samples and the focus actuator positions corresponding to the digital band-pass filter loaded with the i set of spatial frequency parameters.

13. The automatic focusing apparatus as recited in claim 1, wherein the final in-focus position ({circumflex over (n)}₃^(k*)) is estimated by

14. The automatic focusing apparatus as recited in claim 1, wherein the automatic focusing apparatus is integrated into a digital camera, a cell-phone cameras, a computer, a personal data assistant, a mobile communication device, a hand-held device, or a vehicle.

15. A method for automatically focusing a focusing lens using j digital band-pass filters wherein j>

=2, the method comprising the steps of;

(a) generating M sets of spatial frequency parameters and M focus actuator step sizes using a set of device parameters wherein M is a number of segments of a spatial frequency axis, and with i=1;

(b) loading the i to j sets of spatial frequency parameters into the digital band-pass filters;

(c) obtaining a matrix of focus value samples from the digital band-pass filters;

(d) controlling the focusing lens to sample a local surface behavior using the i focus actuator step size;

(e) repeating steps (b) to (d) with i=i+1 whenever a switch condition is satisfied; and

(f) estimating a final in-focus position for the image and controlling the focusing lens to move to the final in-focus position whenever a stop condition is satisfied.

16. The method as recited in claim 15, wherein the set of device parameters comprises a F-number of the focusing lens (F_#), a pixel sampling period of the image sensor (T_1,2), and a focus actuator sampling period (T₃).

17. The method as recited in claim 15, wherein the step of generating the M sets of spatial frequency parameters and the M focus actuator step sizes comprises the steps of:

(a) producing a set of spatial frequencies along the spatial frequency axis using the set of device parameters;

(b) determines an effective width in focus actuator steps for each spatial frequency using the set of device parameters,

(c) segments the spatial frequency axis into a set of candidate band edges using the effective width for each spatial frequency,

(d) constructs the M sets of spatial frequency parameters from the set of candidate band edges, and

(e) computes the M focus actuator step sizes using the spatial frequency parameters and the set of device parameters.

18. The method as recited in claim 17, further comprising the steps of:

converting the set of spatial frequencies from cycles per pixel to cycles per mm using the set of device parameters;

periodically sampling the effective width to create a balance between an auto-focus search speed and an auto-focus accuracy;

or

rounding the focus actuator steps to a nearest integer, or (iv) merges the candidate band edges to eliminate any narrow bands.

19. The method as recited in claim 15, wherein each set of spatial frequency parameters comprises a lower frequency band edge, an upper frequency band edge, and a center frequency.

20. The method as recited in claim 15, wherein the switch condition comprises an indication of a slope sign change.

21. The method as recited in claim 15, wherein the stop condition comprises either an end of a search range has been reached or the search moved to a position beyond the end of the search range.

22. The method as recited in claim 15, wherein the final in-focus position is estimated via an inverse quadratic interpolation using the focus value samples and the focus actuator positions corresponding to the digital band-pass filter loaded with the i set of spatial frequency parameters.

23. The method as recited in claim 15, wherein the final in-focus position ({circumflex over (n)}₃^(k*)) is estimated by

24. A computer readable medium encoded with a computer program for execution by a processor for automatically focusing a focusing lens using j digital band-pass filters wherein j>

=2, the computer program comprising;

(a) a code segment for generating M sets of spatial frequency parameters and M focus actuator step sizes using a set of device parameters wherein M is a number of segments of a spatial frequency axis, and with i=1;

(b) a code segment for loading the i to i+j sets of spatial frequency parameters into the digital band-pass filters;

(c) a code segment for obtaining a matrix of focus value samples from the digital band-pass filters;

(d) a code segment for controlling the focusing lens to sample a local surface behavior using the i focus actuator step size;

(e) a code segment for repeating steps (b) to (d) with i=i+1 whenever a switch condition is satisfied; and

(f) a code segment for estimating a final in-focus position for the image and controlling the focusing lens to move to the final in-focus position whenever a stop condition is satisfied.

25. The computer program as recited in claim 24, wherein the set of device parameters comprises a F-number of the focusing lens (F_#), a pixel sampling period of the image sensor (T_1,2), and a focus actuator sampling period (T₃).

26. The computer program as recited in claim 24, wherein the code segment for generating the M sets of spatial frequency parameters and the M focus actuator step sizes comprises:

(a) a code segment for producing a set of spatial frequencies along the spatial frequency axis using the set of device parameters;

(b) a code segment for determining an effective width in focus actuator steps for each spatial frequency using the set of device parameters,

(c) a code segment for segmenting the spatial frequency axis into a set of candidate band edges using the effective width for each spatial frequency,

(d) a code segment for constructing the M sets of spatial frequency parameters from the set of candidate band edges, and

(e) a code segment for computing the M focus actuator step sizes using the spatial frequency parameters and the set of device parameters.

27. The computer program as recited in claim 26, further comprising:

a code segment for converting the set of spatial frequencies from cycles per pixel to cycles per mm using the set of device parameters;

a code segment for periodically sampling the effective width to create a balance between an auto-focus search speed and an auto-focus accuracy;

or

a code segment for rounding the focus actuator steps to a nearest integer, or (iv) merges the candidate band edges to eliminate any narrow bands.

28. The computer program as recited in claim 24, wherein each set of spatial frequency parameters comprises a lower frequency band edge, an upper frequency band edge, and a center frequency.

29. The computer program as recited in claim 24, wherein the switch condition comprises an indication of a slope sign change.

30. The computer program as recited in claim 24, wherein the stop condition comprises either an end of a search range has been reached or the search moved to a position beyond the end of the search range.

31. The computer program as recited in claim 24, wherein the final in-focus position is estimated via an inverse quadratic interpolation using the focus value samples and the focus actuator positions corresponding to the digital band-pass filter loaded with the i set of spatial frequency parameters.

32. An automatic focusing apparatus comprising:

a focusing lens;

a focus actuator connected to the focusing lens to move the focusing lens;

an image sensor aligned with the focusing lens to detect an image through the focusing lens;

a first and a second digital band-pass filter connected to the image sensor to extract a set of sharpness samples from a noise-free, blurred, Bayer-pattern sampled image within a focus window from the detected image;

a memory storing a set of device parameters; and

a processor connected to the focus actuator, the digital band-pass filters and the memory, wherein the processor (a) generates M sets of spatial frequency parameters and M focus actuator step sizes using the set of device parameters wherein M is a number of segments of a spatial frequency axis, (b) loads the first and second sets of spatial frequency parameters into the first and second digital band-pass filters, respectively, (c) initializes an iteration k, (d) estimates a final in-focus position and controls the focusing lens to move to the final in-focus position whenever a stop condition is satisfied, (e) whenever the stop condition is not satisfied;

(1) updates a sample state matrix with the kth focus value samples, (2) normalizes the samples in the state matrix for comparison of the focus values, (3) starts a next iteration and repeats steps (1)-(3) whenever a switch condition is not satisfied, (4) whenever the switch condition is satisfied, (5) computes local estimates of the second partial derivatives of the focus value surface and computes the switch condition, (6) computes the switch matrix to be used in the next iteration, (7) computes the analysis filters to be used at the next iteration, (8) updates the sample state matrix if the switch condition is satisfied, (9) updates the step size and updating the focus actuator position, and (10) repeats steps(d)-(e) until the final in-focus position is estimated.

33. A method for automatically focusing a focusing lens using a first and a second digital band-pass filter, the method comprising the steps of:

(a) generating M sets of spatial frequency parameters and M focus actuator step sizes using a set of device parameters wherein M is a number of segments of a spatial frequency axis, and with i=1;

(b) loading the first and second sets of spatial frequency parameters into the first and second digital band-pass filters, respectively;

(c) initializing an iteration k;

(d) estimating a final in-focus position and controlling the focusing lens to move to the final in-focus position whenever a stop condition is satisfied;

(e) whenever the stop condition is not satisfied;

(1) updating a sample state matrix with the kth focus value samples, (2) normalizing the samples in the state matrix for comparison of the focus values, (3) starting a next iteration and repeating steps (1)-(3) whenever a switch condition is not satisfied, (4) whenever the switch condition is satisfied, (5) computing local estimates of the second partial derivatives of the focus value surface and computing the switch condition, (6) computing the switch matrix to be used in the next iteration, (7) computing the analysis filters to be used at the next iteration, (8) updating the sample state matrix if the switch condition is satisfied, (9) updating the step size and updating the focus actuator position, and (10) repeating steps(d)-(e) until the final in-focus position is estimated.

34. A computer readable medium encoded with a computer program for execution by a processor for automatically focusing a focusing lens using j digital band-pass filters wherein j>

=2, the computer program comprising;

(a) a code segment for generating M sets of spatial frequency parameters and M focus actuator step sizes using a set of device parameters wherein M is a number of segments of a spatial frequency axis, and with i=1;

(b) a code segment for loading the first and second sets of spatial frequency parameters into the first and second digital band-pass filters, respectively;

(c) a code segment for initializing an iteration k;

(d) a code segment for estimating a final in-focus position and controlling the focusing lens to move to the final in-focus position whenever a stop condition is satisfied;

(e) a code segment for whenever the stop condition is not satisfied;

(1) updating a sample state matrix with the kth focus value samples, (2) normalizing the samples in the state matrix for comparison of the focus values, (3) starting a next iteration and repeating steps (1)-(3) whenever a switch condition is not satisfied, (4) whenever the switch condition is satisfied, (5) computing local estimates of the second partial derivatives of the focus value surface and computing the switch condition, (6) computing the switch matrix to be used in the next iteration, (7) computing the analysis filters to be used at the next iteration, (8) updating the sample state matrix if the switch condition is satisfied, (9) updating the step size and updating the focus actuator position, and (10) repeating steps(d)-(e) until the final in-focus position is estimated.