Apparatus, method, and manufacture for iterative auto-focus using depth-from-defocus

• US 8,218,061 B2
• Filed: 03/23/2009
• Issued: 07/10/2012
• Est. Priority Date: 09/04/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: CSR Technology Incorporated (Qualcomm, Inc.)
• Inventors: Baxansky, Artemy
• Primary Examiner(s): Laballe, Clayton E
• Assistant Examiner(s): Hancock, Dennis
• Application Number:
  US12/409,416
• Publication Number:
  US 20100053417A1
• Time in Patent Office: 1,205 Days
• Field of Search: 348/345
• US Class Current: 348/345
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions

Abstract

A camera with auto-focusing performs multiple iterations in which the distance from the camera to an object is estimated for the purpose of auto-focusing the camera. Each estimate is made using depth-from-defocus with at least two images used to make the estimate. When the two images are taken, the camera parameters are different. For example, the focus position, aperture, or zoom may be different between the two images. In each subsequent iteration, the previous estimate of distance from the camera to the object is used so that the parameter varied (focus position or zoom) is closer to that corresponding to the estimated distance from the camera to the object, so that the estimated distance is closer to the actual distance in each iteration.

Claims

1. A method for calculating a distance from a camera to an object, comprising acts of:

obtaining a first image of an object employing a first set of parameters;

obtaining a second image of the object employing a second set of parameters;

determining a width of blur at corresponding edges for each of the first and second images;

generating a first estimate of the distance from the camera to the object such that the first estimate is obtained from at least the first image and the second image using depth-from-defocus based at least on an equality comparison of the width of blur determined for each of the first image and the second image;

obtaining a third image of the object employing a third set of parameters, wherein the third set of parameters are based, at least in part, on the first estimate;

obtaining a fourth image of the object employing a fourth set of parameters, wherein the fourth set of parameters are based, at least in part, on the first estimate; and

determining a width of blur at corresponding edges for each of the third and fourth images;

generating a second estimate of the distance from the camera to the object such that the second estimate is obtained from at least the third image and the fourth image using depth-from-defocus based at least on an equality comparison of the width of blur determined for each of the third image and the fourth image, the calculated distance from the camera to the object being based at least in part on the second estimate.

2. The method of claim 1, wherein the first, second, third, and fourth set of parameters each include a focus position, a zoom, or an aperture.

3. The method of claim 1, further comprising acts of:

obtaining a fifth image of the object employing a fifth set of parameters, wherein the fifth set of parameters are based, at least in part, on the second estimate;

obtaining a sixth image of the object employing a sixth set of parameters, wherein the sixth set of parameters are based, at least in part, on the second estimate; and

generate a third estimate of the distance from the camera to the object such that the third estimate is obtained from at least the fifth image and the sixth image using depth-from-defocus, the calculated distance from the camera to the object being based at least in part on the third estimate.

4. The method of claim 3, further comprising acts of:

calculating a maximum error for the first estimate of the distance from the camera to the object, wherein obtaining the third image is accomplished such that the third set of parameters correspond to a distance from the camera to the object of u1−

Δ

u1, and wherein obtaining the fourth image is accomplished such that the fourth set of parameters correspond to a distance from the camera to the object of u1+Δ

u1, where u1 represents the first estimate of the distance from the camera to the object, and where Δ

u1 represents the calculated maximum error for the first estimate of the distance from the camera to the object; and

calculating a maximum error for the second estimate of the distance from the camera to the object, wherein obtaining the fifth image is accomplished such that the fifth set of parameters correspond to a distance from the camera to the object of u2−

Δ

u2, and wherein obtaining the sixth image is accomplished such that the sixth set of parameters correspond to a distance from the camera to the object of u2+Δ

u2, where u2 represents the second estimate of the distance from the camera to the object, and where Δ

u2 represents the calculated maximum error for the second estimate of the distance from the camera to the object.

5. The method of claim 3, wherein the first, second, third, fourth, fifth, and sixth set of parameters each include a focus position, and

wherein performing the automatic focusing further includes:

calculating a maximum error for the first estimate of the distance from the camera to the object, wherein obtaining the third image is accomplished such that the focus position of the third set of parameters corresponds to a distance from the camera to the object of u1−

Δ

u1, and wherein obtaining the fourth image is accomplished such that the focus position of the fourth set of parameters corresponds to a distance from the camera to the object of u1+Δ

u1, where u1 represents the first estimate of the distance from the camera to the object, and where Δ

u1 represents the calculated maximum error for the first estimate of the distance from the camera to the object; and

calculating a maximum error for the second estimate of the distance from the camera to the object, wherein obtaining the fifth image is accomplished such that the focus position of the fifth set of parameters corresponds to a distance from the camera to the object of u2−

Δ

u2, and wherein obtaining the sixth image is accomplished such that the focus position of the sixth set of parameters corresponds to a distance from the camera to the object of u2+Δ

u2, where u2 represents the second estimate of the distance from the camera to the object, and where Δ

u2 represents the calculated maximum error for the second estimate of the distance from the camera to the object.

6. The method of claim 1, wherein performing the automatic focusing further includes:

obtaining another image of the object employing another set of parameters, wherein generating the first estimate is obtained from the first image, the second image, and said another image using depth-from-defocus.

7. The method of claim 1, wherein the third set of parameters includes a first focus position, the fourth set of parameters include a second focus position, and wherein the third set of parameters are substantially identical to the fourth set of parameters except that second position is different from the first focus position.

8. The method of claim 1, wherein generating the first estimate using depth-from-defocus includes:

calculating a gradient of a portion of the first image that is common to the first and second images;

determining the maximum gradient of the portion of the first image based on the calculated gradient of the portion of the first image;

calculating a gradient of a portion of the second image that is common to the first and second images;

determining the maximum gradient of the portion of the second image based on the calculated gradient of the portion of the second image; and

generating the first estimate based on the calculated maximum gradient of the portion of the first image and the calculated maximum gradient of the portion of the second image.

9. The method of claim 1, wherein generating the first estimate using depth-from-defocus includes:

calculating a minimum edge width of a portion of the first image that is common to the first and second images;

calculating a minimum edge width of a portion of the second image that is common to the first and second images; and

generating the first estimate based on the calculated minimum edge width of the portion of the first image and the calculated minimum edge width of the portion of the second image.

11. The method of claim 1, wherein the width of blur for the first, second, third, and fourth images are determined by at least a distance in pixels and the corresponding edges are identified based on a change of pixel values and a direction of the change at the corresponding edges.

10. A camera, comprising:

a lens that is arranged to focus light;

a sensor that is arranged to receive light and to provide images; and

a controller that is arranged to receive the image, and configured to perform actions, including;

receiving, from the sensor, a first image of an object employing a first set of parameters;

receiving, from the sensor, a second image of the object employing a second set of parameters;

determining a width of blur at corresponding edges for each of the first and second images;

generating a first estimate of a distance from the camera to the object such that the first estimate is obtained from at least the first image and the second image using depth-from-defocus based at least on an equality comparison of the width of blur determined for each of the first image and the second image;

receiving, from the sensor, a third image of the object employing a third set of parameters, wherein the third set of parameters are based, at least in part, on the first estimate;

receiving, from the sensor, a fourth image of the object employing a fourth set of parameters, wherein the fourth set of parameters are based, at least in part, on the first estimate; and

determining a width of blur at corresponding edges for each of the third and fourth images;

generating a second estimate of the distance from the camera to the object such that the second estimate is obtained from at least the third image and the fourth image using depth-from-defocus based at least on an equality comparison of the width of blur determined for each of the third image and the fourth image.

12. The camera of claim 10, wherein the first, second, third, and fourth, set of parameters each include at least one of a focus position, an aperture, or a zoom.

13. The camera of claim 10, wherein the controller is further configured such that performing the automatic focusing further includes:

obtaining a fifth image of the object employing a fifth set of parameters, wherein the fifth set of parameters are based, at least in part, on the second estimate;

obtaining a sixth image of the object employing a sixth set of parameters, wherein the sixth set of parameters are based, at least in part, on the second estimate; and

generate a third estimate of the distance from the camera to the object such that the third estimate is obtained from at least the fifth image and the sixth image using depth-from-defocus.

14. The camera of claim 10, wherein the controller is further configured such that generating the first estimate using depth-from-defocus includes:

calculating a gradient of a portion of the first image that is common, to the first and second images;

determining the maximum gradient of the portion of the first image based on the calculated gradient of the portion of the first image;

calculating a gradient of a portion of the second image that is common to the first and second images;

determining the maximum gradient of the portion of the second image based on the calculated gradient of the portion of the second image; and

generating the first estimate based on the calculated maximum gradient of the portion of the first image and the calculated maximum gradient of the portion of the second image.

15. The camera of claim 10, wherein

the controller is further configured such that generating the first estimate using depth-from-defocus includes:

calculating a minimum edge width of a portion of the first image that is common to the first and second images;

calculating a minimum edge width of a portion of the second image that is common to the first and second images; and

generating the first estimate based on the calculated minimum edge width of the portion of the first image and the calculated minimum edge width of the portion of the second image.

16. The camera of claim 10 further including at least one motor to move the lens and wherein the processor causes the motor to move the lens to auto focus the camera based at least in part on estimate calculated by the processor.

17. The camera of claim 10, wherein the width of blur for the first, second, third, and fourth images are determined by at least a distance in pixels and the corresponding edges are identified based on a change of pixel values and a direction of the change at the corresponding edges.

18. An article of manufacture including a processor-readable non-transitory medium having processor-executable code stored therein, which when executed by one or more processors, enables actions for focusing, comprising:

calculating a distance from a camera to an object, wherein calculating the distance includes;

obtaining a first image of an object employing a first set of parameters;

obtaining a second image of the object employing a second set of parameters;

determining a width of blur at corresponding edges for each of the first and second images;

generating a first estimate of a distance from the camera to the object such that the first estimate is obtained from at least the first image and the second image using depth-from-defocus based at least on an equality comparison of the width of blur determined for each of the first image and the second image;

obtaining a third image of the object employing a third set of parameters, wherein the third set of parameters are based, at least in part, on the first estimate;

obtaining a fourth image of the object employing a fourth set of parameters, wherein the fourth set of parameters are based, at least in part, on the first estimate; and

determining a width of blur at corresponding edges for each of the third and fourth images;

generating a second estimate of the distance from the camera to the object such that the second estimate is obtained from at least the third image and the fourth image using depth-from-defocus based at least on an equality comparison of the width of blur determined for each of the third image and the fourth image.

19. The article of manufacture of claim 18, wherein

the first, second, third, and fourth set of parameters each include at least one of a focus position, an aperture, or a zoom.

20. The article of manufacture of claim 18, wherein

generating the first estimate using depth-from-defocus includes:

calculating a gradient of a portion of the first image that is common to the first and second images;

determining the maximum gradient of the portion of the first image based on the calculated gradient of the portion of the first image;

calculating a gradient of a portion of the second image that is common to the first and second images;

determining the maximum gradient of the portion of the second image based on the calculated gradient of the portion of the second image; and

generating the first estimate based on the calculated maximum gradient of the portion of the first image and the calculated maximum gradient of the portion of the second image.

21. The article of manufacture of claim 18, wherein

generating the first estimate using depth-from-defocus includes:

calculating a minimum edge width of a portion of the first image that is common to the first and second images;

calculating a minimum edge width of a portion of the second image that is common to the first and second images; and

generating the first estimate based on the calculated minimum edge width of the portion of the first image and the calculated minimum edge width of the portion of the second image.

22. The article of manufacture of claim 18, wherein the width of blur for the first, second, third, and fourth images are determined by at least a distance in pixels and the corresponding edges are identified based on a change of pixel values and a direction of the change at the corresponding edges.