Method for progressively determining depth from defocused images

• US 8,644,697 B1
• Filed: 07/14/2011
• Issued: 02/04/2014
• Est. Priority Date: 08/13/2010
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: CSR Technology Incorporated (Qualcomm, Inc.)
• Inventors: Tzur, Meir, Rapaport, Guy
• Primary Examiner(s): Mahoney, Christopher
• Application Number:
  US13/183,363
• Time in Patent Office: 936 Days
• Field of Search: 396/89, 396/125, 348/345, 359/698
• US Class Current: 396/89
• CPC Class Codes: G06T 7/529 from texture
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method

Abstract

A system, method, and computer program product are provided for automatically progressively determining focus depth estimates for an imaging device from defocused images. After a depth-from-defocus (DFD) system generates sometimes-noisy estimates for focus depth and optionally a confidence level that the focus depth estimate is correct, embodiments of the present invention process a sequence of such input DFD measures to iteratively decrease the likelihood of focus depth ambiguity and to increase an overall focus depth estimate confidence level. Automatic focus systems for imaging devices may use the outputs of the embodiments to operate more quickly and accurately, either directly or in combination with other focus depth estimation methods, such as calculated sharpness measures. A depth map of a 3D scene may be estimated for creating a pair of images based on a single image.

Claims

1. A method for improving depth-from-defocus measurements for an imaging device, comprising:

receiving a sequence of input depth-from-defocus measurements, wherein each measurement comprising at least a depth estimation, an ambiguity indication, and a confidence level;

combining a current measurement with at least one previous measurement based on at least the current measurement'"'"'s ambiguity indication and the at least one previous measurement'"'"'s ambiguity indication; and

if an improved depth estimation, a reduced ambiguity indication, and a confidence level exceeding a threshold are calculated, responsively performing at least one of (a) calculating, storing, and outputting the improved depth estimation, (b) repeating the method with another available input measurement, and (c) ending the method.

2. The method of claim 1 wherein if an ambiguity indication is not provided for at least one input measurement, the at least one input measurement'"'"'s ambiguity indication is set to non-ambiguous.

3. The method of claim 1 wherein if a confidence level is not provided for at least one input measurement, the confidence level is set to be equal for all input measurements.

4. The method of claim 1 wherein each input measurement includes at least a depth estimation, an ambiguity indication, and a confidence level, from a non-depth-from defocus estimator.

5. The method of claim 1, wherein a user determines when to end the method.

6. The method of claim 1 further comprising controlling an automatic focusing system with the improved depth estimation.

7. The method of claim 1 further comprising estimating a depth map with the improved depth estimation.

8. The method of claim 1 wherein at least one input measurement is from at least one of an image and an image portion.

9. The method of claim 1 wherein the previous measurement is from an immediately preceding iteration.

10. The method of claim 1 wherein calculating the improved depth estimation includes using a non-depth-from-defocus depth estimation based on a sharpness metric.

11. The method of claim 1 wherein the calculating comprises, if either the current measurement or the previous measurement is non-ambiguous, then the improved depth estimation corresponds with the non-ambiguous measurement.

12. The method of claim 1 wherein the calculating comprises, if the current measurement and the previous measurement are both ambiguous, then:

find two nearest depth estimations from the current measurement and the previous measurement;

compare the two nearest depth estimations with a predefined threshold, and if an absolute difference between the two nearest depth estimations is less than the predefined threshold then the improved depth estimation corresponds with a mean of the two nearest depth estimations, the confidence level exceeding the threshold corresponds with a mean of the two nearest depth estimations corresponding confidence levels, and set the reduced ambiguity indication to non-ambiguous, else use all the input measurements'"'"' depth estimations to determine the improved depth estimation.

13. The method of claim wherein the calculating further comprises, if the current measurement and the previous measurement are both non-ambiguous, then:

compare the current depth measurement and the previous depth measurement with a predefined threshold, and if an absolute difference between the previous and current depth measurements is larger than the predefined threshold and each confidence level of the current and previous depth measurements are larger than another threshold, then end the method, else the improved depth estimation corresponds with that measurement having a highest confidence level.

14. A system for improving depth-from-defocus measurements for an imaging device, comprising a decision engine that performs a system operation that:

receives a sequence of input depth-from-defocus measurements, wherein each measurement comprising at least a depth estimation, an ambiguity indication, and a confidence level;

combines a current measurement with at least one previous measurement based on at least the current measurement'"'"'s ambiguity indication and the at least one previous measurement'"'"'s ambiguity indication; and

if an improved depth estimation, a reduced ambiguity indication, and a confidence level exceeding a threshold are calculated, responsively performing at least one of (a) calculates, stores, and outputs the improved depth estimation, (b) repeats the system operation with another available input measurement, and (c) ends the system operation.

15. The system of claim 14 wherein if an ambiguity indication is not provided for at least one input measurement the at least one input measurement'"'"'s ambiguity indication is set to non-ambiguous.

16. The system of claim 14 wherein if a confidence level is not provided for at least one input measurement, the confidence level is set to be equal for all input measurements.

17. The system of claim 14 wherein each input measurement includes at least a depth estimation, an ambiguity indication, and a confidence level, from a non-depth-from defocus estimator.

18. The system of claim 14, wherein a user determines when to end the system operation.

19. The system of claim 14 further comprising controlling an automatic focusing system with the improved depth estimation.

20. The system of claim 14 further comprising estimating a depth map with the improved depth estimation.

21. The system of claim 14 wherein at least one input measurement is from at least one of an image and an image portion.

22. The system of claim 14 wherein the previous measurement is from an immediately preceding iteration.

23. The system of claim 14 wherein calculating the improved depth estimation includes using a non-depth-from-defocus depth estimation based on a sharpness metric.

24. The system of claim 14 wherein the calculating comprises, if either the current measurement or the previous measurement is non-ambiguous, then the improved depth estimation corresponds with the non-ambiguous measurement.

25. The system of claim 14 wherein the calculating further comprises, if the current measurement and the previous measurement are both ambiguous, then:

find two nearest depth estimations from the current measurement and the previous measurement;

compare the two nearest depth estimations with a predefined threshold, and if an absolute difference between the two nearest depth estimations is less than the predefined threshold then the improved depth estimation corresponds with a mean of the two nearest depth estimations, the confidence level exceeding the threshold corresponds with a mean of the two nearest depth estimations corresponding confidence levels, and set the reduced ambiguity indication to non-ambiguous, else use all the input measurements'"'"' depth estimations to determine the improved depth estimation.

26. The system of claim 14 wherein the calculating further comprises, if the current measurement and the previous measurement are both non-ambiguous, then:

compare the current depth measurement and the previous depth measurement with a predefined threshold, and if an absolute difference between the previous and current depth measurements is larger than the predefined threshold and each confidence level of the current and previous depth measurements are larger than another threshold, then end the system operation, else the improved depth estimation corresponds with that measurement having a highest confidence level.

27. A computer program product comprising a machine-readable medium tangibly embodying non-transitory program instructions thereon that, an execution by a computer, causes the computer to:

receive a sequence of input depth-from-defocus measurements, wherein each measurement comprising at least a depth estimation, an ambiguity indication, and a confidence level;

combine a current measurement with at least one previous measurement based on at least the current measurement'"'"'s ambiguity indication and the at least one previous measurement'"'"'s ambiguity indication; and

if an improved depth estimation, a reduced ambiguity indication, and a confidence level exceeding a threshold are calculated, responsively performing at least one of (a) calculate, store, and output the improved depth estimation, (b) repeat the execution with another available input measurement, and (c) end the execution.

28. A system comprising means for a system operation comprising:

receiving a sequence of input depth-from-defocus measurements, wherein each measurement comprising at least a depth estimation, an ambiguity indication, and a confidence level;

combining a current measurement with at least one previous measurement based on at least the current measurement'"'"'s ambiguity indication and the at least one previous measurement'"'"'s ambiguity indication; and

if an improved depth estimation, a reduced ambiguity indication, and a confidence level exceeding a threshold are calculated, responsively performing at least one of (a) calculating, storing, and outputting the improved depth estimation, (b) repeating the system operation with another available input measurement, and (c) ending the system operation.