Depth estimation based on interpolation of inverse focus statistics

• US 9,215,357 B2
• Filed: 10/24/2014
• Issued: 12/15/2015
• Est. Priority Date: 11/13/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm Technologies Incorporated (Qualcomm, Inc.)
• Inventors: Tzur, Meir
• Primary Examiner(s): FOSSELMAN, JOEL W
• Application Number:
  US14/523,648
• Publication Number:
  US 20150042841A1
• Time in Patent Office: 417 Days
• Field of Search: 348/345, 348/348
• US Class Current: 1/1
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

Embodiments are directed towards performing depth estimation within a digital camera system based on interpolation of inverse focus statistics. After an image is captured, various statistics or focus measure may be calculated using, for example, a high pass filter. Depth is estimated by interpolating the inverse of the statistics for three positions of focus for the image. The inverse of the statistics, St(n), may be 1/St(n), or 1/St²(n), or even 1/St^Z(n), where Z≧1. Several approaches to interpolating the inverse values of the statistics to obtain a depth estimate are disclosed, including a general parabolic minimum approach, using a parabolic minimum within a progressive scheme, or within a continuous AF scheme. The depth estimate may then be used for a variety of applications, including automatic focusing, as well as converting 2D images to 3D images.

Claims

1. A method for estimating a depth for a plurality of image frames, comprising:

receiving image data corresponding to the plurality of the image frames based on varying a camera focus position to a plurality of predefined focusing positions;

selecting a subset of the plurality of the predefined focusing positions that are noncontiguous;

calculating statistics based at least in part on the image data at the subset of the plurality of the predefined focusing positions; and

estimating the depth based at least in part on a minimum of inverse values of the calculated statistics.

2. The method of claim 1, wherein estimating the depth comprises utilizing a subset of the calculated statistics to estimate the depth.

3. The method of claim 1, wherein estimating the depth comprises determining interpolated inverse values of corresponding calculated statistics and estimating the depth based at least in part on the interpolated inverse values.

4. The method of claim 1, wherein estimating the depth comprises performing a curve fitting determination via calculating the minimum location on a curve based at least in part on the inverse values of corresponding calculated statistics.

5. The method of claim 4, wherein estimating the depth further comprises selectively utilizing the minimum as the estimated depth.

6. The method of claim 1, further comprising accumulating the calculated statistics in a buffer or a storage location.

21. The method of claim 1, further comprising capturing, via an image sensor, image data corresponding to the plurality of image frames.

7. A device for estimating a depth for a plurality of image frames, comprising:

a processor configured to receive image data corresponding to the plurality of the image frames based on varying a camera focus position to a plurality of predefined focusing positions;

the processor configured to select a subset of the plurality of the predefined focusing positions that are noncontiguous;

the processor configured to calculate statistics based at least in part on the image data at the subset of the plurality of the predefined focusing positions; and

the processor configured to estimate the depth based at least in part on a minimum of inverse values of the calculated statistics.

8. The device of claim 7, wherein the processor is configured to estimate the depth based at least in part on utilizing a subset of the calculated statistics to estimate the depth.

9. The device of claim 7, wherein the processor is configured to estimate the depth based at least in part on determining interpolated inverse values of corresponding calculated statistics and estimating the depth based at least in part on the interpolated inverse values.

10. The device of claim 7, wherein the processor is configured to estimate the depth based at least in part on performing a curve fitting determination via calculating the minimum on a curve based at least in part on the inverse values of corresponding calculated statistics.

11. The device of claim 10, wherein the processor is configured to estimate the depth based at least in part on selectively utilizing the minimum as the estimated depth.

12. The device of claim 7, wherein the processor is configured to accumulate the calculated statistics in a buffer or a storage location.

22. The device of claim 7, further comprising an image sensor configured to capture image data corresponding to the plurality of image frames.

13. An apparatus for estimating a depth for a plurality of image frames, comprising:

means for receiving image data corresponding to the plurality of the image frames based on varying a camera focus position to a plurality of predefined focusing positions;

means for selecting a subset of the plurality of the predefined focusing positions that are noncontiguous;

means for calculating statistics based at least in part on the image data at the subset of the plurality of the predefined focusing positions; and

means for estimating the depth based at least in part on a minimum of inverse values of the calculated statistics.

14. The apparatus of claim 13, wherein the means for estimating the depth comprises means for utilizing a subset of the calculated statistics to estimate the depth.

15. The apparatus of claim 13, wherein the means for estimating the depth comprises means for determining interpolated inverse values of corresponding calculated statistics and means for estimating the depth based at least in part on the interpolated inverse values.

16. The apparatus of claim 13, wherein the means for estimating the depth comprises means for performing a curve fitting determination via calculating the minimum on a curve based at least in part on the inverse values of corresponding calculated statistics.

17. The apparatus of claim 16, wherein the means for estimating the depth further comprise means for selectively utilizing the minimum as the estimated depth.

18. The apparatus of claim 13, further comprising means for accumulating the calculated statistics.

23. The apparatus of claim 13, further comprising means for capturing image data corresponding to the plurality of image frames.

19. A non-transitory computer readable medium comprising code that, when executed, causes an apparatus to perform a process comprising:

receiving image data corresponding to a plurality of image frames based on varying a camera focus position to a plurality of predefined focusing positions;

selecting a subset of the plurality of the predefined focusing positions that are noncontiguous;

calculating statistics based at least in part on the image data at the subset of the plurality of the predefined focusing positions; and

estimating the depth based at least in part on a minimum of inverse values of the calculated statistics.

20. The non-transitory computer readable medium of claim 19, further comprising code that, when executed, causes the apparatus to estimate the depth via utilizing a subset of the calculated statistics to estimate the depth.

24. The non-transitory computer readable medium of claim 19, comprising code that, when executed, causes the apparatus to capture, via an image sensor, image data corresponding to the plurality of image frames.