DETERMINING A DEPTH MAP FROM IMAGES OF A SCENE

• US 20130266210A1
• Filed: 04/02/2013
• Published: 10/10/2013
• Est. Priority Date: 04/04/2012
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: MORGAN-MAR, DAVID PETER, LARKIN, KIERAN GERARD, ARNISON, MATTHEW RAPHAEL, FLETCHER, PETER ALLEINE, PHAM, TUAN QUANG
• Granted Patent: US 9,117,277 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 7/571 from focus

Abstract

Methods for determining a depth measurement of a scene which involve capturing at least two images of the scene with different camera parameters, and selecting corresponding image patches in each scene. A first approach calculates a plurality of complex responses for each image patch using a plurality of different quadrature filters, each complex response having a magnitude and a phase, assigns, for each quadrature filter, a weighting to the complex responses in the corresponding image patches, the weighting being determined by a relationship of the phases of the complex responses, and determines the depth measurement of the scene from a combination of the weighted complex responses.

Claims

1. A method of determining a depth measurement of a scene, the method comprising:

capturing at least two images of the scene with different camera parameters;

selecting corresponding image patches in each scene;

calculating a plurality of complex responses for each image patch using a plurality of different quadrature filters, each said complex response having a magnitude and a phase,

assigning, for each quadrature filter, a weighting to the complex responses in the corresponding image patches, said weighting being determined by a relationship of the phases of the complex responses; and

determining the depth measurement of the scene from a combination of the weighted complex responses.

2. A method according to claim 1, wherein the different quadrature filters are configured to operate in the spatial domain and at least one of radial frequency, orientation and bandwidth vary across the plurality of quadrature filters.

3. A method according to claim 2, wherein the quadrature filters are Gabor filters, each with orientation one of 0°

, 45°

, 90°

, 135°

, implemented separably using one-dimensional filters.

4. A method according to claim 1, where in the quadrature filters are one of log-Gabor filters, Gaussian derivative filters, difference-of-Gaussian filters, or Cauchy filters.

5. A method according to claim 1, where in the quadrature filters are the filters used in a complex wavelet transform such as the dual-tree complex wavelet transform.

6. A method according to claim 1, wherein the weighting for each pixel is assigned to be the value of a function of the phase of the pixel value.

7. A method according to claim 1, wherein the depth measurement is determined from the weighted pixel values by performing a weighted fit to a functional form with a free parameter, and the depth measurement is determined by the fitted value of the free parameter.

8. A method according to claim 7, wherein the functional form approximates the weighted pixel values of the combined function.

9. A method according to claim 1, wherein the phase of each spectral ratio is multiplied by 2 and wrapped, with this multiplied and wrapped phase then being used to determine the weightings.

10. A method according to claim 1, wherein a confidence measure of the depth measurement is also calculated from the weighted pixel values of the combined function.

11. A method of determining a shift estimation between two images comprising the steps of:

selecting corresponding image patches in each image;

calculating an output set of pixel values from a combined complex response of the image patches to a set of quadrature filters, each said pixel value in the output set of pixel values having a magnitude and a phase;

assigning a weighting to each of the pixel values in the output set of pixel values, said weighting being determined by the phase of at least some of the output set of pixel values and at least some of the pixel values being assigned a weighting different to the weighting assigned to other pixel values; and

determining the shift estimation between the two images from the weighted pixel values of the combined function.

12. A method according to claim 11, wherein the shift estimate is calculated by forming a weighted phase correlation.

13. A method according to claim 11, wherein the shift estimate is calculated by fitting a planar ramp to the phase of the pixels of the spectral ratio.

14. A computer readable storage medium having a program recorded thereon, the program being executable by computerized apparatus to determine a depth measurement of a scene, the program comprising:

code for capturing at least two images of the scene with different camera parameters;

code for selecting corresponding image patches in each scene;

code for calculating a plurality of complex responses for each image patch using a plurality of quadrature filters, each said complex response having a magnitude and a phase, with at least one of radial frequency, orientation and bandwidth varying across the plurality of quadrature filters;

code for assigning, for each quadrature filter, a weighting to the complex responses in the corresponding image patches, said weighting being determined by a relationship of the phases of the complex responses; and

code for determining the depth measurement of the scene from a combination of the weighted complex responses.

15. A computer readable storage medium having a program recorded thereon, the program being executable by computerized apparatus to determine a shift estimation between two images, the program comprising:

code for capturing at least two images of the scene with different camera parameters;

code for selecting corresponding image patches in each scene;

code for calculating a plurality of complex responses for each image patch using a plurality of quadrature filters, each said complex response having a magnitude and a phase, and at least one of radial frequency, orientation and bandwidth varying across the plurality of quadrature filters;

code for assigning, for each quadrature filter, a weighting to the complex responses in the corresponding image patches, said weighting being determined by a relationship of the phases of the complex responses; and

code for determining the depth measurement of the scene and the shift estimation between the two images of the scene from a weighted combination of the complex responses.

16. A method of determining a depth measurement of a scene comprising the steps of:

capturing at least two images of the scene with different camera parameters;

selecting corresponding image patches in each scene;

calculating an output set of pixel values from a combined complex-valued function of Fourier transforms of the image patches, each said pixel value in the output set of pixel values having a magnitude and a phase;

assigning a weighting to each of the pixel values in the output set of pixel values, said weighting being determined by the phase of at least some of the output set of pixel values and at least some of the pixel values being assigned a weighting different to the weighting assigned to other pixel values; and

determining the depth measurement of the scene from the weighted pixel values of the combined function.

17. A method according to claim 16, wherein the weighting for each pixel is assigned to be the value of a function of the phase of the pixel value.

18. A method according to claim 16, wherein each pixel is defined to have a set of neighbouring pixels and the weighting for each pixel is assigned to be the value of a function of the phase of the pixel values and of the phases of the neighbouring pixel values.

19. A method according to claim 16, wherein the depth measurement is determined from the weighted pixel values by performing a weighted fit to a functional form with a free parameter, and the depth measurement is determined by the fitted value of the free parameter.

20. A method according to claim 19, wherein the functional form approximates the weighted pixel values of the combined function.

21. A method according to claim 16, wherein the phase of each pixel value is multiplied by 2 and wrapped, with this multiplied and wrapped phase then being used to determine the weightings.

22. A method according to claim 16, wherein a confidence measure of the depth measurement is also calculated from the weighted pixel values of the combined function.

23. A method of determining a shift estimation between two images comprising the steps of:

selecting corresponding image patches in each image;

calculating an output set of pixel values from a combined complex-valued function of Fourier transforms of the image patches, each said pixel value in the output set of pixel values having a magnitude and a phase;

assigning a weighting to each of the pixel values in the output set of pixel values, said weighting being determined by the phase of at least some of the output set of pixel values and at least some of the pixel values being assigned a weighting different to the weighting assigned to other pixel values; and

determining the shift estimation between the two images from the weighted pixel values of the combined function.

24. A method according to claim 23, wherein the shift estimate is calculated by forming a weighted phase correlation.

25. A method according to claim 23, wherein the shift estimate is calculated by fitting a planar ramp to the phase of the pixels of the spectral ratio.

26. A computer readable storage medium having a program recorded thereon, the program being executable by computerized apparatus to determine a depth measurement of a scene, the program comprising:

code for obtaining at least two images of the scene captured with different camera parameters;

code for selecting corresponding image patches in each scene;

code for calculating an output set of pixel values from a combined complex-valued function of Fourier transforms of the image patches, each said pixel value in the output set of pixel values having a magnitude and a phase;

code for assigning a weighting to each of the pixel values in the output set of pixel values, said weighting being determined by the phase of at least some of the output set of pixel values and at least some of the pixel values being assigned a weighting different to the weighting assigned to other pixel values; and

code for determining the depth measurement of the scene from the weighted pixel values of the combined function.

27. A computer readable storage medium having a program recorded thereon, the program being executable by computerized apparatus to determine a shift estimation between two images, the program comprising:

code for selecting corresponding image patches in each image;

code for calculating an output set of pixel values from a combined complex-valued function of Fourier transforms of the image patches, each said pixel value in the output set of pixel values having a magnitude and a phase;

code for assigning a weighting to each of the pixel values in the output set of pixel values, said weighting being determined by the phase of at least some of the output set of pixel values and at least some of the pixel values being assigned a weighting different to the weighting assigned to other pixel values; and

code for determining the shift estimation between the two images from the weighted pixel values of the combined function.