IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND NON-TRANSITORY COMPUTER READABLE RECORDING MEDIUM

• US 20140125844A1
• Filed: 04/09/2012
• Published: 05/08/2014
• Est. Priority Date: 06/27/2011
• Status: Active Grant

• Current Assignee: Konica Minolta Inc.
• Original Assignee: Konica Minolta Inc.
• Inventors: Sumitomo, Hironori
• Granted Patent: US 9,042,655 B2
• Time in Patent Office: Days
• US Class Current: 348/231.99
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 5/907 using static stores, e.g. storage tubes or semiconductor memories H04N5/91 takes precedence

Abstract

A converted reference image generated from a reference image is searched for a first corresponding point corresponding to a first standard point included in a converted standard image generated from a standard image. Based on a position of the first corresponding point in the converted reference image, a second search standard point is determined on the reference image. Further, based on phase information on each frequency component regarding a second standard area including a second standard point corresponding to the first standard point of the standard image, and based on phase information on each frequency component regarding a second reference area including the second search standard point of the reference image, the reference image is searched for a second corresponding point corresponding to the second standard point. The information on a frequency component obtained in the computation on another stage is reused in the search on the two stages above.

Claims

1. An image processing apparatus, comprising:

an acquisition section that obtains a first image and a second image;

a generation section that, in accordance with a predetermined conversion factor, reduces a level of resolution of the first image to generate a first converted standard image and reduces a level of resolution of the second image to generate a first converted reference image;

a search section that performs;

a first search process of searching the first converted reference image for a first corresponding point that corresponds to a first standard point based on first phase information on each frequency component of first frequency information regarding a first standard area including the first standard point as a standard in the first converted standard image and based on second phase information on each frequency component of second frequency information regarding a first reference area including a first search standard point as a standard in the first converted reference image; and

a second search process of searching the second image for a second corresponding point that corresponds to a second standard point based on third phase information on each frequency component of third frequency information regarding a second standard area including the second standard point corresponding to the first standard point as a standard in the first image and based on fourth phase information on each frequency component of fourth frequency information regarding a second reference area including a second search standard point as a standard in the second image;

a determination section that determines a position of the second search standard point in the second image based on a position of the first corresponding point in the first converted reference image; and

a storage section that stores computed information regarding at least part of the first frequency information and the second frequency information obtained through the computation in the first search process and the third frequency information obtained through the computation in the second search process,

wherein the search section performs at least one of;

in the second search process, the process of using the computed information obtained through the computation in the first search process as part of information for computations regarding the third frequency information and the fourth frequency information to calculate the other of the information for computations regarding the third frequency information and the fourth frequency information based on the first and second images; and

in the first search process, the process of using the computed information obtained through the computation in the second search process as part of information for computation regarding the first frequency information to calculate the other of the information for computation regarding the first frequency information in based on the first converted standard image.

3. The image processing apparatus according to claim 1, wherein the search section uses a phase-only correlation computation in the first and second search processes.

4. The image processing apparatus according to claim 1, wherein the computed information includes at least part of the first frequency information to the third frequency information.

5. The image processing apparatus according to claim 1, wherein the computed information includes at least part of the first phase information to the third phase information.

6. The image processing apparatus according to claim 1, wherein the computed information includes information indicative of a phase difference calculated based on the first phase information and the second phase information.

7. The image processing apparatus according to claim 1, wherein

in the first search process, for each frequency component, the search section calculates a first phase difference based on the first phase information and the second phase information to convert the first phase difference into a first spatial deviation amount, and votes the first spatial deviation amount to a first voting space to obtain a first voting result, to thereby search for the first corresponding point in accordance with the first voting result,

in the second search process, for each frequency component, the search section calculates a second phase difference based on the third phase information and the fourth phase information to convert the second phase difference into a second spatial deviation amount, and votes the second spatial deviation amount to a second voting space to obtain a second voting result, to thereby search for the second corresponding point in accordance with the second voting result, and

the computed information includes information indicative of a voting result obtained by converting the first phase difference of one or more frequency components into a spatial deviation amount and then voting the spatial deviation amount to the first voting space in the first search process.

8. The image processing apparatus according to claim 1, wherein

in the second search process, the search section corrects the first phase information by a deviation amount of a phase corresponding to a reduction in the level of resolution and uses the corrected first phase information in computation upon obtaining the first phase information from the computed information calculated in the first search process and using the first phase information in computation, and corrects the second phase information by a deviation amount of a phase corresponding to a reduction in the level of resolution and uses the corrected second phase information in computation upon obtaining the second phase information from the computed information calculated in the first search process and using the second phase information in computation, and

in the first search process, the search section corrects the third phase information by a deviation amount of a phase corresponding to a reduction in the level of resolution and uses the corrected third phase information in computation upon obtaining the third phase information from the computed information calculated in the second search process and using the third phase information in computation.

9. The image processing apparatus according to claim 1, wherein

the predetermined conversion factor is 1/n where n is any number larger than one,

in the second search process, the search section uses the first frequency information regarding a first frequency as the third frequency information regarding a second frequency obtained by multiplying the first frequency by 1/n upon obtaining the first frequency information regarding the first frequency from the computed information calculated in the first search process and using the first frequency information in computation, and uses the second frequency information regarding the first frequency as the fourth frequency information regarding the second frequency upon obtaining the second frequency information regarding the first frequency from the computed information calculated in the first search process and using the second frequency information in computation, and

in the first search process, the search section uses the third frequency information regarding the second frequency as the first frequency information regarding the first frequency obtained by multiplying the second frequency by n upon obtaining the third frequency information regarding the second frequency from the computed information calculated in the second search process and using the third frequency information in computation.

10. The image processing apparatus according to claim 1, wherein in the second search process, the search section corrects a deviation of a phase of the second phase information in accordance with a spatial deviation amount between the first search standard point and the first corresponding point and uses the corrected second phase information in computation upon obtaining the second phase information from the computed information calculated in the first search process and using the second phase information in computation.

11. The image processing apparatus according to claim 1, wherein in the second search process, the search section performs weighting of information such that the computed information used in computation as part of information for computations regarding the third frequency information and the fourth frequency information less affects the search results on the second corresponding point than the other of the information for computations.

2. An image processing apparatus, comprising:

an acquisition section that obtains a first image and a second image;

a generation section that, in accordance with a predetermined conversion factor, reduces a level of resolution of the first image to generate a first converted standard image and a second converted standard image having a lower level of resolution than that of the first converted standard image and reduces a level of resolution of the second image to generate a first converted reference image and a second converted reference image having a lower level of resolution than that of the first converted reference image;

a search section that performs;

a first search process of searching the second converted reference image for a first corresponding point that corresponds to a first standard point based on first phase information on each frequency component of first frequency information regarding a first standard area including the first standard point as a standard in the second converted standard image and based on second phase information on each frequency component of second frequency information regarding a first reference area including a first search standard point as a standard in the second converted reference image; and

a second search process of searching the first converted reference image for a second corresponding point that corresponds to a second standard point corresponding to the first standard point based on third phase information on each frequency component of third frequency information regarding a second standard area including the second standard point as a standard in the first converted standard image and based on fourth phase information on each frequency component of fourth frequency information regarding a second reference area including a second search standard point as a standard in the first converted reference image;

a determination section that determines a position of the second search standard point in the first converted reference image based on a position of the first corresponding point in the second converted reference image; and

a storage section that stores computed information regarding at least part of the first frequency information and the second frequency information obtained through the computation in the first search process and the third frequency information obtained through the computation in the second search process,

wherein the search section performs at least one of;

in the second search process, the process of using the computed information obtained through the computation in the first search process as part of information for computations regarding the third frequency information and the fourth frequency information to calculate the other of the information for computations regarding the third frequency information and the fourth frequency information based on the first converted standard image and the first converted reference image; and

in the first search process, the process of using the computed information obtained through the computation in the second search process as part of information for computation regarding the first frequency information to calculate the other of the information for computation regarding the first frequency information based on the second converted standard image.

17. The image processing apparatus according to claim 2, wherein the search section uses a phase-only correlation computation in the first and second search processes.

18. The image processing apparatus according to claim 2, wherein the computed information includes at least part of the first frequency information to the third frequency information.

19. The image processing apparatus according to claim 2, wherein the computed information includes at least part of the first phase information to the third phase information.

20. The image processing apparatus according to claim 2, wherein the computed information includes information indicative of a phase difference calculated based on the first phase information and the second phase information.

21. The image processing apparatus according to claim 2, wherein

in the first search process, for each frequency component, the search section calculates a first phase difference based on the first phase information and the second phase information to convert the first phase difference into a first spatial deviation amount, and votes the first spatial deviation amount to a first voting space to obtain a first voting result, to thereby search for the first corresponding point in accordance with the first voting result,

in the second search process, for each frequency component, the search section calculates a second phase difference based on the third phase information and the fourth phase information to convert the second phase difference into a second spatial deviation amount, and votes the second spatial deviation amount to a second voting space to obtain a second voting result, to thereby search for the second corresponding point in accordance with the second voting result, and

the computed information includes information indicative of a voting result obtained by converting the first phase difference of one or more frequency components into a spatial deviation amount and then voting the spatial deviation amount to the first voting space in the first search process.

22. The image processing apparatus according to claim 2, wherein

in the second search process, the search section corrects the first phase information by a deviation amount of a phase corresponding to a reduction in the level of resolution and uses the corrected first phase information in computation upon obtaining the first phase information from the computed information calculated in the first search process and using the first phase information in computation, and corrects the second phase information by a deviation amount of a phase corresponding to a reduction in the level of resolution and uses the corrected second phase information in computation upon obtaining the second phase information from the computed information calculated in the first search process and using the second phase information in computation, and

in the first search process, the search section corrects the third phase information by a deviation amount of a phase corresponding to a reduction in the level of resolution and uses the corrected third phase information in computation upon obtaining the third phase information from the computed information calculated in the second search process and using the third phase information in computation.

23. The image processing apparatus according to claim 2, wherein

the predetermined conversion factor is 1/n where n is any number larger than one,

in the second search process, the search section uses the first frequency information regarding a first frequency as the third frequency information regarding a second frequency obtained by multiplying the first frequency by 1/n upon obtaining the first frequency information regarding the first frequency from the computed information calculated in the first search process and using the first frequency information in computation, and uses the second frequency information regarding the first frequency as the fourth frequency information regarding the second frequency upon obtaining the second frequency information regarding the first frequency from the computed information calculated in the first search process and using the second frequency information in computation, and

in the first search process, the search section uses the third frequency information regarding the second frequency as the first frequency information regarding the first frequency obtained by multiplying the second frequency by n upon obtaining the third frequency information regarding the second frequency from the computed information calculated in the second search process and using the third frequency information in computation.

24. The image processing apparatus according to claim 2, wherein in the second search process, the search section corrects a deviation of a phase of the second phase information in accordance with a spatial deviation amount between the first search standard point and the first corresponding point and uses the corrected second phase information in computation upon obtaining the second phase information from the computed information calculated in the first search process and using the second phase information in computation.

25. The image processing apparatus according to claim 2, wherein in the second search process, the search section performs weighting of information such that the computed information used in computation as part of information for computations regarding the third frequency information and the fourth frequency information less affects the search results on the second corresponding point than the other of the information for computations.

12. An image processing method, comprising the steps of:

(a) by an acquisition section, obtaining a first image;

(b) by the acquisition section, obtaining a second image;

(c) by a generation section, in accordance with a predetermined conversion factor, reducing a level of resolution of the first image to generate a first converted standard image and reducing a level of resolution of the second image to generate a first converted reference image;

(d) by a search section, performing a first search process of searching the first converted reference image for a first corresponding point that corresponds to a first standard point based on first phase information on each frequency component of first frequency information regarding a first standard area including the first standard point as a standard in the first converted standard image and based on second phase information on each frequency component of second frequency information regarding a first reference area including a first search standard point as a standard in the first converted reference image, and by a storage section, storing computed information regarding at least part of the first frequency information and the second frequency information obtained through the computation in the first search process;

(e) by a determination section, determining a position of a second search standard point in the second image based on a position of the first corresponding point in the first converted reference image; and

(f) by the search section, performing a second search process of searching the second image for a second corresponding point that corresponds to a second standard point corresponding to the first standard point based on third phase information on each frequency component of third frequency information regarding a second standard area including the second standard point as a standard in the first image and based on fourth phase information on each frequency component of fourth frequency information regarding a second reference area including the second search standard point as a standard in the second image,

wherein in the step (f), the search section uses the computed information as part of information for computations regarding the third frequency information and the fourth frequency information to calculate the other of the information for computations based on the first and second images.

13. An image processing method, comprising the steps of:

(A) by an acquisition section, obtaining a first image;

(B) by the acquisition section, obtaining a second image;

(C) by a generation section, in accordance with a predetermined conversion factor, reducing a level of resolution of the first image to generate a first converted standard image and a second converted standard image having a lower level of resolution than that of the first converted standard image, and reducing a level of resolution of the second image to generate a first converted reference image and a second converted reference image having a lower level of resolution than that of the first converted reference image;

(D) by a search section, performing a first search process of searching the second converted reference image for a first corresponding point that corresponds to a first standard point based on first phase information on each frequency component of first frequency information regarding a first standard area including the first standard point as a standard in the second converted standard image and based on second phase information on each frequency component of second frequency information regarding a first reference area including a first search standard point as a standard in the second converted reference image, and by a storage section, storing computed information regarding at least part of the first frequency information and the second frequency information obtained through the computation in the first search process;

(E) by a determination section, determining a position of a second search standard point in the first converted reference image based on a position of the first corresponding point in the second converted reference image; and

(F) by the search section, performing a second search process of searching the first converted reference image for a second corresponding point that corresponds to a second standard point corresponding to the first standard point based on third phase information on each frequency component of third frequency information regarding a second standard area including the second standard point as a standard in the first converted standard image and based on fourth phase information on each frequency component of fourth frequency information regarding a second reference area including the second search standard point as a standard in the first converted reference image,

wherein in the step (F), the search section uses the computed information as part of information for computations regarding the third frequency information and the fourth frequency information to calculate the other of the information for computations based on the first converted standard image and the first converted reference image.

14. (canceled)

15. A non-transitory computer readable recording medium storing a computer-readable program, the program controlling an image processing apparatus to operate as one image processing apparatus, and the one image processing apparatus comprising:

an acquisition section that obtains a first image and a second image;

a generation section that, in accordance with a predetermined conversion factor, reduces a level of resolution of the first image to generate a first converted standard image and reduces a level of resolution of the second image to generate a first converted reference image;

a search section that performs;

a first search process of searching the first converted reference image for a first corresponding point that corresponds to a first standard point based on first phase information on each frequency component of first frequency information regarding a first standard area including the first standard point as a standard in the first converted standard image and based on second phase information on each frequency component of second frequency information regarding a first reference area including a first search standard point as a standard in the first converted reference image; and

a second search process of searching the second image for a second corresponding point that corresponds to a second standard point based on third phase information on each frequency component of third frequency information regarding a second standard area including the second standard point corresponding to the first standard point as a standard in the first image and based on fourth phase information on each frequency component of fourth frequency information regarding a second reference area including a second search standard point as a standard in the second image;

a determination section that determines a position of the second search standard point in the second image based on a position of the first corresponding point in the first converted reference image; and

a storage section that stores computed information regarding at least part of the first frequency information and the second frequency information obtained through the computation in the first search process and the third frequency information obtained through the computation in the second search process,

wherein the search section performs at least one of;

in the second search process, the process of using the computed information obtained through the computation in the first search process as part of information for computations regarding the third frequency information and the fourth frequency information to calculate the other of the information for computations regarding the third frequency information and the fourth frequency information based on the first and second images; and

in the first search process, the process of using the computed information obtained through the computation in the second search process as part of information for computation regarding the first frequency information to calculate the other of the information for computation regarding the first frequency information based on the first converted standard image.

16. A non-transitory computer readable recording medium storing a computer-readable program, the program controlling an image processing apparatus to operate as one image processing apparatus, and the one image processing apparatus comprising:

an acquisition section that obtains a first image and a second image;

a generation section that, in accordance with a predetermined conversion factor, reduces a level of resolution of the first image to generate a first converted standard image and a second converted standard image having a lower level of resolution than that of the first converted standard image and reduces a level of resolution of the second image to generate a first converted reference image and a second converted reference image having a lower level of resolution than that of the first converted reference image;

a search section that performs;

a first search process of searching the second converted reference image for a first corresponding point that corresponds to a first standard point based on first phase information on each frequency component of first frequency information regarding a first standard area including the first standard point as a standard in the second converted standard image and based on second phase information on each frequency component of second frequency information regarding a first reference area including a first search standard point as a standard in the second converted reference image; and

a second search process of searching the first converted reference image for a second corresponding point that corresponds to a second standard point corresponding to the first standard point based on third phase information on each frequency component of third frequency information regarding a second standard area including the second standard point as a standard in the first converted standard image and based on fourth phase information on each frequency component of fourth frequency information regarding a second reference area including a second search standard point as a standard in the first converted reference image;

a determination section that determines a position of the second search standard point in the first converted reference image based on a position of the first corresponding point in the second converted reference image; and

a storage section that stores computed information regarding at least part of the first frequency information and the second frequency information obtained through the computation in the first search process and the third frequency information obtained through the computation in the second search process,

wherein the search section performs at least one of;

in the second search process, the process of using the computed information obtained through the computation in the first search process as part of information for computations regarding the third frequency information and the fourth frequency information to calculate the other of the information for computations regarding the third frequency information and the fourth frequency information based on the first converted standard image and the first converted reference image; and

in the first search process, the process of using the computed information obtained through the computation in the second search process as part of information for computation regarding the first frequency information to calculate the other of the information for computation regarding the first frequency information based on the second converted standard image.