Compound eye imaging apparatus, distance measuring apparatus, disparity calculation method, and distance measuring method

• US 8,090,195 B2
• Filed: 02/10/2009
• Issued: 01/03/2012
• Est. Priority Date: 02/12/2008
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Oyama, Ichiro
• Primary Examiner(s): Seth, Manav
• Application Number:
  US12/594,975
• Publication Number:
  US 20100150455A1
• Time in Patent Office: 1,057 Days
• Field of Search: 382100-317, 348 42- 60, 348180-194
• US Class Current: 382/154
• CPC Class Codes: G01C 3/085 with electronic parallax measurement
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 7/596 from three or more stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

A compound eye imaging apparatus can estimate a sub-pixel disparity with a high degree of accuracy without depending on a brightness of a surface of an object. The compound eye imaging apparatus includes: a standard imaging optical system generating a picture including a standard image; reference imaging optical systems of an even number equal to or larger than two arranged substantially point-symmetrically with respect to the standard imaging optical system, which generate an picture including a reference image; a correlation value calculation unit (6) calculating a correlation value for each shifting amount by which one of the reference images is shifted parallel to a baseline, with respect to the standard image, for each of the reference imaging optical systems of an even number equal to or larger than two; a correlation value addition unit (7) calculating an integrated correlation value by adding, to each corresponding shifting amount, the correlation value calculated for each of said reference imaging optical systems of an even number equal to or larger than two; and a disparity calculation unit (8) calculating a disparity that is a shifting amount resulting in the largest degree of similarity between the standard image and the reference image.

Claims

1. A compound eye imaging apparatus which calculates a disparity occurring in a plurality of imaging optical systems that take an image of a same object,

said apparatus comprising:

a standard imaging optical system which generates a picture including a standard image by taking an image of the object;

reference imaging optical systems of an even number equal to or larger than two, each of which has an optical center and generates a picture including a reference image by taking an image of the object, the optical centers being arranged substantially point-symmetrically with respect to an optical center of said standard imaging optical system;

a correlation value calculation unit configured to calculate a correlation value indicating a degree of similarity between the standard image and the reference image, for each shifting amount by which a search position of the reference image in the picture generated by said reference imaging optical systems is shifted in a direction parallel to a baseline so that an image position of the reference image which is similar to the standard image is searched for, for each of said reference imaging optical systems of an even number equal to or larger than two, the baseline being a straight line connecting the optical center of said standard imaging optical system and the optical center of said reference imaging optical systems;

a correlation value addition unit configured to calculate an integrated correlation value by adding the correlation value to each corresponding shifting amount, the correlation value being calculated for each of said reference imaging optical systems of an even number equal to or larger than two; and

a disparity calculation unit configured to calculate, at a sub-pixel level, a disparity that is a shifting amount which results in a largest degree of similarity between the standard image and the reference image.

2. The compound eye imaging apparatus according to claim 1,

wherein said disparity calculation unit is configured to calculate a disparity at the sub-pixel level by interpolating the correlation value with use of an interpolation function using a symmetric property, the correlation value having been added for each of the shifting amounts by said correlation value addition unit.

3. The compound eye imaging apparatus according to claim 1,

wherein said compound eye imaging apparatus includes at least four reference imaging optical systems including a pair of first reference imaging optical systems and a pair of second reference imaging optical systems that are different from said first reference imaging optical systems arranged so that a direction of a baseline of said pair of first reference imaging optical systems is at an angle of a predetermined degree to a direction of a baseline of said pair of second reference imaging optical systems, said pair of first reference imaging optical systems being arranged substantially point-symmetrically with respect to the optical center of said standard imaging optical system and said pair of second reference imaging optical systems being arranged substantially point-symmetrically with respect to the optical center of said standard imaging optical system.

4. The compound eye imaging apparatus according to claim 3,

wherein said at least four reference imaging optical systems are arranged so that a first baseline length differs from a second baseline length, the first baseline length being a length of the baseline of said first reference imaging optical system and said standard imaging optical system, the second baseline length being a length of the baseline of said second reference imaging optical system and said standard imaging optical system, and

said correlation value calculation unit is configured to calculate, when calculating a correlation value of the reference image generated by said second reference imaging optical systems, the correlation value for each second shifting amount, the second shifting amount being a value obtained by multiplying a value resulted from subtracting the first baseline length from the second baseline length by the first shifting amount used for calculating the correlation value of the reference image generated by said first reference imaging optical systems.

5. The compound eye imaging apparatus according to claim 3,

wherein said standard imaging optical system and said at least four reference imaging optical systems are arranged to have a same positional relationship as a positional relationship of pixels included in an imaging apparatus included in said standard imaging optical system.

6. The compound eye imaging apparatus according to claim 1,

Wherein, in each pair of said reference imaging optical systems arranged substantially point-symmetrically with respect to the optical center of said standard imaging optical system, an optical center position error satisfies:

the optical center position error ≦

D·

pitch·

0.15/f, where D is a distance to the object, pitch is a pixel pitch, f is a focal length, and the optical center position error is a distance between a straight line and the optical center of one of said pair of reference imaging optical systems, the straight line connecting the optical center of an other of said pair of reference imaging optical systems and the optical center of said standard imaging optical system.

7. The compound eye imaging apparatus according to claim 1,

Wherein, in each pair of said reference imaging optical systems arranged substantially point-symmetrically with respect to the optical center of said standard imaging optical system, a baseline length error that is a difference in a length between a first baseline length and a second baseline length satisfies:

the baseline length error ≦

D·

pitch·

0.2/f, where D is a distance to the object, pitch is a pixel pitch, and f is a focal length, the first baseline length being a distance between the optical center of one of said pair of reference imaging optical systems and the optical center of said standard imaging optical system.

8. The compound eye imaging apparatus according to claim 1, further comprising

a preprocessing unit configured to apply a smoothing filter to the standard image and the reference image,

wherein said correlation value calculation unit is configured to calculate the correlation value based on the standard image and reference image to which smoothing filter has been applied.

9. A distance measuring apparatus which calculates a distance to an object or a three-dimensional location of the object by calculating a disparity occurring in a plurality of imaging optical systems that take an image of the same object, said apparatus comprising:

a standard imaging optical system which generates a picture including a standard image by taking an image of the object;

reference imaging optical systems of an even number equal to or larger than two, each of which has an optical center and generates a picture including a reference image by taking an image of the object, the optical centers being arranged substantially point-symmetrically with respect to an optical center of said standard imaging optical system;

a correlation value calculation unit configured to calculate a correlation value indicating a degree of similarity between the standard image and the reference image, for each shifting amount by which a search position of the reference image in the picture generated by said reference imaging optical systems is shifted in a direction parallel to a baseline so that an image position of the reference image which is similar to the standard image is searched for, for each of said reference imaging optical systems of an even number equal to or larger than two, the baseline being a straight line connecting the optical center of said standard imaging optical system and the optical center of said reference imaging optical systems;

a correlation value addition unit configured to calculate an integrated correlation value by adding the correlation value to each corresponding shifting amount, the correlation value being calculated for each of said reference imaging optical systems of an even number equal to or larger than two;

a disparity calculation unit configured to calculate, at a sub-pixel level, a disparity that is a shifting amount which results in a largest degree of similarity between the standard image and the reference image; and

a distance calculation unit configured to calculate a distance from said distance measuring apparatus to the object or a three-dimensional location of the object based on the calculated disparity, a focal length of said standard imaging optical system, and a length of the baseline.

10. A disparity calculation method for calculating a disparity occurring in a plurality of imaging optical systems that take an image of a same object, the plurality of imaging optical systems including:

a standard imaging optical system which generates a picture including a standard image by taking an image of the object; and

reference imaging optical systems of an even number equal to or larger than two, each of which has an optical center and generates a picture including a reference image by taking an image of the object, the optical centers being arranged substantially point-symmetrically with respect to an optical center of the standard imaging optical system,

said disparity calculation method comprising;

calculating a correlation value indicating a degree of similarity between the standard image and the reference image, for each shifting amount by which a search position of the reference image in the picture generated by the reference imaging optical systems is shifted in a direction parallel to a baseline so that an image position of the reference image which is similar to the standard image is searched for, for each of the reference imaging optical systems of an even number equal to or larger than two, the baseline being a straight line connecting the optical center of the standard imaging optical system and the optical center of the reference imaging optical systems;

calculating an integrated correlation value by adding the correlation value to each corresponding shifting amount, the correlation value being calculated for each of the reference imaging optical systems of an even number equal to or larger than two; and

calculating, at a sub-pixel level, a disparity that is a shifting amount which results in a largest degree of similarity between the standard image and the reference image.

11. A distance measuring method for calculating a distance to an object or a three-dimensional location of the object by calculating a disparity occurring in a plurality of imaging optical systems that take an image of the same object, the plurality of imaging optical systems including:

a standard imaging optical system which generates a picture including a standard image by taking an image of the object; and

reference imaging optical systems of an even number equal to or larger than two, each of which has an optical center and generates a picture including a reference image by taking an image of the object, the optical centers being arranged substantially point-symmetrically with respect to an optical center of the standard imaging optical system,

said distance measuring method comprising;

calculating a correlation value indicating a degree of similarity between the standard image and the reference image, for each shifting amount by which a search position of the reference image in the picture generated by the reference imaging optical systems is shifted in a direction parallel to a baseline so that an image position of the reference image which is similar to the standard image is searched for, for each of the reference imaging optical systems of an even number equal to or larger than two, the baseline being a straight line connecting the optical center of the standard imaging optical system and the optical center of the reference imaging optical systems;

calculating an integrated correlation value by adding the correlation value to each corresponding shifting amount, the correlation value being calculated for each of the reference imaging optical systems of an even number equal to or larger than two;

calculating, at a sub-pixel level, a disparity that is a shifting amount which results in a largest degree of similarity between the standard image and the reference image; and

calculating a distance from the distance measuring apparatus to the object or a three-dimensional location of the object based on the calculated disparity, a focal length of the standard imaging optical system, and a length of the baseline.