Image-correspondence position detection device, distance measuring device and apparatus using the same

• US 20010055418A1
• Filed: 05/16/2001
• Published: 12/27/2001
• Est. Priority Date: 05/17/2000
• Status: Active Grant

• Current Assignee: Minolta Corporation Limited (Konica Minolta Inc.)
• Original Assignee: Minolta Corporation Limited (Konica Minolta Inc.)
• Inventors: Nakamura, Kenji
• Granted Patent: US 6,909,802 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

An image-correspondence position detection device comprising an extractor 12 for extracting a sampling image of specific size from a pair of images, correlation calculator 114 for generating a correlation value line and calculating an integer correlation position and a decimal correlation position while sequentially shifting the extraction position, measured correlation value calculator 118 for calculating a measured correlation value for a partial area at the integer correlation position, estimated correlation value calculator 116 for calculating an estimated correlation value assuming uniform shifting in each pixel of similar sampling images by the difference between the integer correlation position and the decimal correlation position for the partial area, and a correspondence degree calculator 120 for calculating the degree of correspondence between pixels in the partial area from the measured correlation value and the estimated correlation value.

Claims

1. An image-correspondence position detecting device comprising:

an extractor for sampling image data of specific size from a pair of images to obtain a sample image;

a correlation calculator for creating a correlation value row by calculating correlation values representing the degree of correspondence between a pair of sample images while sequentially shifting the sampling position from at least one of the images, and for calculating minimum or maximum positions of correspondence values among the correlation value row which represents an integer correspondence position representing a sampling position of optimum degree of correspondence, and for interpolating between the correlation values and calculates a minimum value or maximum value which represents a decimal correspondence position having the optimum degree of correspondence;

measured correlation value calculator for calculating the measured correlation value of a partial correlation value for an optional partial area smaller than the sampling image, which is an area corresponding to each sampling image at the integer correspondence position;

estimated correlation value calculator for calculating an estimated correlation value of a partial correlation value from the difference between the integer correspondence position and the decimal correspondence position in the partial area based on the assumption that each sampling image is uniformly shifted in each pixel; and

correspondence degree calculator for calculating the degree of correspondence between optional pixels in the partial area from the measured correlation value and the estimated correlation value.

2. A distance measuring device comprising:

a pair of optical systems for forming a pair of object images;

a pair of image sensing elements provided near the image forming position of each optical system;

an extractor for sampling image data of specific size from a pair of images on said image sensing elements to obtain a sample image;

a correlation calculator for creating a correlation value row by calculating correlation values representing the degree of correspondence between a pair of sample images while sequentially shifting the sampling position from at least one of the images, and for calculating minimum or maximum positions of correspondence values among the correlation value row which represents an integer correspondence position representing a sampling position of optimum degree of correspondence, and for interpolating between the correlation values and calculates a minimum value or maximum value which represents a decimal correspondence position having the optimum degree of correspondence;

a measured correlation value calculator for calculating the measured correlation value of a partial correlation value for an optional partial area smaller than the sampling image, which is an area corresponding to each sampling image at the integer correspondence position;

an estimated correlation value calculator for calculating an estimated correlation value of a partial correlation value from the difference between the integer correspondence position and the decimal correspondence position in the partial area based on the assumption that each sampling image is uniformly shifted in each pixel;

a correspondence degree calculator for calculating the degree of correspondence between optional pixels in the partial area from the measured correlation value and the estimated correlation value;

a distance calculator for calculating a distance based on the integer correspondence position and a decimal correspondence position; and

a discriminator for discriminating whether the degree of correspondence is good or poor by comparing for each pixel a specific value and the degree of correspondence, so that if the degree is discriminated as poor, the pixel is determined unmeasurable, and if the degree is discriminated as good, the distance measurement data is adopted.

3. A distance measuring device according to

claim 2

, wherein, a plurality of distance detecting areas are provided for the pair of object images,

at least part of the pixels included in the pair of images overlap a plurality of the distance detecting areas, and

the distance data is determined for optional pixels i nclud ed in the overlapping distance detecting areas based on the degree of correspondence discriminated a s good among the degrees of correspondence obtained from the plurality of distance detecting areas.

4. A distance measuring device according to

claim 3

, wherein a distance data is obtained based on a distance measuring area having the best degree of correspondence among the degrees of correspondence obtained from a plurality of distance measuring areas determined to have good degrees of correspondence.

5. An image-correspondence position detection device comprising:

an extractor for extracting data of images A′ and

B′

while sequentially shifting the sampling position of at least one image among a pair of images A and B including a plurality of pixel rows arranged unidimensionally;

correlation calculator for calculating correlation values of only a specific number of samples to generate a correlation value row f(j);

an interpolator for obtaining a sampling position having the highest correlation to a decimal value d between pixels by interpolating the correlation values from the generated correlation value row f(j); and

an image correlation degree calculator for calculating the degree of correlation dY_i of optional pixels in images A′ and

B′

by the following formula,

dY_i=|Y_i−

Y_i′

| 

 

(1) Y_i=PA_i−

PB_i 

 

(2) Y_i′

=C_i×

δ

 

 

(3) δ

=d (where d≦

0.5) or 1−

d (where d>

0.5) 

 

(4) C_i=|PA_i−

PA_i+1| 

 

(5)

where PA_i and PB_i are data of the number i optional pixels of A′ and

B′

in the combination of A′ and

B′

having the smallest correspondence values f(j).

6. An image-correspondence position detection device comprising:

an extractor for extracting data of images A′ and

B′

while sequentially shifting the sampling position of at least one image among a pair of images A and B including a plurality of pixel rows arranged unidimensionally;

correlation calculator for calculating correlation values of only a specific number of samples to generate a correlation value row f(j);

an interpolator for obtaining a sampling position having the highest correlation to a decimal value d between pixels by interpolating the correlation values from the generated correlation value row f(j); and

an image correlation degree calculator for calculating the degree of correlation dYi of optional pixels in images A′ and

B′

by the following formula,

dY_i=|Y_i−

Y_i′

| 

 

(1) Y_i=PA_i−

PB_i 

 

(2) Y_i′

=C_i×

δ

 

 

(3) δ

=d (where d≦

0.5) or 1−

d (where d>

0.5) 

 

(4) C_i=|PA<sub>i−

1</sub>−

PA_i+1|/2 

 

(5a)

where PA_l and PB_i are data of the number i optional pixels of A′ and

B′

in the combination of A′ and

B′

having the smallest correspondence values f(j).

7. An image-correspondence position detection device comprising:

an extractor for extracting data of images A′ and

B′

while sequentially shifting the sampling position of at least one image among a pair of images A and B including a plurality of pixel rows arranged unidimensionally;

correlation calculator for calculating correlation values of only a specific number of samples to generate a correlation value row f(j);

an interpolator for obtaining a sampling position having the highest correlation to a decimal value d between pixels by interpolating the correlation values from the generated correlation value row f(j); and

an image correlation degree calculator for calculating the degree of correlation dYi of optional pixels in images A′ and

B′

by the following formula,

dY_i=|Y_i−

Y_i′

| 

 

(1) Y_i=PA_i−

PB_i 

 

(2) Y_i′

=C_i×

δ

 

 

(3) δ

=d (where d≦

0.5) or 1−

d (where d>

0.5) 

 

(4) C_i=(PA<sub>i−

1</sub>−

PA_l+1) /2+(PB<sub>i−

1</sub>−

PB_i+1)/2|/2 

 

(5b)

where PA_i and PB_i are data of the number i optional pixels of A′ and

B′

in the combination of A′ and

B′

having the smallest correspondence values f(j).

8. An image-correspondence position detecting device comprising:

an extractor for sampling image data of specific size from a pair of images to obtain a sample image;

a correlation calculator for creating a correlation value row by calculating correlation values representing the degree of correspondence between a pair of sample images while sequentially shifting the sampling position from at least one of the images, and for calculating minimum or maximum positions of correspondence values among the correlation value row which represents an integer correspondence position representing a sampling position of optimum degree of correspondence, and for interpolating between the correlation values and calculates a minimum value or maximum value which represents a decimal correspondence position having the optimum degree of correspondence;

measured correlation value calculator for calculating the measured correlation value of a partial correlation value for an optional partial area smaller than the sampling image, which is an area corresponding to each sampling image at the integer correspondence position;

estimated correlation value calculator for calculating an estimated correlation value of a partial correlation value from the difference between the integer correspondence position and the decimal correspondence position in the partial area based on the assumption that each sampling image is uniformly shifted in each pixel;

correspondence degree calculator for calculating the degree of correspondence between optional pixels in the partial area from the measured correlation value and the estimated correlation value; and

a reliability evaluation value calculator for calculating a reliability evaluation value for evaluating the reliability of the sampling image at the integer correspondence position from the degree of correspondence and the change in luminance of the object in the sampling image at the integer correspondence position.

9. A distance measuring device having the image-correspondence position detecting device according to

claim 8

, comprising:

a selector for selecting one or more distance measurement areas, or determining not select any distance measurement area based on the calculated reliability evaluation value.

10. An image-correspondence position detection device comprising:

an extractor for extracting data of images A′ and

B′

while sequentially shifting the sampling position of at least one image among a pair of images A and B including a plurality of pixel rows arranged unidimensionally;

correlation calculator for calculating correlation values of only a specific number of samples to generate a correlation value row f(j);

an interpolator for obtaining a sampling position having the highest correlation to a decimal value d between pixels by interpolating the correlation values from the generated correlation value row f(j); and

a reliability evaluation value calculator for calculating a reliability evaluation value SY/C for evaluating the degree of correspondence at the correspondence positions of images A′ and

B′

by the following formula,

SY/C=Σ

|Y_i−

Y_i′

|Σ

C_i 

 

(11) Y_i=PA_i−

PB_i 

 

(12) Y_i′

=Ci_i×

δ

 

 

(13) δ

=d (where d≦

0.5) or 1−

d (where d>

0.5) 

 

(14) C_i=PA_i−

PA₁₊₁| 

 

(15)

where, PA_i and PB_i are data of optional number i pixels of images A′ and

B′

in the combination of A′

, B′

having the smallest correlation value f(j), and

where, the Σ

s are the sum of i=0, 1, . . . , Np, and Np is the number of pixels included in image A′

.

11. A distance measuring device having the image-correspondence position detecting device according to

claim 10

, comprising:

a selector for selecting one or more distance measurement areas, or determining not select any distance measurement area based on the calculated reliability evaluation value.

12. An image-correspondence position detection device comprising:

an extractor for extracting data of images A′ and

B′

while sequentially shifting the sampling position of at least one image among a pair of images A and B including a plurality of pixel rows arranged unidimensionally;

correlation calculator for calculating correlation values of only a specific number of samples to generate a correlation value row f(j);

an interpolator for obtaining a sampling position having the highest correlation to a decimal value d between pixels by interpolating the correlation values from the generated correlation value row f(j); and

a reliability evaluation value calculator for calculating a reliability evaluation value SY/C for evaluating the degree of correspondence at the correspondence positions of images A′ and

B′

by the following formula,

SY/C=Σ

|Y_i−

Y_i′

|Σ

C_i 

 

(11) Y_i=PA_i−

PB_i 

 

(12) Y_i′

=Ci_i×

δ

 

 

(13) δ

=d (where d≦

0.5) or 1−

d (where d>

0.5) 

 

(14) C_i=PA_i−

PA₁₊₁| 

 

(15) C_i=|PA<sub>i−

1</sub>−

PA_i+1|/2 

 

(15a)

where, PA_i and PB_l are data of optional number i pixels of images A′ and

B′

in the combination of A′

, B′

having the smallest correlation value f(j), and

where, the Σ

s are the sum of i=0, 1, . . . , Np, and Np is the number of pixels included in image A′

.

13. A distance measuring device having the image-correspondence position detecting device according to

claim 12

, comprising:

a selector for selecting one or more distance measurement areas, or determining not select any distance measurement area based on the calculated reliability evaluation value.

14. An image-correspondence position detection device comprising:

an extractor for extracting data of images A′ and

B′

while sequentially shifting the sampling position of at least one image among a pair of images A and B including a plurality of pixel rows arranged unidimensionally;

correlation calculator for calculating correlation values of only a specific number of samples to generate a correlation value row f(j);

an interpolator for obtaining a sampling position having the highest correlation to a decimal value d between pixels by interpolating the correlation values from the generated correlation value row f(j); and

a reliability evaluation value calculator for calculating a reliability evaluation value SY/C for evaluating the degree of correspondence at the correspondence positions of images A′ and

B′

by the following formula,

SY/C=Σ

|Y_i−

Y_i′

|Σ

C_i 

 

(11) Y_i=PA_i−

PB_i 

 

(12) Y_i′

=Ci_i×

δ

 

 

(13) δ

=d (where d≦

0.5) or 1−

d (where d>

0.5) 

 

(14) C_i=PA_i−

PA₁₊₁| 

 

(15) C_i=|(PA<sub>i−

1</sub>−

PA_i+1) /2+(PB<sub>i−

1</sub>−

PB_i+1) /2|/2 

 

(15b)

where, PA_i and PB_i are data of optional number i pixels of images A′ and

B′

in the combination of A′

, B′

having the smallest correlation value f(j), and

where, the Σ

s are the sum of i=0, 1, . . . , Np, and Np is the number of pixels included in image A′

.

15. A distance measuring device having the image-correspondence position detecting device according to

claim 14

, comprising:

a selector for selecting one or more distance measurement areas, or determining not select any distance measurement area based on the calculated reliability evaluation value.