Apparatus and method for stereo matching and method of calculating an infinite distance corresponding point

• US 7,092,015 B1
• Filed: 09/21/2000
• Issued: 08/15/2006
• Est. Priority Date: 09/22/1999
• Status: Expired due to Term

• Current Assignee: Subaru Corp.
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Sogawa, Yoshiyuki
• Primary Examiner(s): Ometz, David
• Assistant Examiner(s): SELBY, GEVELL V
• Application Number:
  US09/665,952
• Time in Patent Office: 2,154 Days
• Field of Search: 348/222.1, 348/373, 348/116, 348/333.06, 348/118, 348/148, 348/221.1, 348/42, 358/88, 358/229, 358/3, 358/209, 352/57, 382/106, 382/154, 356/3, 356/3.14, 356/12, 396/128
• US Class Current: 348/222.1
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

The reference image data corresponding to one pixel block stored in a reference image line memory 6 is output to a stereo matching circuit 8. The comparative image data corresponding to a “search range” stored in a comparative image line memory 7 is output to the stereo matching circuit 8 according to an instruction of a search range address generator 13. The stereo matching circuit 8 calculates parallaxes by performing the stereo matching based on the reference image data and the comparative image data. A position where the search range S is set is corrected based on the amount of the deviation of an infinite distance corresponding point with respect to a position of the reference image data.

Claims

1. A stereo matching apparatus comprising:

a stereo camera system for taking pictures of a predetermined area and producing first and second images of the predetermined area;

a memory for storing first image data within a reference pixel region in the first image and second image data on a horizontal line in the second image, the horizontal line being corresponding to a vertical position of the reference pixel region in the first image;

a correction table for storing values based on a calibration process carried out with respect to the stereo camera system;

an address generator for setting a search range in the second image for a stereo matching and instructing to read out from said memory a part of the second image data within the search range and the first image data within the reference pixel region, the search range being set according to a value read from the correction table; and

a stereo matching unit for identifying a correlated destination of the reference pixel region by the stereo matching based on the part of the second image data within the search range and the first image data within the reference pixel region and calculating a parallax of the reference pixel region based on the correlated destination of the reference pixel region,

wherein the value read from the correction table is indicative at least in part of an amount of deviation of an infinite distance corresponding point with respect to a horizontal position of the reference pixel region, said amount being individually set for each reference pixel region in the first image so as to set the search range in the second image in relation to each reference pixel region in the first image.

2. The stereo matching apparatus as recited in claim 1, wherein said address generator shifts the location of the search range in the same direction as a direction of the deviation of the infinite distance corresponding point with respect to the horizontal position of the reference pixel region.

3. The stereo matching apparatus as recited in claim 2, wherein said address generator shifts a starting point of the search range in the same direction as the direction of the deviation of the infinite distance corresponding point.

4. The stereo matching apparatus as recited in claim 2, wherein said address generator shifts an end point of the search range in the same direction as the direction of the deviation of the infinite distance corresponding point.

5. The stereo matching apparatus as recited in claim 1, wherein said address generator identifies the amount of correction of the search range in relation to the amount of deviation of the infinite distance corresponding point.

6. The stereo matching apparatus as recited in claim 2, wherein said address generator identifies the amount of correction of the search range in relation to the amount of deviation of the infinite distance corresponding point.

7. The stereo matching apparatus as recited in claim 3, wherein said address generator identifies the amount of correction of the search range in relation to the amount of deviation of the infinite distance corresponding point.

8. The stereo matching apparatus as recited in claim 4, wherein said address generator identifies the amount of correction of the search range in relation to the amount of deviation of the infinite distance corresponding point.

9. The stereo matching apparatus as recited in claim 2, further comprising:

a correction value memory for storing a correction value for correcting the location of the search range in relation to a location of the reference pixel region in the first image;

wherein said address generator sets the location of the search range for the reference pixel region based on the location of the reference pixel region and the correction value stored in said correction value memory for said location of the reference pixel region.

10. The stereo matching apparatus as recited in claim 3, further comprising:

a correction value memory for storing a correction value for correcting the location of the search range in relation to a location of the reference pixel region in the first image;

wherein said address generator sets the starting point of the search range for the reference pixel region based on the location of the reference pixel region anal the correction value stored in said correction value memory for said location of the reference pixel region.

11. The stereo matching apparatus as recited in claim 1, wherein said stereo camera system comprises a pair of stereo cameras mounted on a vehicle, the pair of stereo cameras taking pictures of scenes outside of the vehicle, and said stereo snatching unit calculates a distance to an object outside the vehicle on the basis of the parallax of the reference pixel region.

12. The stereo matching apparatus according to claim 1, wherein:

the stereo camera system takes a picture of a screen by a pair of stereo cameras and produces third and fourth images of the screen with being drawn a first vertical line and a second vertical line at an interval equal to a baseline of the stereo cameras;

the apparatus calculates a deviation of apposition of the second vertical line shown on the fourth image with respect to a position of the first vertical line shown on the third image and an infinite distance corresponding point corresponding to each position on the first vertical line shown on the third image on the basis of the deviation of the position of the second vertical line.

13. A stereo matching method comprising:

taking pictures of a predetermined area and producing first and second images of the predetermined area;

identifying a reference pixel region in the first image;

identifying a search range for the reference pixel region in the second image;

defining a search range for the reference pixel region based on a previously determined and stored value representing an amount of deviation of an infinite distance corresponding point with respect to position of the reference pixel region;

identifying a correlated destination of pixel region by searching a horizontal the reference through the search range; and

wherein said amount is individually set for each reference pixel region in the first image so as to set the search range in the second image in relation to a position of the reference pixel region in the first image.

14. The stereo matching method as recited in claim 13, wherein said step of correcting the location of the search range shifts the location of the search range in the same direction as the direction of the deviation of the infinite distance corresponding point with respect to the horizontal position of the reference pixel region.

15. The stereo matching method as recited in claim 14, wherein said step of correcting the location of the search range shifts a starting point of the search range in the same direction as the direction of the deviation of the infinite distance corresponding point with respect to the horizontal position of the reference pixel region.

16. The stereo matching method as recited in claim 14, wherein said step of correcting the location of the search range shifts an end point of the search range in the same direction as the direction of the deviation of the infinite distance corresponding point with respect to the horizontal position of the reference pixel region.

17. The stereo matching method as recited in claim 13, wherein said step of correcting the location of the search range corrects the location of the search range with a correction value corresponding to the deviation of the infinite distance corresponding point.

18. The stereo matching method as recited in claim 14, wherein said step of correcting the location of the search range corrects the location of the search range with a correction value corresponding to the deviation of the infinite distance corresponding point.

21. The stereo matching method as recited in claim 13, further comprising:

calculating a distance to an object outside a vehicle on the basis of the correlated destination of the reference pixel region,

wherein said step of taking pictures of the predetermined area takes pictures of scenes outside of the vehicle.

22. The stereo matching method as recited in claim 13, further comprising:

taking picture of a screen by a pair of stereo cameras and producing third and fourth images of the screen said screen being drawn a first vertical line and a second vertical line at an interval equal to a baseline of the stereo cameras;

identifying a deviation of a position of the second vertical line shown on the fourth image with respect to a position of the first vertical line shown on the third image; and

calculating an infinite distance corresponding point corresponding to each position on the first vertical line shown on the third image on the basis of the deviation of the position of the second vertical line,

wherein said step of correcting the location of the search range uses the infinite distance corresponding point thus calculated as the infinite distance corresponding point corresponding to the horizontal position of the reference pixel region.

19. The stereo matching method as recited in 15, wherein said step of correcting the location of the search range corrects the location of the search range with a correction value corresponding to the deviation of the infinite distance corresponding point.

20. The stereo matching method as recited in 16, wherein said step of correcting the location of the search range corrects the location of the search range with a correction value corresponding to the deviation of the infinite distance corresponding point.

23. A method of calculating an infinite distance corresponding point, the method comprising:

taking picture of a screen by a pair of stereo cameras and producing first and second images of the screen, said screen being drawn a first vertical line and a second vertical line at an interval equal to a baseline of the stereo cameras;

identifying a deviation of a position of the second vertical line shown on the second image with respect to a position of die first vertical line shown on the first image; and

calculating each infinite distance corresponding point corresponding to each position on the first vertical line shown on the first image on the basis of the deviation of the position of the second vertical line.