Distance correcting apparatus of surroundings monitoring system and vanishing point correcting apparatus thereof

• US 6,985,619 B1
• Filed: 09/22/2000
• Issued: 01/10/2006
• Est. Priority Date: 09/22/1999
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Seta, Itaru, Hanawa, Keiji
• Primary Examiner(s): AHMED, SAMIR ANWAR
• Application Number:
  US09/667,534
• Time in Patent Office: 1,936 Days
• Field of Search: 382/154, 382/103, 382/104, 382/106, 382/202, 382/285, 382/291, 382/294, 348/113, 348/139, 348/140, 348/135, 348/116, 348/118, 348/148, 356/3, 356/614, 356/12, 356/14, 345/419
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/133 Equalising the characteristics of different image components, e.g. their average brightness or colour balance
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0096 Synchronisation or controlling aspects

Abstract

A distance correcting apparatus of a surroundings monitoring system includes a stereo imaging means for stereoscopically taking a pair of images of a frontal scenery, a parallax calculating means for calculating a parallax based on the pair of images, a distance calculating means for calculating a distance to an object based on the parallax and a parameter for correcting distance, an approximation line calculating means for calculating a plurality of approximation lines extending in the distance direction in parallel with each other based on the images, a vanishing point calculating means for calculating a vanishing point of the images from a point of intersection of the approximation lines and a parameter correcting means for correcting the parameter based on the calculated vanishing point.

Claims

1. A distance correcting apparatus of a surroundings monitoring system, comprising:

stereo imaging cameras for producing a pair of stereoscopic images including a reference image and a comparison image;

parallax calculating means for calculating a parallax for a pixel block including calculating a horizontal deviation amount between the reference and comparison images;

distance calculating means for calculating a distance to an object based on said pixel block parallax and a vanishing point parallax;

approximation line calculating means for calculating a plurality of approximation lines extending in the distance direction in parallel with each other based on said images;

vanishing point calculating means for calculating a vanishing point of said images from a point of intersection of said approximation lines;

reference object detecting means for calculating a lane marker model expressing a change of a road surface height with respect to distance;

means for identifying a condition of change of an actual road surface height based on said vanishing point, and for identifying a condition of change of a calculated road surface height based on said lane marker model; and

parameter correcting means for correcting said vanishing point parallax so that said condition of change of said calculated road surface height becomes close to said condition of change of said actual road surface height.

2. The distance correcting apparatus according to claim 1, wherein

said parameter correcting means corrects vanishing point parallax by reducing a deviation between a first gradient indicating said condition of change of road surface height with respect to distance and a second gradient indicating said condition of change of actual road surface height.

3. The apparatus according to claim 2, wherein

said vanishing point calculating means calculates an approximation line in said image plane for respective reference objects, when a plurality of reference objects are detected by said reference objects detecting means.

4. The apparatus according to claim 2, wherein

said reference objects are lane markers on a road projected in said images and when left and right lane markers are detected on said road, and

said vanishing point calculating means calculates an approximation line in said image plane for said respective left and right lane markers.

5. The apparatus according to claim 4, wherein

said vanishing point calculating means calculates said approximation line based on said left and right lane markers existing within a specified distance range.

6. The apparatus according to claim 4, wherein

said vanishing point calculating means judges whether or not a lane marker projected in said images is a straight line and when so judged, calculates said vanishing point of said images.

7. The apparatus according to claim 6, wherein

said vanishing point calculating means evaluates a time-versus change of the position of a lane marker projected in said images, if said time-versus change is small, judges that said lane marker has a high reliability as lane markers, and calculates said vanishing point in said images.

8. A distance correcting apparatus of a surroundings monitoring system, comprising:

stereo cameras for producing a pair of stereoscopic images including a reference image and a comparison image;

transforming means for geometrically transforming said reference and comparison images based on a first parameter indicating a transference in the horizontal direction;

parallax calculating means for calculating a parallax based on said pair of images by calculating a horizontal deviation amount between the reference and comparison images;

distance calculating means for calculating a distance to an object based on said parallax and a first parameter for correcting said distance;

vanishing point calculating means for calculating a plurality of approximation lines extending in the distance direction in parallel with each other and calculating a vanishing point of said images from a point of intersection of said approximation lines;

reference object detecting means for calculating a lane marker model expressing a change of a road surface height with respect to distance;

means for identifying a condition of change of an actual road surface height based on said vanishing point, and for identifying a condition of change of a calculated road surface height based on said lane marker model; and

parameter correcting means for correcting said first parameter based on said vanishing point so that said condition of change of said calculated road surface height becomes close to said condition of change of said actual road surface height.

9. The apparatus according to claim 8, wherein

said parameter correcting means corrects said first parameter by reducing a deviation between a first gradient indicating said condition of change of road surface height with respect to distance and a second gradient indicating said condition of change of actual road surface height.

10. The apparatus according to claim 9, wherein

said vanishing point calculating means calculates an approximation line in said image plane for respective reference objects, when a plurality of reference objects are detected by said reference objects detecting means.

11. The apparatus according to claim 9, wherein

said reference objects are lane markers on a road projected in said images and when left and right lane markers are detected on said road, said vanishing point calculating means calculates an approximation line in said image plane for said respective left and right lane markers.

12. The apparatus according to claim 11, wherein

said vanishing point calculating means calculates said approximation line based on said left and right lane markers existing within a specified distance range.

13. The apparatus according to claim 11, wherein

said vanishing point calculating means judges whether or not a lane marker projected in said images is a straight line and and when so judged, calculates said vanishing point of said images.

14. The apparatus according to claim 13, wherein

said vanishing point calculating means evaluates a time-versus change of the position of a lane marker projected in said images, if said time-versus change is small, judges that said lane marker has a high reliability as lane markers, and calculates said vanishing point in said images.

15. A vanishing point correcting apparatus of a surroundings monitoring system for taking images of a scenery in front of a vehicle and for obtaining a three-dimensional information of an object projected in said images by making use of an established vanishing point established beforehand, comprising:

reference object detecting means for detecting lane markers on a road projected in said images and for identifying a position of said lane markers on an image plane of said images;

vanishing point calculating means, when a left and right lane marker is detected on said road and it is judged that said lane marker projected in said images is a straight line, for calculating an approximation line in said image plane for said respective left and right lane markers and for calculating a vanishing point from a point of intersection of said approximation lines; and

a vanishing point correcting means for correcting said established vanishing point so that said established vanishing point comes close to said vanishing point calculated by said vanishing point calculating means.

16. The apparatus according to claim 15, wherein

said vanishing point calculating means evaluates a time-versus change of the position of a lane marker projected in said images, if said time-versus change is small, judges that said lane marker has a high reliability as lane markers, and calculates said vanishing point in said images.