Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof

• US 5,530,420 A
• Filed: 12/27/1994
• Issued: 06/25/1996
• Est. Priority Date: 12/27/1993
• Status: Expired

• Current Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Hanawa, Keiji, Tsuchiya, Hideaki, Saneyoshi, Keiji
• Primary Examiner(s): Crosland, Donnie L.
• Application Number:
  US08/364,104
• Time in Patent Office: 546 Days
• Field of Search: 340/435, 340/901, 340/903, 180/167, 180/169, 348/42, 348/47, 348/51, 348/135, 348/143, 348/148, 348/169, 348/113, 348/116, 348/118, 364/424.02, 364/424.01
• US Class Current: 340/435
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0096 Synchronisation or controlling aspects

Abstract

A running guide apparatus for a vehicle includes stereoscopic picture processing element for processing a pair of stereoscopic pictures of an object outside a vehicle imaged by a imaging system mounted on the vehicle, construction detection element for detecting a plurality of various constructions by using three dimensional position data calculated with each portion of the object corresponding to the distance distribution data from the stereoscopic picture processing element; gap distance calculation element for respectively calculating the nearest distance as right and left gap distances each between an extended line of right or left side of the vehicle and each of ends on the vehicle side of the plurality of various constructions detected by the construction detection element; and information element for informing the driver of data related to the right and left gap data calculated by the gap distance calculation element; so as to obtain a distance distribution of an entire picture in dependency upon a discrepancy amount positions corresponding to of the pair of the stereoscopic pictures in accordance with a principle of a triangulation.

Claims

1. A running guide apparatus for a vehicle having a pair of cameras mounted on a front portion of said vehicle for taking a three-dimensional picture of objects in front of the vehicle and for generating a picture signal, said apparatus comprising:

stereoscopic picture processing means responsive to said picture signal for processing a pair of stereoscopic pictures of said object outside said vehicle and for producing a distance signal by obtaining a distance distribution of an entire picture in dependency upon a discrepancy amount positions corresponding to said pair of the stereoscopic pictures in accordance with a principle of a triangulation;

object detection means responsive to said distance signal for detecting a plurality of various objects by using three dimensional position data calculated with each portion of the object corresponding to the distance distribution and for generating a distance signal;

gap distance calculation means responsive to a distribution signal for respectively calculating a nearest distance of right and left gap distances each between an extended line of right or left side of the vehicle and each end of the vehicle side from the plurality of various objects and for producing a distance signal; and

information means responsive to said distance signal for informing a driver of data related to the right and left gap data so as to entirely and exactly display surrounding conditions of said vehicle and to largely decrease burdens of said driver when running on a narrow road.

2. The running guide apparatus for the vehicle according to claim 1, further comprising

two pairs of charge coupled device (CCD) cameras respectively arranged as said imaging system to a front surface of said vehicle with a predetermined interval, one pair provided for a long distance and the provided for a short distance, and each pair including a right and left cameras each other;

an image processor provided as said stereoscopic picture processing means, and including a distance detection circuit for searching a portion in which a same object is imaged in each small region of the stereoscopic picture pair from said CCD cameras and for calculating a distance to the object by obtaining a discrepancy amount of a position corresponding to said portion, and a distance picture memory for storing distance data outputted from said distance detection circuit;

a distance picture processing computer provided as said object detection means and said gap distance calculation means, and comprised of a multi-microprocessor system construction including a first microprocessor for mainly performing detection processing of individual object, a second microprocessor for mainly performing detection processing of a side wall, a third microprocessor for mainly performing calculation processing of a gap distance, and a system bus for connecting said first through third microprocessors in parallel; and

a discrepancy as said information means for displaying to a driver gap data by obtaining said calculation processing of said gap distance.

3. The running guide apparatus for the vehicle according to claim 2, further comprising

an object recognizer comprised of said first microprocessor;

a side wall detector comprised of said second microprocessor;

a gap distance calculator comprised of said third microprocessor;

a construction parameter memory for storing a processed result of said object recognizer and said side wall detector to output said result to said gap distance calculator.

4. The running guide apparatus for the vehicle according to claim 3, further comprising

object detector for dividing the distance picture from a distance picture memory comprised of said image processor into a predetermined interval in the stripe shape and for selecting only construction data having the danger as an object for running to calculate the detected distance;

a three dimension window generator for setting a three dimensional space region of a rectangular solid shape with respect to respective object detected by said object detector, and for calculating how to be shown set three dimension window on a two dimensional picture, so as to set the detection object with respect to only data within a window contour line as the two dimension window; and

an object contour extractor for surveying one by one each of the data in the two dimension window, for selecting only data included in said three dimension window, extracting a contour image of the detected object, and for detecting a positional relationship between the detected object and vehicle own body.

5. The running guide apparatus for the vehicle according to claim 4, further comprising

an object data extractor for extracting only data over the predetermined road surface previously set from the distance distribution data supplied from a distance picture memory comprised of said image processor;

a side wall straight line detector for extracting only data within a side wall search region previously set, for processing the extracted data by the Hough transformation, thereby detecting a linear equation showing presence or absence of the side wall and the position of the side wall; and

a side wall area detector for setting the side wall candidate region in which the side wall is estimated to exist on the basis of the straight line showing the position of the side wall, and for detecting the positions of before and after ends of the side wall in dependency on the distribution condition of the construction data in the side wall candidate region.

6. A running guide method for a vehicle having a pair of cameras mounted on a front portion of said vehicle for taking a three-dimensional picture of objects in front of the vehicle and for generating a picture signal, said apparatus comprising:

a step of processing a pair of stereoscopic pictures of said object outside said vehicle to produce a distance signal by obtaining a distance distribution of an entire picture in dependency upon a discrepancy amount positions corresponding to said pair of the stereoscopic pictures in accordance with a principle of a triangulation;

a step of detecting a plurality of various objects by using three dimensional position data calculated with each portion of the object corresponding to the distance distribution and for generating a distance signal;

a step of respectively calculating a nearest distance of right and left gap distances each between an extended line of right or left side of the vehicle and each end of the vehicle side from the plurality of various objects; and

a step of informing a driver of data related to the right and left gap data so as to entirely and exactly display surrounding conditions of said vehicle and to largely decrease burdens of said driver when running on a narrow road.

7. The running guide method for the vehicle according to claim 6, further comprising

a step of obtaining long distance data of a picture in front of the vehicle;

a step of obtaining short distance data of a picture in front of the vehicle;

a step of searching a portion in which a same object is imaged in each small region of the stereoscopic picture pair from said CCD cameras;

a step of calculating a distance to the object by obtaining a discrepancy amount of a position corresponding to said portion, and a distance picture memory for storing distance data outputted from a distance detection circuit;

a step of performing detection processing of a side wall, a third microprocessor for mainly performing calculation processing of a gap distance, and a system bus for connecting said first through third microprocessors in parallel; and

a step of displaying to a driver gap data by obtaining said calculation processing of said gap distance.

8. The running guide apparatus for the vehicle according to claim 7, further comprising

a step of dividing the distance picture from a distance picture memory comprised of a image processor into a predetermined interval in the stripe shape;

a step of selecting only construction data having the danger as an object for running to calculate the detected distance;

a step of setting a three dimensional space region of a rectangular solid shape with respect to respective object detected by said object detector;

a step of calculating how to be shown set three dimension window on a two dimensional picture, so as to set the detection object with respect to only data within a window contour line as the two dimension window;

a step of surveying one by one each of the data in the two dimension window;

a step of selecting only data included in said three dimension window;

a step of extracting a contour image of the detected object; and

a step of detecting a positional relationship between the detected object and vehicle own body.

9. The running guide apparatus for the vehicle according to claim 8, further comprising

a step extracting only data over the predetermined road surface previously set from the distance distribution data supplied from a distance picture memory comprised of said image processor;

a step of extracting only data within a side wall search region previously set;

a step of processing the extracted data by the Hough transformation, thereby detecting a linear equation showing presence or absence of the side wall and the position of the side wall;

a step of setting the side wall candidate region in which the side wall is estimated to exist on the basis of the straight line showing the position of the side wall; and

a step of detecting the positions of before and after ends of the side wall in dependency on the distribution condition of the construction data in the side wall candidate region.