Method and apparatus for monitoring the surroundings of a vehicle and for detecting failure of the monitoring apparatus

• US 5,617,085 A
• Filed: 05/30/1996
• Issued: 04/01/1997
• Est. Priority Date: 11/17/1995
• Status: Expired due to Term

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Tsutsumi, Kazumichi, Okamura, Shigekazu, Irie, Tatsuji
• Primary Examiner(s): Hofsass, Jeffery
• Assistant Examiner(s): TWEEL JR, JOHN ALEXANDER
• Application Number:
  US08/652,683
• Time in Patent Office: 306 Days
• Field of Search: 340/903, 340/901, 340/435, 348/169, 180/167, 180/169, 180/271, 382/107, 356/375
• US Class Current: 340/903
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60T 2201/08 Lane monitoring; Lane Keeping Systems
• CPC Class Codes: B60T 2201/089 using optical detection
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A camera and a beam-scan type laser radar whose optical axes are coincident are mounted on a vehicle. A lane detector detects the vehicle'"'"'s own lane on a display image by processing an image picked up by the camera. A coordinate transforming device transforms the coordinates of positional information of an object, most likely a vehicle, detected by the laser radar in conformity with coordinates on the image of the camera. Another detector compares a position of the vehicle'"'"'s own lane with a position of the detected object, judges whether or not the detected object is within the vehicle'"'"'s own lane, and separates the detected object within the vehicle'"'"'s own lane from objects detected outside the vehicle'"'"'s own lane. A failure detection method and a corresponding apparatus are provided to determine whether the optical axes of the camera and the laser radar are coincident.

Claims

1. A surroundings monitoring apparatus for monitoring the surroundings of a vehicle comprising:

a camera having an optical axis, said camera being mounted on said vehicle for detecting an image of a road;

a lane detecting means for detecting coordinates of a lane wherein said vehicle is located by processing image signals output from said camera onto a display image having coordinate axes;

a beam-scan type laser radar mounted on said vehicle and installed such that an optical axis center thereof is coincident with the optical axis of said camera;

a coordinate transforming means for transforming the coordinates of an object detected by said laser radar in conformity with the coordinate axes of said display image to provide transformed coordinates; and

a forward vehicle detecting means for separating objects detected within said lane from objects detected outside said lane by comparing said transformed coordinates with the coordinates of said lane.

2. A surroundings monitoring apparatus in accordance with claim 1, wherein said lane detecting means determines boundaries of said lane by distinguishing the white lines of said lane from the surroundings and further uses interpolative computations to determine said boundaries if said white lines are not continuous.

3. A surroundings monitoring apparatus in accordance with claim 1, wherein said forward vehicle detecting means distinguishes between vehicles located within said lane and vehicles located outside said lane.

4. A surroundings monitoring apparatus in accordance with claim 3, wherein said forward vehicle detecting means distinguishes between vehicles located within said lane and vehicles located outside said lane by comparing the horizontal locations of the reflections from said vehicles to the horizontal locations of respective points of said lanes along a vertical line.

5. A surroundings monitoring apparatus in accordance with claim 1, further comprising:

a representative coordinate computing means for computing representative coordinates of an object detected by the laser radar, said representative coordinates being transformed in conformity with said coordinate axes on said display image.

6. A surroundings monitoring apparatus in accordance with claim 5, wherein said representative coordinates represent a center of gravity of said object.

9. A failure judgment apparatus for judging a failure of a surroundings monitoring apparatus in accordance with claim 4, further comprising:

an optical axis judgment inhibiting means for inhibiting the judgment of coincidence or non-coincidence between the optical axis of the camera and the optical axis of the laser radar when the distance indicated by the representative coordinates detected by the laser radar is greater than a prescribed distance.

7. A failure judgment apparatus for judging a failure of a surroundings monitoring apparatus comprising:

a camera having an optical axis, said camera being mounted on a vehicle for picking up an image of a road;

a beam-scan type laser radar having an optical axis mounted on said vehicle and installed such that an optical axis center thereof is coincident with the optical axis of said camera;

a representative coordinate computing means for computing representative coordinates of other vehicle detected by said laser radar;

a coordinate transforming means for transforming said representative coordinates in conformance with coordinate axes on a display image of said camera;

a window setting means for setting a window for designating a prescribed region on the basis of said representative coordinates transformed by said coordinate transforming means;

an optical axis judging means for judging whether or not the optical axis of said camera is coincident with the optical axis of said laser radar depending upon whether or not the other vehicle is within said window by processing image signals in said window.

8. A failure judgment apparatus for judging a failure of a surroundings monitoring apparatus in accordance with claim 7, further comprising:

a window correcting means for correcting dimensions of the prescribed region designated by the window corresponding to a distance indicated by the representative coordinates of the other vehicle detected by the laser radar,

wherein said optical axis judging means comprises;

a histogram computing means for computing a histogram of an outline existing in the corrected window;

a comparative reference value setting means for setting a comparative reference value to be compared with said histogram corresponding to said distance; and

a comparing means for comparing said histogram with said comparative reference value and judging that the other vehicle is within said corrected window when a maximum value of said histogram is greater than the comparative reference value for a prescribed time.

10. A surroundings monitoring method for monitoring surroundings of a vehicle, comprising the steps of:

detecting image signals of a lane in which said vehicle is located with a camera mounted on said vehicle;

determining coordinates of said lane on a display image by processing said image signals;

detecting an object ahead of said vehicle with a beam-scan type laser radar the center of whose optical axes are coincident with the optical axes of said camera;

transforming coordinates of said detected object in conformity with coordinate axes on said display image; and

judging whether said detected object is within the lane of said vehicle by comparing said transformed coordinates with said coordinates of said lane.

11. A failure judging method for judging a failure of a surroundings monitoring apparatus, said apparatus being provided with a camera mounted on a vehicle for picking up an image of a road and a beam-scan type laser radar mounted on said vehicle and installed such that an optical axis center thereof is coincident with an optical axis of said camera, comprising the steps of:

computing representative coordinates of another vehicle detected by said laser radar;

transforming said representative coordinates in conformity with coordinate axes of a display image detected by said camera;

setting a window for designating a prescribed region on the basis of said transformed representative coordinates of said other vehicle;

processing image signals in said window and judging whether or not the optical axes of said camera are coincident with the optical axes of said laser radar depending upon whether said other vehicle is within said window.