Environment monitoring system

• US 6,466,684 B1
• Filed: 09/13/1999
• Issued: 10/15/2002
• Est. Priority Date: 09/14/1998
• Status: Expired due to Term

• Current Assignee: Yazaki Corporation
• Original Assignee: Yazaki Corporation
• Inventors: Ishikawa, Naoto, Sasaki, Kazuyuki, Nakajima, Masato, Imai, Teruhisa
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US09/394,689
• Time in Patent Office: 1,128 Days
• Field of Search: 382/103, 382/104, 382/106, 382/199, 340/903, 340/435, 340/937, 348/149
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

In an environment monitoring system using an optical flow, a monitoring region is set in one'"'"'s own vehicle lane or an adjacent lane on the basis of the white lines. In this case, an early image is divided into a plurality of small regions. An edge is extracted which is an candidate of each the white lines for each the small regions using a virtual FOE for each the small regions acquired previously of the early image. A white line is determined is determined on the basis of the edge extracted. Finally, white line segments are extended for each small region determined to acquire a coordinate of a crossing point thereof and that of the virtual FOE is stored for each the small regions. The virtual FOE is used to extract the edge for each of the small regions of an image newly picked-up in order to detect the optical flow.

Claims

1. An environment monitoring system for monitoring a relative relation between one'"'"'s own vehicle and another surrounding vehicle comprising:

image pick-up means for picking up two early and later successive images which are located at a front, rear or diagonal-rear position from one'"'"'s own vehicle at two early and later timings and on the basis of the images thus picked-up, detecting white lines of one'"'"'s own vehicle lane on which one'"'"'s own vehicle runs;

monitoring region setting means for setting a monitoring region in one'"'"'s own vehicle lane or an adjacent lane on the basis of the white lines thus detected; and

optical-flow detecting means for detecting an optical flow generated from another surrounding vehicle within the monitoring region, wherein the monitoring setting means includes;

a small region dividing means for diving the early image into a plurality of small regions,

an edge extracting means for extracting an edge which is an candidate of each the white lines for each the small regions using a virtual FOE for each the small regions acquired previously of the early image,

a white line determining means for determining each the white lines on the basis of the edge extracted by the edge extracting means, and

a virtual FOE estimating means for extending segments of white lines for each small region determined by the white line determining means to acquire a coordinate of a crossing point thereof and storing that of the virtual FOE for each the small regions whereby the virtual FOE is used to extract the edge for each the small regions of the later image.

2. An environment monitoring system according to claim 1, wherein the edge extracting means comprises:

means for gradually shifting a fan-like region by a predetermined angle around the coordinate of the virtual FOE to extract the edge for each fan-like region, and

means for taking the extracted edges whose number exceeds a threshold value as a candidate of each the white lines.

3. An environment monitoring system according to claim 2, wherein the white line determining means performs linear approximation for the edges to acquire the corresponding line segments and determines the line segments satisfying continuity between the small regions as the white lines.

4. An environment monitoring system according to claim 1, wherein the white line determining means comprises:

means for acquiring the coordinates of crossing points of extended lines of all pairs of white line segments in each of the small regions subjected to the linear approximation and angles formed thereby; and

means for acquiring differences between the coordinates and angles thus acquired and the coordinate of the virtual FOE and angle of two white line segments in each the small regions which have been acquired previously from the early image so that a pair of lines providing minimum values in coordinate and angle are determined as the white lines of the small region at issue.

5. An environment monitoring system according to claim 1, wherein the optical-flow detecting means comprises:

means for reverse-projection converting the early image on the basis of a predetermined optical arrangement of the image pick-up means onto an x-z plane in parallel to a road surface in a real space to acquire a road surface image;

means for computing a moving distance of one'"'"'s own vehicle between the two timings on the basis of a time interval between the two timings and speed information of one'"'"'s own vehicle;

means for parallel-shifting the road surface image by the moving distance thus computed;

means for computing angles of the white lines in the vicinity of one'"'"'s own vehicle formed together with a z-axis on the road surface image parallel-shifted to acquire an average angle thereof;

means for rotating the road surface by the average angle around an origin of the x-z coordinate;

means for projection-converting the road surface image after parallel-shifted to acquire an estimated image of the later image in timing;

means for acquiring a differential image between the later image and its estimated image to extract a feature point; and

means for searching a corresponding point of the feature point extracted, thereby detecting the optical flow.

6. An environment monitoring system according to claim 5, wherein the optical flow detecting means searches the corresponding point of the feature point in each the small regions in only a radial direction of the FOE.

7. An environment monitoring system according to claim 1, further comprising means for evaluating the degree of danger which decides dangerous when the degree of danger in a front, rear or diagonal-rear field of view expressed by $D=l=\left(1/N\right)\times$

∑

i=1N

 



li

(where li denotes the magnitude of the optical flow, and N denotes the number of generated optical flows) exceeds a prescribed value.

8. An environment monitoring system according to claim 7, wherein the means for evaluating the degree of danger decides dangerous when the number N of the optical flows generated exceeds a prescribed value.

9. An environment monitoring system according to claim 7, wherein the means for evaluating the degree of danger evaluates the degree of danger for each the small regions.

10. A method for monitoring a relative relation between one'"'"'s own vehicle and another surrounding vehicle comprising the steps of:

picking up two early and later successive images which are located at a front, rear or diagonal-rear position from one'"'"'s own vehicle at two early and later timings and on the basis of the images thus picked-up, detecting white lines of one'"'"'s own vehicle lane on which one'"'"'s own vehicle runs;

setting a monitoring region in one'"'"'s own vehicle lane or an adjacent lane on the basis of the white lines thus detected; and

detecting an optical flow generated from another surrounding vehicle within the monitoring region, wherein the monitoring region setting step includes the sub-steps;

dividing the early image into a plurality of small regions,

extracting an edge which is an candidate of each the white lines for each the small regions using a virtual FOE for each the small regions acquired previously of the early image,

determining each the white lines on the basis of the edge extracted by the edge extracting means, and

extending segments of white lines for each small region determined by the white line determining means to acquire a coordinate of a crossing point thereof and storing that of the virtual FOE for each the small regions whereby the virtual FOE is used to extract the edge for each the small regions.