Automobile apparatus for road lane and vehicle ahead detection and ranging

• US 5,555,312 A
• Filed: 04/28/1994
• Issued: 09/10/1996
• Est. Priority Date: 06/25/1993
• Status: Expired due to Term

• Current Assignee: Fujitsu Ten Limited (DENSO Corporation), Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Kamada, Hiroshi, Shima, Nobukazu
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Johns, Andrew W.
• Application Number:
  US08/234,676
• Time in Patent Office: 866 Days
• Field of Search: 382/1, 382/22, 382/48, 382/104, 382/199, 382/291, 382/106, 364/424.02, 364/436, 364/460, 364/461, 340/903, 340/933, 340/937, 348/118, 348/119, 348/140, 348/148
• US Class Current: 382/104
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An apparatus recognizes road traffic lines and a car running ahead without regard to obstacles and noise signals. A video image recording unit on a car provides an image of the road ahead. Horizontal edges in the image are detected for determining the presence of a car ahead. Successive lateral sweeps in both directions from a center line that advance at predetermined sampling intervals from a proximal portion of the image toward a distal portion of the image provide data by which horizontal edges are determined for comparison against predetermined standards. A base line on the road for the car ahead is determined as the most proximal horizontal edge. Lines marking road lanes are detected by similar two directional search lines from a center line. Lane lines are identified in right and left proximal portions of the image and extrapolated to the distal portion of the image. Missing portions of lines are interpolated. In determining the presence of marked road lanes, vertical edge detection provides basic data for establishing the thickness and distance between probable lines for comparison against predetermined standards. Comparison of successive road images determines relative motion between the vehicle bearing the apparatus and detected images and aids in updating the image and eliminating noise signals.

Claims

1. An apparatus for recognizing road environment, comprising:

road image recording means mounted on a car to pick up an image of a road;

edge imaging means for preparing an edge image showing contours of objects within said road image;

base line decision means for detecting probable base lines having long horizontal edges in the edge image by moving a search line in a predetermined range and for selecting one of the probable base lines as a baseline of a car running ahead, which baseline is a boundary between said car running ahead and the road, according to an arrangement of the detected horizontal edges, said base line decision means moving the search line at a predetermined sampling interval from a proximal end toward a distal end in the edge image, and determining the search lines containing at least a predetermined number of pixels, each pixel containing a horizontal edge component, as probable base lines by detecting pixel values along the search line;

search area decision means for determining horizontal positions of left and right edges of the base line and setting a search area including the base line and the horizontal positions of left and right edges of the base line; and

ahead car position determining means for determining a position in said search area of the car running ahead, the ahead car position determining means determining the horizontal positions of the car ahead based upon said horizontal edge components in the search area wherein said pixels are counted by count means.

2. The apparatus according to claim 1, wherein the search area has predetermined height and width and a lower edge of the search area coincides with the base line.

3. The apparatus according to claim 2, wherein the position determining means determines the position of the car ahead according to a distribution of horizontal edges found in the search area.

4. The apparatus according to claim 3, wherein the position determining means determines the position of the car ahead according to vertical projection values of pixels that exist along the horizontal edges found in the search area and respectively contain horizontal edge components.

5. The apparatus according to claim 3, wherein the means for determining a position of the car running ahead prepares a recognition frame of predetermined size in the search area, moves the recognition frame in the search area, and determines a position where the recognition frame involves the largest number of pixels each containing a horizontal edge component as the position of the car running ahead.

6. The apparatus according to claim 5, wherein the sizes of the search area and the recognition frame are determined according to setting conditions of the road image record means such that the scaling of an actual object corresponding to the search area is constant.

7. The apparatus according to claim 3, wherein the means for determining a position of the car running ahead prepares a recognition frame of predetermined size in the search area, moves the recognition frame in the search area, and determines a position where the recognition frame shows the highest symmetry along a vertical center line thereof as the position of the car running ahead.

8. The apparatus according to claim 3, wherein the means for determining a position of the car running ahead prepares a recognition frame of predetermined size in the search area, moves the recognition frame in the search area, detects at each position the number of pixels each containing a horizontal edge component as well as the symmetry of the recognition frame along a vertical center line thereof, provides as evaluation value of the recognition frame at each position according to the symmetry, and determines a position that provides the highest evaluation value as the position of the car running ahead.

9. The apparatus according to claim 1, further comprising means for calculating a distance to the car ahead according to the base line of the car ahead.

10. The apparatus according to claim 1, further comprising means for recognizing traffic lines for defining lanes and searching the position of the car running ahead in the lanes.

11. An apparatus for recognizing road environment, comprising:

road image recording means mounted on a car to pick up an image of a road;

edge imaging means for preparing an edge image showing contours of objects within said road image;

base line decision means for detecting probable base lines having long horizontal edges in the edge image by moving a search line in a predetermined range and for selecting one of the probable base lines as a baseline of a car running ahead, which baseline is a boundary between said car running ahead and the road, according to an arrangement of the detected horizontal edges, said base line decision means moving the search line at a predetermined sampling interval from a proximal end toward a distal end in the edge image, and determining the search lines containing at least a predetermined number of pixels, each pixel containing a horizontal edge component, as probable base lines by detecting pixel values along the search line, said base line decision means determining that a car running ahead exists when a plurality of probable base lines exist and selecting the lowest probable base line in said road image as the base line of the car running ahead;

search area decision means for determining horizontal positions of left and right edges of the base line and for setting a search area according to the base line and the horizontal positions of left and right edges of the base line; and

position determining means for determining a position of the car running ahead, the ahead car position determining means determining the horizontal positions of the car ahead based upon said horizontal edge components in the search area counted by count means.

12. An apparatus for recognizing road environment, comprising:

road image recording means mounted on a car to pick up an image of a road;

edge imaging means for preparing an edge image showing contours of objects according to the road image;

proximal line recognition means for recognizing traffic lines in a proximal part of said edge image; and

distal line recognition means for recognizing traffic lines in a distal part of the edge image,

wherein the proximal line recognition means halves the proximal part of the edge image into a proximal left area and a proximal right area, leftwardly searches the proximal left area for probable points to determine as edge points of a left traffic line, rightwardly searches the proximal right area for probable points to determine as edge points of a right traffic line, recognizes traffic lines according to the probable points, and

the distal line recognition means tracing the traffic lines recognized by the proximal line recognition means in a direction extended into the distal part, and completely constructs the traffic lines.

13. The apparatus according to claim 12, wherein proximal line recognition means horizontally scans pixels with a spatial filter in the road image to obtain absolute values of the pixels, determines any pixels whose absolute values exceed a predetermined threshold as edge points, and prepares a traffic line image in which the edge points are connected to form a traffic line.

14. The apparatus according to claim 12, wherein the traffic line image is formed of positive edges including pixels each having a positive value and an absolute value greater than the threshold, and negative edges including pixels each having a negative value and an absolute value greater than the threshold.

15. The apparatus according to claim 14, wherein the proximal line recognition means recognizes pairs of positive and negative neighboring edges, whose distances apart are within a predetermined range corresponding to a traffic line width as probable points.

16. The apparatus according to claim 15, wherein the proximal line recognition means recognizes pairs of edges whose distances apart are within a predetermined range corresponding to a lane width.

17. The apparatus according to claim 16, wherein the proximal line recognition means determines whether or not recognized lines are narrow traffic lines or wide traffic lines, and recognizes points interpolated from the wide traffic lines at a predetermined interval as probable points.

18. The apparatus according to claim 16, wherein the proximal line recognition means recognizes lines connected with probable points included in a group of pairs of edges whose distances apart are within the predetermined range as traffic lines when a number of the probable points included in the group exceeds a preset value.

19. The apparatus according to claim 18, wherein the proximal line recognition means recognizes the traffic lines by interpolating the lines connected with the probable points included in the groups when a number of groups exists.

20. The apparatus according to claim 15, wherein the proximal line recognition means divides the probable points into groups in which horizontal differences between longitudinally neighboring probable points are less than a predetermined threshold and horizontal distances between horizontally neighboring probable points are within a predetermined range, and recognizes lines connected with the probable points in respective groups as traffic lines when a number of the probable points included in these groups exceeds a preset value.

21. The apparatus according to claim 20, wherein the proximal line recognition means recognizes a group including the largest number of the probable points as the traffic line when a number of the groups exists.

22. The apparatus according to claim 15, wherein the traffic line is white.

23. The apparatus according to claim 12, wherein when no traffic line is determined in one of the proximal left or right area, the proximal line recognition means estimates points in the area with no traffic line at positions spaced apart from the traffic line determined in the other proximal area by a predetermined lane width and recognizes lines connected with the probable points estimated as the traffic line.

24. The apparatus according to claim 12, wherein the proximal line recognition means defines a horizontal line nearest to the image recording means as a nearest line, and when any one of traffic lines determined in the proximal left and right areas has no common point on the nearest line, extends the traffic line up to the nearest line.

25. The apparatus according to claim 12, wherein when the traffic lines are discontinuous in the distal area, the distal line recognition means traces the traffic lines for a predetermined interval beyond the discontinuous parts, and continues the tracing by interpolating the discontinuous parts when the traffic lines exist beyond the discontinuous parts.

26. The apparatus according to claim 12, further comprising traffic line position calculating means for calculating positions of traffic lines, the traffic line position calculating means calculates widths on the image corresponding to widths of the traffic lines and lanes according to the setting conditions of the record image recording means and actual widths of the traffic lines and lanes, and confirms the positions of the traffic lines according to the calculated widths.

27. An apparatus for recognizing road environment, comprising:

road image recording means mounted on a car to pick up an image of a road;

image smoothing means which obtains a first road image through the road image recording means, obtains a second road image through the road image recording means, compares the first and second road images with each other, leaves the first road image as it is when a difference between pixel values at the same coordinate position in the first and second road images is within a preset range, updates the pixel value of the first road image by bringing the pixel value of the first road image closer to the pixel value of the second road image by a preset value when the difference is out of the preset range, and repeats the updating process to form a smoothed image;

edge imaging means for preparing an edge image showing contours of objects in the road image; and

base line decision means for detecting horizontal edges in the edge image by moving a search line and determining a base line of a car running ahead, said base line being a boundary between a car running ahead and the road, according to an arrangement of the detected horizontal edges.

28. The apparatus according to claim 27, wherein the image smoothing means brings the pixel value of the first road image closer to the pixel value of the second road image by a value based on the difference between the pixel values of the first and second road images when the difference is outside the preset range.