Run environment recognizing apparatus

• US 6,191,704 B1
• Filed: 12/19/1997
• Issued: 02/20/2001
• Est. Priority Date: 12/19/1996
• Status: Expired due to Term

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Takenaga, Hiroshi, Kuroda, Hiroshi, Yoshikawa, Tokuji, Kayano, Mitsuo, Kuragaki, Satoru, Takano, Kazuaki, Morizane, Hiroto
• Primary Examiner(s): SWARTHOUT, BRENT
• Application Number:
  US08/994,750
• Time in Patent Office: 1,159 Days
• Field of Search: 340/903, 340/901, 340/904, 340/435, 348/148, 348/149, 348/170, 348/169, 180/169, 701/117, 356/4.01, 356/4.03
• US Class Current: 340/903
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: G01S 13/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S13/46
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/932 using own vehicle data, e.g. ground speed, steering wheel direction
• CPC Class Codes: G01S 2013/9321 Velocity regulation, e.g. cruise control
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/9325 for inter-vehicle distance regulation, e.g. navigating in platoons
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS
• CPC Class Codes: G08G 1/09623 Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle G08G1/0967 takes precedence
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

Described herein is a run environment recognizing apparatus. The run environment recognizing apparatus comprises an imaging device for sensing the direction of traveling of a host vehicle as imaging vision, a radar for detecting the direction of traveling of the host vehicle as a detection range, an image processing device for recognizing traveling lanes, leading vehicles and stationary objects located ahead of the host vehicle, based on the image information sensed by the imaging device and the result of detection by the radar, and a display for displaying the results of recognition by the image processing device. The image processing device includes means for setting vehicle regions indicative of the leading vehicles within the image information, based on the result of detection by the radar. Thus, the influence of the leading vehicles in the process for recognizing the traveling lanes within the image information is eliminated by the vehicle regions.

Claims

1. A run environment recognizing apparatus comprising:

an imaging device for sensing the direction of traveling of a host vehicle as imaging vision;

a radar for detecting the direction of traveling of the host vehicle as a detection range;

an image processing device for recognizing traveling lanes, leading vehicles and stationary objects located ahead of the host vehicle, based on the image information sensed by said imaging device and the result of detection by said radar; and

a display for displaying the results of recognition by said image processing device;

said image processing device including means for setting vehicle regions indicative of the leading vehicles within said image information, based on the result of detection by said radar;

whereby the influence of the leading vehicles in the process for recognizing the traveling lanes within said image information is eliminated by said vehicle regions, further including a road map device for associating a vehicle position obtained from a vehicle position detecting device using satellite radio waves with a road map,

wherein said image processing device receives information about an intersection ahead of the host vehicle while running, from said road map device, estimates positions of a traffic signal and a pedestrian crossing within the image obtained from said imaging device in the vicinity of the intersection to thereby recognize the state of the traffic signal, pedestrians and right- and left-turned vehicles and display them on said display.

2. A run environment recognizing apparatus according to claim 1, wherein an imaging optical axis of said imaging device and a detection axis of said radar are placed on the same axis.

3. A run environment recognizing apparatus according to claim 1, wherein said imaging device is provided on the driver'"'"'s seat side of the vehicle and records an image extending in the viewpoint direction of a driver by photography.

4. A run environment recognizing apparatus according to claim 1, wherein said imaging device is placed on the side opposite to the driver'"'"'s seat side of the vehicle and records images including portions invisible from the driver by photography.

5. A run environment recognizing apparatus comprising:

an imaging device for sensing the direction of traveling of a host vehicle as imaging vision;

a radar for detecting the direction of traveling of the host vehicle as a detection range;

an image processing device for recognizing traveling lanes, leading vehicles and stationary objects located ahead of the host vehicle, based on the image information sensed by said imaging device and the result of detection by said radar; and

a display for displaying the results of recognition by said image processing device;

said image processing device including means for setting vehicle regions indicative of the leading vehicles within said image information, based on the result of detection by said radar;

whereby the influence of the leading vehicles in the process for recognizing the traveling lanes within said image information is eliminated by said vehicle regions,

wherein said image processing device associates the position of the host vehicle with that of a leading vehicle in the image information of said imaging device in accordance with the result of detection of distances and bearings by said radar, examines luminances or colors of horizontal pixels within said leading vehicle position, measures horizontal widths of said vehicle regions by integration of the pixels closely analogous to each other in luminance or color, and determines the type (large, middle or small size) of vehicle from the horizontal widths and the distances to thereby estimate the height of said each vehicle region.

6. A run environment recognizing apparatus comprising:

an imaging device for sensing the direction of traveling of a host vehicle as imaging vision;

a radar for detecting the direction of traveling of the host vehicle as a detection range;

an image processing device for recognizing traveling lanes, leading vehicles and stationary objects located ahead of the host vehicle, based on the image information sensed by said imaging device and the result of detection by said radar; and

a display for displaying the results of recognition by said image processing device;

said image processing device including means for setting vehicle regions indicative of the leading vehicles within said image information, based on the result of detection by said radar;

whereby the influence of the leading vehicles in the process for recognizing the traveling lanes within said image information is eliminated by said vehicle regions,

wherein said image processing device sets a vertical line segment to the center of the screen used for said image information, and examines a change in luminance of each pixel from the vertical line segment to the horizontal direction, thereby detecting edges of white lines drawn on a road surface and creating white line models, using position coordinates of the edges,

wherein said image processing device displaces a small white-line tracking region along said created white line model and changes said white line model so as to fit to the inclination of the white-line edge in said white-line tracking region.

7. A run environment recognizing apparatus comprising:

an imaging device for sensing the direction of traveling of a host vehicle as imaging vision;

a radar for detecting the direction of traveling of the host vehicle as a detection range;

an image processing device for recognizing traveling lanes, leading vehicles and stationary objects located ahead of the host vehicle, based on the image information sensed by said imaging device and the result of detection by said radar; and

a display for displaying the results of recognition by said image processing device;

said image processing device including means for setting vehicle regions indicative of the leading vehicles within said image information, based on the result of detection by said radar;

whereby the influence of the leading vehicles in the process for recognizing the traveling lanes within said image information is eliminated by said vehicle regions,

wherein said image processing device sets a vertical line segment to the center of the screen used for said image information, and examines a change in luminance of each pixel from the vertical line segment to the horizontal direction, thereby detecting edges of white lines drawn on a road surface and creating white line models, using position coordinates of the edges,

wherein said image processing device displaces said white-line tracking region to an adjacent white-line model that pairs up with said white-line model so as to continue a white-line model fitting process where said white-line tracking region collides with the vehicle region when the white-line model is varied so as to fit to the inclination of the white-line edge in said white-line tracking region.

8. A run environment recognizing apparatus according to claim 7, wherein said image processing device examines the movement of a disappearing point calculated from the white line model varied so as to fit to the inclination of the white-line edge in said white-line tracking region, determines that a curve exists forward when the amount of movement exceeds a predetermined value, and determines the curve as a right or left curve according to the direction of movement of said disappearing point.

9. A run environment recognizing apparatus comprising:

an imaging device for sensing the direction of traveling of a host vehicle as imaging vision;

a radar for detecting the direction of traveling of the host vehicle as a detection range;

an image processing device for recognizing traveling lanes, leading vehicles and stationary objects located ahead of the host vehicle, based on the image information sensed by said imaging device and the result of detection by said radar; and

a display for displaying the results of recognition by said image processing device;

said image processing device including means for setting vehicle regions indicative of the leading vehicles within said image information, based on the result of detection by said radar;

whereby the influence of the leading vehicles in the process for recognizing the traveling lanes within said image information is eliminated by said vehicle regions,

wherein the display of the results of recognition by said image processing device is configured so as to result in displaying emphasized by either a display for putting the coloring of the corresponding recognized portion in the image displayed on said display to a predetermined color, or a display for allowing the recognized portion to be surrounded by frame graphics,

further including a road map device for associating a vehicle position obtained from a vehicle position detecting device using satellite radio waves with a road map,

wherein said image processing device receives information about an intersection ahead of the host vehicle while running, from said road map device, estimates positions of a traffic signal and a pedestrian crossing within the image obtained from said imaging device in the vicinity of the intersection to thereby recognize the state of the traffic signal, pedestrians and right- and left-turned vehicles and display them on said display.

10. A run environment recognizing apparatus, comprising:

an imaging device for sensing the direction of traveling of a host vehicle as imaging vision;

a radar for detecting the direction of traveling of the host vehicle as a detection range;

an image processing device for recognizing traveling lanes, leading vehicles and stationary objects located ahead of the host vehicle, based on the image information sensed by said imaging device and the result of detection by said radar; and

a display for displaying the results of recognition by said image processing device;

said image processing device including means for setting vehicle regions indicative of the leading vehicles within said image information, based on the result of detection by said radar;

whereby the influence of the leading vehicles in the process for recognizing the traveling lanes within said image information is eliminated by said vehicle regions,

wherein the display of the results of recognition by said image processing device is configured so as to result in displaying emphasized by either a display for putting the coloring of the corresponding recognized portion in the image displayed on said display to a predetermined color, or a display for allowing the recognized portion to be surrounded by frame graphics,

wherein said image processing device associates the position of the host vehicle with that of a leading vehicle in the image information of said imaging device in accordance with the result of detection of distances and bearings by said radar, examines luminances or colors of horizontal pixels within said leading vehicle position, measures horizontal widths of said vehicle regions by integration of the pixels closely analogous to each other in luminance or color, and

determines the type (large, middle or small size) of vehicle from the horizontal widths and the distances to thereby estimate the height of said each vehicle region.

11. A run environment recognizing apparatus according to claim 10, further including a road map device for associating a vehicle position obtained from a vehicle position detecting device using satellite radio waves with a road map,

wherein said image processing device receives information about an intersection ahead of the host vehicle while running, from said road map device, estimates positions of a traffic signal and a pedestrian crossing within the image obtained from said imaging device in the vicinity of the intersection to thereby recognize the state of the traffic signal, pedestrians and right- and left-turned vehicles and display them on said display.

12. A run environment recognizing apparatus, comprising:

an imaging device for sensing the direction of traveling of a host vehicle as imaging vision;

a radar for detecting the direction of traveling of the host vehicle as a detection range;

an image processing device for recogizing traveling lanes, leading vehicles and stationary objects located ahead of the host vehicle, based on the image information sensed by said imaging device and the result of detection by said radar; and

a display for displaying the results of recognition by said image processing device;

said image processing device including means for setting vehicle regions indicative of the leading vehicles within said image information, based on the result of detection by said radar;

whereby the influence of the leading vehicles in the process for recognizing the traveling lanes within said image information is eliminated by said vehicle regions,

wherein said image processing device sets a vertical line segment to the center of the screen used for said image information, and examines a change in luminance of each pixel from the vertical line segment to the horizontal direction, thereby detecting edges of white lines drawn on a road surface and creating white line models, using position coordinates of the edges,

further including a road map device for associating a vehicle position obtained from a vehicle position detecting device using satellite radio waves with a road map,

wherein said image processing device receives information about an intersection ahead of the host vehicle while running, from said road map device, estimates positions of a traffic signal and a pedestrian crossing within the image obtained from said imaging device in the vicinity of the intersection to thereby recognize the state of the traffic signal, pedestrians and right- and left-turned vehicles and display them on said display.

13. A run environment recognizing apparatus according to claim 12, wherein the display of the results of recognition by said image processing device is configured so as to result in displaying emphasized by either a display for putting the coloring of the corresponding recognized portion in the image displayed on said display to a predetermined color, or a display for allowing the recognized portion to be surrounded by frame graphics.

14. A run environment recognizing apparatus according to claim 13, wherein said image processing device associates the position of the host vehicle with that of a leading vehicle in the image information of said imaging device in accordance with the result of detection of distances and bearings by said radar, examines luminances or colors of horizontal pixels within said leading vehicle position, measures horizontal widths of said vehicle regions by integration of the pixels closely analogous to each other in luminance or color, and

determines the type (large, middle or small size) of vehicle from the horizontal widths and the distances to thereby estimate the height of said each vehicle region.

15. A run environment recognizing apparatus according to claim 12, wherein said image processing device displaces a small white-line tracking region along said created white line model and changes said white line model so as to fit to the inclination of the white-line edge in said white-line tracking region.

16. A run environment recognizing apparatus according to claim 15, wherein said image processing device displaces said white-line tracking region to an adjacent white line model that pairs up with said white line model so as to continue a white-line model fitting process where said white-line tracking region collides with the vehicle region when the white line model is varied so as to fit to the inclination of the white-line edge in said white-line tracking region.