Road-shoulder detecting device and vehicle using road-shoulder detecting device

• US 8,873,803 B2
• Filed: 08/09/2010
• Issued: 10/28/2014
• Est. Priority Date: 12/28/2009
• Status: Active Grant

• Current Assignee: Hitachi Astemo Ltd. (Hitachi, Ltd.)
• Original Assignee: Hitachi Automotive Systems Limited (Hitachi, Ltd.)
• Inventors: Shima, Takeshi, Higuchi, Mirai, Muramatsu, Shoji, Monji, Tatsuhiko, Naka, Takuya
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US13/519,347
• Publication Number:
  US 20120288154A1
• Time in Patent Office: 1,541 Days
• Field of Search: 382/103, 382/104, 382/106, 382/107, 382/108, 382/278, 340/907, 340/933
• US Class Current: 382/103
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

Disclosed is a road-shoulder detecting device including a distance-information calculating portion for calculating the presence of a physical object and the distance from the subject vehicle to the object from input three-dimensional image information relating to an environment around the vehicle, a vehicular road surface detecting portion for detecting a vehicular road surface with the subject vehicle from a distance image, a height difference calculating portion for measuring height difference between the detected vehicular road and an off-road region, and a road shoulder decision portion for deciding height difference as to whether the road shoulder is boundary between the surface and the region in a case where there is an off-road region lower than the vehicular road surface.

Claims

1. A road-shoulder detecting device comprising:

a distance information calculating portion that calculates a presence or absence of a physical object and a distance from a subject vehicle to the object from input three-dimensional image information about an environment outside of the vehicle;

a vehicular road surface detecting portion that detects a vehicular road surface traveled by the subject vehicle from a distance image based on the calculated distance;

a height difference calculating portion that measures a height difference between the detected vehicular road surface and an off-road region; and

a discontinuity calculating portion that;

i) separates a distance image based on the calculated distance into a plurality of image regions, ii) calculates whether a distance between any two of the image regions is equal to or greater than a predetermined threshold value, and iii) judges that the object is a road shoulder because a case in which the predetermined threshold value is reached or exceeded is regarded as discontinuous; and

a road shoulder decision portion that makes a decision, based on the height difference measured by the height difference calculating portion or based on the road shoulder calculated by the discontinuity calculation portion, as to whether a road shoulder being a boundary between the vehicular road surface and the off-road region is a road shoulder occurring in a case where there is an off-road region lower than the vehicular road surface.

2. The road-shoulder detecting device of claim 1, comprising a vehicle departure decision portion that makes a decision as to whether or not the subject vehicle will run off the road based on the result of the decision made by the road shoulder decision portion, on a detected speed of the subject vehicle, and on a steering angle.

3. A vehicle having the road-shoulder detecting device of claim 2, further comprising a warning need decision portion that makes a decision based on the result of the decision made by said vehicle departure decision portion as to whether it is necessary to issue a warning to the driver about running off the road.

4. The vehicle having the road-shoulder detecting device of claim 2, comprising an automatic control decision portion that makes a decision based on the result of the decision made by said vehicle departure decision portion as to whether automatic steering control or automatic brake control that prevents the vehicle from running off the road is needed.

5. The road-shoulder detecting device of claim 1, wherein left image information and right image information about the environment outside of the vehicle is applied to said distance information calculating portion which calculates the presence or absence of the object and the distance from the subject vehicle to the object from three-dimensional image information based on the input left and right image information.

6. The road-shoulder detecting device of claim 5, wherein said distance information calculating portion has a corresponding point calculating processing portion that compares the input left and right image information to identify portions of the images taken from the same object, a distance calculating portion that calculates a distance to said same object identified from imaging devices that obtain image information, and a distance information output processing portion that creates distance information based on the calculated distance and outputting it.

7. The road-shoulder detecting device of claim 1, wherein said vehicular road surface detecting portion has a three-dimensional information calculation processing portion that calculates three-dimensional positions of pixels from the distance image based on the distance calculated by said distance information calculating portion and a vehicular road equation application processing portion that extracts pixels that are highly likely to be the vehicular road surface based on the calculated three-dimensional positions and detecting the vehicular road surface.

8. The road-shoulder detecting device of claim 7, wherein said height difference calculating portion measures the height difference between said vehicular road surface and the off-road region based on the three-dimensional positions calculated by said three-dimensional information calculation processing portion and on the vehicular road surface detected by said vehicular road surface equation application processing portion.

9. The road-shoulder detecting device of claim 1, wherein said road shoulder decision portion extracts off-road regions lower than said vehicular road surface based on said height difference, extracts contour elements regarding all pixels within the extracted off-road regions, takes the extracted contour elements as contour lines of the off-road regions, and judges from the contour lines road shoulders occurring in a case where off-road regions are lower than the vehicular road surface.

10. The road-shoulder detecting device of claim 1, further comprising a vehicle departure decision portion that makes a decision as to whether the subject vehicle will run off the road based on the result of the decision made by said road shoulder decision portion and on a detected speed of the subject vehicle and steering angle.

11. The road-shoulder detecting device of claim 1, wherein left image information and right image information about the environment outside of the vehicle is applied to said distance information calculating portion which calculates the presence or absence of the object and the distance from the subject vehicle to the object from three-dimensional image information based on the input left and right image information.

12. The road-shoulder detecting device of claim 11, wherein said distance information calculating portion has a corresponding point calculating processing portion that compares the input left and right image information to identify portions of the images taken from the same object, a distance calculating portion that calculates a distance to said same object identified from imaging devices that obtain image information, and a distance information output processing portion that creates distance information based on the calculated distance and outputting it.