Stereo image processing device for vehicle

• US 9,378,553 B2
• Filed: 03/24/2015
• Issued: 06/28/2016
• Est. Priority Date: 03/28/2014
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Tamura, Yuichiroh, Natsume, Kensaku
• Primary Examiner(s): Yentrapati, Avinash
• Application Number:
  US14/667,265
• Publication Number:
  US 20150279017A1
• Time in Patent Office: 462 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

In the lane line detection performed by a left-right similar object detection module, changes in brightness in the road width direction in a image plane are evaluated, a point group serving as a lane line candidate is extracted, and a lane line model is generated by calculating the parallax of left and right lane line candidate points and transforming the result into real space coordinates. A camera displacement diagnosis module determines a camera displacement by calculating a diagnostic value based on a difference between the parallaxes of the left and right lane lines in the same horizontal line between reference comparative images captured by a stereo camera unit. Where the camera displacement has occurred, a vehicle control device is instructed to stop the control function or make a transition to failsafe control, and a warning is issued instructing a user to take the vehicle to a maintenance site for inspection.

Claims

1. A stereo image processing device for a vehicle that processes images captured by a stereo camera unit installed on the vehicle and three-dimensionally recognizes a traveling environment, the device comprising:

a left-right similar object detection module that detects a left side object present along a road and right side object which is present along the road and is similar to the left side object present as a pair on the left and right side of the vehicle from an image captured by the stereo camera unit, performs matching processing of the left side object and the right side object, and calculates a parallax of the left side object and a parallax of the right side object on a same horizontal line of the image; and

a camera displacement diagnosis module that calculates a diagnostic value based on a difference between the parallax of the left side object and the parallax of the right side object on the same horizontal line, compares the diagnostic value with a preset threshold value, and diagnoses a vertical mechanical displacement in the stereo camera unit.

2. The stereo image processing device for a vehicle according to claim 1, wherein the left side object and the right side object are left and right lane lines on a road surface.

3. The stereo image processing device for a vehicle according to claim 1, wherein a coordinate value based on a parallax of the left side object at any point and a parallax of the right side object at any corresponding point and a coordinate value based on a distance of the left side object at the any point and a distance of the right side object at the any corresponding point is used as the diagnostic value.

4. The stereo image processing device for a vehicle according to claim 1, wherein a difference between a height of the left side object in a real space and a height of the right side object in the real space is used as the diagnostic value, the height of the left side object and the height of the right side object being determined based on the parallax at a same distance from the vehicle in the real space.

5. The stereo image processing device for a vehicle according to claim 2, wherein a difference between a height of the left side object in a real space and a height of the right side object in the real space is used as the diagnostic value, the height of the left side object and the height of the right side object being determined based on the parallax at a same distance from the vehicle in the real space.

6. The stereo image processing device for a vehicle according to claim 1, wherein an inclination of a straight line obtained by linearly approximating a relationship between the height and distance of each of the left side object and the right side object in a real space is used as the diagnostic value.

7. The stereo image processing device for a vehicle according to claim 2, wherein an inclination of a straight line obtained by linearly approximating a relationship between the height and distance of each of the left side object and the right side object in a real space is used as the diagnostic value.

8. The stereo image processing device for a vehicle according to claim 2, wherein a difference between a height of the left side object in a real space and a height of the right side object in the real space is used as the diagnostic value, the height of the left side object and the height of the right side object being determined based on the parallax at a same distance from the vehicle in the real space.