Driving assistant apparatus of vehicle and operating method thereof

• US 9,582,867 B2
• Filed: 10/14/2014
• Issued: 02/28/2017
• Est. Priority Date: 07/09/2014
• Status: Active Grant

• Current Assignee: Hyundai Mobis Co., Ltd., Industry-University Cooperation Foundation Hanyang University
• Original Assignee: Hyundai Mobis Co., Ltd., Industry-University Cooperation Foundation Hanyang University
• Inventors: Suhr, Jae Kyu, Jung, Ho Gi
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): DHILLON, PUNEET S
• Application Number:
  US14/513,287
• Publication Number:
  US 20160014394A1
• Time in Patent Office: 868 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20061 Hough transform
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 5/10 using non-spatial domain filtering
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Disclosed is an apparatus for assisting driving, including: a stereo camera module mounted in a vehicle, and configured to obtain a first image through a first camera and a second image through a second camera; and a processor configured to stereo-match the first image and the second image to obtain a dense disparity map, estimate a cubic b-spline curved line from the dense disparity map by using a coarse-to-fine method, and perform a road profile for two-dimensionally converting a 3D road, on which the vehicle travels, by using the cubic b-spline curved line.

Claims

1. An apparatus for assisting driving, comprising:

a stereo camera module mounted in a vehicle, and configured to obtain a first image through a first camera and a second image through a second camera; and

a processor configured to stereo-match the first image and the second image to obtain a dense disparity map, estimate a cubic b-spline curved line from the dense disparity map by using a coarse-to-fine method, and perform a road profile for two-dimensionally converting a 3D road, on which the vehicle travels, by using the cubic b-spline curved line,

wherein the processor includes;

a stereo-matching unit configured to stereo-match the first image and the second image to obtain the dense disparity map; and

a road surface estimating unit configured to estimate the cubic b-spline curved line from the dense disparity map by using the coarse-to-fine method,

wherein the road surface estimating unit includes;

a pixel processing unit configured to three-dimensionally restore all of the pixels of the dense disparity map, and accumulate the three-dimensionally restored pixels on an YZ plane;

a piece-wise linear function estimating unit configured to estimate a piece-wise linear function from a result of the accumulation; and

a cubic b-spline curved line estimating unit configured to estimate a cubic b-spline curved line based on the estimated piece-wise linear function.

2. The apparatus of claim 1, wherein the piece-wise linear function estimating unit includes:

a Hough Transform applying unit configured to apply Hough Transform to the 3D points existing in the result of the accumulation to obtain a voting result; and

a dynamic programming performing unit configured to set the voting result as a data term, set a connection relationship between straight lines of each section and a change in an inclination of the straight lines of each section as a smoothness term, and perform dynamic programming to estimate the piece-wise linear function.

3. The apparatus of claim 1, wherein the piece-wise linear function estimating unit estimates the piece-wise linear function by Equation 2:

Y=tan(θ

_n)·

Z+ρ

_n,Z<sub>n−

1</sub>≦

Z<

Z_n 

 

[Equation 2]

where Z<sub>n−

1</sub> to Z_n means an n^th section, and θ

_n and p_n mean an angle of a straight line existing in the n^th section and a segment of a Y axis, respectively, and in this case, p_n is calculated based on a position at which each straight line starts.

4. The apparatus of claim 1, wherein the pixel processing unit divides the pixels, which are accumulated on the YZ-plane, for each predetermined section of a distance of a road included in the dense disparity map.

5. The apparatus of claim 1, further comprising:

a display unit configured to display an image received by the processor.

6. The apparatus of claim 5, wherein the display unit is a cluster or a head up display (HUD) on a front surface inside the vehicle.

7. The apparatus of claim 1, further comprising:

an input unit configured to receive a user input through a plurality of buttons or a touch screen.

8. The apparatus of claim 1, further comprising:

a memory configured to store a program for processing or control by the processor.

9. A vehicle comprising the apparatus for assisting driving of claim 1.

10. A method of operating an apparatus for assisting driving for a vehicle including a stereo camera module, which obtains a first image through a first camera and a second image through a second camera, the method comprising:

stereo-matching the first image and the second image to obtain a dense disparity map;

estimating a cubic b-spline curved line from the dense disparity map by using a coarse-to-fine method; and

performing a road profile for two-dimensionally converting a 3D road, on which the vehicle travels, by using the cubic b-spline curved line,

wherein the estimating includes;

three-dimensionally restoring all of the pixels of the dense disparity map;

accumulating the three-dimensionally restored pixels on an YZ plane;

estimating a piece-wise linear function from a result of the accumulation; and

estimating a cubic b-spline curved line based on the estimated piece-wise linear function.

11. The method of claim 10, wherein the estimating of the piece-wise linear function includes:

applying Hough Transform to the 3D points existing in the result of the accumulation to obtain a voting result; and

setting the voting result as a data term, setting a connection relationship between straight lines of each section and a change in an inclination of the straight lines of each section as a smoothness term, and performing dynamic programming to estimate the piece-wise linear function.

12. The method of claim 10, wherein the estimating includes estimating the piece-wise linear function by Equation 2:

Y=tan(θ

_n)·

Z+ρ

_n,Z<sub>n−

1</sub>≦

Z<

Z_n 

 

[Equation 2]

where Z<sub>n−

1</sub> to Z_n means an n^th section, and θ

_n and p_n mean an angle of a straight line existing in the n^th section and a segment of a Y axis, respectively, and in this case, p_n is calculated based on a position at which each straight line starts.

13. The method of claim 10, further comprising:

dividing the pixels, which are accumulated on the YZ-plane, for each predetermined section of a distance of a road included in the dense disparity map.

14. The method of claim 10, further comprising:

displaying an image on which the road profile is performed.