ROAD VERTICAL CONTOUR DETECTION

• US 20150317525A1
• Filed: 07/14/2015
• Published: 11/05/2015
• Est. Priority Date: 12/06/2011
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon, Shashua, Amnon
• Granted Patent: US 9,826,199 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2552/15 Road slope, i.e. the inclination of a road segment in the longitudinal direction
• CPC Class Codes: B60W 40/076 Slope angle of the road
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Various driver assistance systems mountable in a host vehicle and computerized methods for detecting a vertical deviation of a road surface. The driver assistance system includes a camera operatively connectible to a processor. Multiple consecutive image frames are captured from the camera including a first image of the road and a second image of the road. Based on the host vehicle motion, the second image is warped toward the first image to produce thereby a warped second image. Image points of the road in the first image and corresponding image points of the road in the warped second image are tracked. Optical flow is computed between the warped second image to the first image. The optical flow is compared with an optical flow based on a road surface model to produce a residual optical flow. The vertical deviation is computed from the residual optical flow.

Claims

1. A computerized method for detecting a vertical deviation of a road surface, the method performed by a driver assistance system mountable in a host vehicle while the host vehicle is moving, wherein the driver assistance system includes a camera operatively connectible to a processor, the method comprising:

capturing from the camera a plurality of consecutive image frames including a first image of the road and a second image of the road;

based on the host vehicle motion,warping the second image toward the first image to produce thereby a warped second image;

tracking image points of the road in the first image and corresponding image points of the road in the warped second image;

computing optical flow between the warped second image to the first image;

comparing the optical flow with an optical flow based on a planar or bi-quadratic road surface model of the road to produce thereby a residual optical flow; and

computing the vertical deviation of the road surface from said residual optical flow.

2. The method of claim 1, further comprising, prior to the tracking:

selecting a grid of a plurality of image points in the first image.

3. The method of claim 2, further comprising:

locating a plurality of image patches around the image points of the selected grid and performing the tracking between the image patches in the first image and corresponding second image patches in the warped second image.

4. The method of claim 1, wherein said warping includes:

aligning said second image toward said first image by adjusting for at least one image shift due to at least one motion of the vehicle relative to the road, wherein said at least one motion is selected from the group consisting of;

yaw, pitch and roll; and

adjusting for relative scale change between said second image and said first image, wherein said relative scale change arises from different distances to the camera.

5. The method of claim 1, further comprising:

fitting the tracked points to a homography.

6. The method of claim 5, further comprising:

using the homography to correct the warp of the second image toward the first image.

7. The method of claim 1, wherein said image points are located in a fixed distribution on the image of the road surface.

8. The method of claim 5, further comprising:

refined warping said warped second image frame toward said first image frame thereby correcting said warping by using said homography and producing a refinely warped second image frame.

9. The method of claim 8, further comprising:

computing optical flow between said refinely warped second image frame and said first image frame;

prior to said comparing said optical flow with said optical flow based on the surface model.

10. A driver assistance system mountable in a host vehicle, wherein the driver assistance system includes a camera operatively connectible to a processor, wherein the driver assistance system operates while the host vehicle is moving to detect a vertical deviation of a road surface, the driver assistance system configured to:

capture from the camera a plurality of consecutive image frames including a first image of the road and a second image of the road;

based on the host vehicle motion, warp the second image toward the first image to produce a warped second image;

track image points of the road in the first image and corresponding image points of the road in the warped second image;

estimate optical flow between the warped second image to the first image;

compare the optical flow with an optical flow based on a planar or bi-quadratic surface model of the road to produce a residual optical flow; and

compute the vertical deviation of the road surface from said residual optical flow.

11. The driver assistance system of claim 10, further configured to:

select a grid of a plurality of image points in the first image.

12. The driver assistance system of claim 11, further configured to:

locate a plurality of image patches around the image points of the selected grid and performing the tracking between the image patches in the first image and corresponding second image patches in the warped second image.

13. The driver assistance system of claim 10, wherein the warp aligns the second image toward the first image.

14. The driver assistance system of claim 13, wherein the warp adjusts for at least one image shift due to at least one motion of the vehicle relative to the road, wherein said at least one motion is selected from the group consisting of:

yaw, pitch and roll.

15. The driver assistance system of claim 14 wherein the warp adjusts for relative scale change between said second image and said first image, wherein said relative scale change arises from different distances to the camera.

16. The driver assistance system of claim 13, further configured to:

fit the tracked points to a homography.

17. The driver assistance system of claim 14, further configured to:

use the homography to correct the warp of the second image toward the first image.