Road vertical contour detection

• US 9,256,791 B2
• Filed: 11/26/2014
• Issued: 02/09/2016
• Est. Priority Date: 12/04/2012
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Shashua, Amnon, Stein, Gideon
• Primary Examiner(s): Chevalier, Robert
• Application Number:
  US14/554,500
• Publication Number:
  US 20150086080A1
• Time in Patent Office: 440 Days
• Field of Search: 348/148, 348/143, 348/147, 348/149, 348/153, 348/154, 348/159, 386/226, 386/248
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A driver assistance/control system includes a camera operatively connectable to a processor mountable in a host vehicle. A vertical deviation in road contour is detected while the host vehicle is moving. First, second and third images of the road are captured from the camera. By matching image points of the road in the first image and corresponding image points of the road in the second image, a first homography is computed which transforms the first image of the road to the second image of the road. A second homography is computed which transforms the second image of the road to the third image of the road. A chained homography is computed by chaining the first and second homographies. By using the chained homography as an initial guess, a third homography is computed which transforms the first image of the road to the third image of the road.

Claims

1. A computerized method for detecting a vertical deviation in contour of a road, the method performable by a driver assistance system mountable in a host vehicle while the host vehicle is moving, wherein the driver assistance system includes a camera operatively connectable to a processor, the method comprising:

capturing from the camera a first image of the road at a first time, a second image of the road at a second time later than the first time, and a third image of the road at a third time later than the second time;

by matching image points of the road in the first image and corresponding image points of the road in the second image, computing a first homography which transforms the first image of the road to the second image of the road;

by matching image points of the road in the second image and corresponding image points of the road in the third image, computing a second homography which transforms the second image of the road to the third image of the road;

chaining the first and second homography to compute a chained homography; and

by using the chained homography as an initial guess, computing a third homography which transforms the first image of the road to the third image of the road.

2. The method of claim 1, wherein the first and second homographies are represented by respective matrices and wherein said chaining to compute a chained homography is performed by multiplying the matrices.

3. The method of claim 1, wherein the third homography is computed from matching image points of the road in the first image and corresponding image points of the road in the third image.

4. The method of claim 1, further comprising:

estimating the vehicle path in the images from at least one of the homographies.

5. The method of claim 1, further comprising:

processing image motion between the first image, the second image and the third image of the road, wherein said processing said image motion between said images of the road includes computing the vertical deviation in the contour of the road using a road surface model of the road.

6. The method of claim 5, wherein said computing the vertical deviation in the contour from said road surface model includes estimating optical flow between the first, second and third images by comparing said optical flow with an optical flow as predicted by said road surface model and producing thereby a residual optical flow indicating the vertical deviation.

7. The method of claim 5, wherein said processing motion of said images of the road includes:

initial warping said second image toward said first image assuming a planar model for the contour of the road, thereby producing a warped second image frame.

8. The method of claim 7, wherein said initial warping includes:

aligning said second image with said first image by adjusting for at least one image shift due to at least one motion of the vehicle relative to the road, wherein said at least one motion is selected from the group consisting of;

yaw, pitch and roll; and

adjusting for relative scale change between said second image frame and said first image frame, wherein said relative scale change arises from different distances to the camera.

9. The method of claim 7, further comprising:

selecting a plurality of selected image points in said first image;

for said selected image points, locating a plurality of image patches disposed respectively about said selected image points; and

tracking the selected image points by correlating said image patches in said first image frame with corresponding image patches in said warped second image frame, thereby producing a plurality of tracked points.

10. A driver assistance system mountable in a host vehicle, wherein the driver assistance system includes a camera operatively connectable to a processor, wherein the driver assistance system is operable while the host vehicle is moving to detect a vertical deviation in contour of a road, wherein a first image is captured at a first time, a second image is captured at a second time later than the first time and a third image is captured at a third time later than the second time,

wherein image points of the road in the first image are matched to corresponding image points of the road in the second image to compute a first homography which transforms the first image of the road to the second image of the road,

wherein image points of the road in the second image are matched to corresponding image points of the road in the third image to compute a second homography which transforms the second image of the road to the third image of the road, and

wherein a chained homography is computed by chaining the first homography with the second homography and by using by using the chained homography as an initial guess, a third homography is computed which transforms the first image of the road to the third image of the road.

11. The driver assistance system of claim 10, wherein the first and second homographies are represented by respective matrices and wherein the chained homography is computed by multiplying the matrices.

12. The driver assistance system of claim 10, wherein the second homography is computed from matching image points of the road in the second image and corresponding image points of the road in the third image.

13. The driver assistance system of claim 10, further operable to:

estimating the vehicle path in the images from at least one of the homographies.

14. The driver assistance system of claim 10, wherein image motion is processed between respective the first, second and third images of the road, wherein the vertical contour of the road is estimated using a road surface model of the road and the vertical deviation in the contour is computed from said road surface model.

15. The driver assistance system of claim 10, wherein optical flow is estimated between a plurality of first image patches of the road derived from said first image and corresponding second image patches of the road derived from said second image.

16. The driver assistance system of claim 15, wherein the vertical deviation in the contour is determined by comparing said optical flow with an optical flow as predicted by said road surface model to produce thereby a residual optical flow indicating the vertical deviation.

17. A computerized method for detecting a vertical deviation in contour of a road, the method performable by a driver assistance system mountable in a host vehicle while the host vehicle is moving, wherein the driver assistance system includes a camera operatively connectable to a processor, the method comprising:

capturing from the camera a first image of the road at a first time and a second image of the road at a second time;

by matching image points of the road in the first image and corresponding image points of the road in the second image, computing a homography which transforms the first image of the road to the second image of the road;

use the homography to align and warp the first image toward the second image to produce thereby a warped first image;

compute the optical flow between the second image to the warped first image;

comparing the optical flow with a road surface optical flow based on a road surface model to produce thereby a residual optical flow different from the road surface optical flow.

18. The computerized method of claim 17, further comprising:

using the residual optical flow to generate a vertical deviation in contour of the road; and

using the vertical deviation as an input to a suspension control system of the vehicle.

19. The computerized method of claim 17, further comprising:

using the residual optical flow to generate a vertical deviation in contour of the road to detect a hazard of height above the road of ten centimeters at a distance of more than thirty meters.