System and method for generating a model of the path of a roadway from an image recorded by a camera

• US 7,151,996 B2
• Filed: 04/14/2001
• Issued: 12/19/2006
• Est. Priority Date: 04/14/2000
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: Mobileye Technologies Limited (Intel Corporation)
• Inventors: Stein, Gideon P.
• Primary Examiner(s): Nguyen, Cuong H.
• APPLICATION: US 20030040864A1
• Time in patent office: 2075 Days
• Field of Search: 701/1, 701/200, 701/201, 701/205, 701223-225, 701/300, 701/301, 701/36, 701/23, 701/25, 701/26, 701/28, 701/209, 701/211, 731/78.R, 340/901, 348/207.99, 348/844, 382/168, 382/170, 382/171, 382/173, 382/181, 382/312, 382/316
• US Class Current: 701/200
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 2201/0213 Road vehicle, e.g. car or truck
• CPC Class Codes: G06K 9/00798 Recognition of lanes or road borders, e.g. of lane markings, or recognition of driver's driving pattern in relation to lanes perceived from the vehicle; Analysis of car trajectory relative to detected road determination of trajectories G06T7/20
• CPC Class Codes: G06T 3/0081 by elastic snapping
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/55 from multiple images

Abstract

A road skeleton estimation system generates an estimate as to a skeleton of at least a portion of a roadway ahead of a vehicle. The road skeleton estimation system includes an image receiver and a processor. The image receiver is configured to receive image information relating to at least one image recorded ahead of the vehicle. The processor is configured to process the image information received by the image receiver to generate an estimate of the skeleton of at least a portion of the roadway ahead of the vehicle.

Claims

1. Apparatus for monitoring motion of a vehicle relative to a road on which the vehicle is traveling, comprising:

a camera that acquires an image of a scene in which the road is present; and

a controller that

a) defines a plurality of real-space curves having different shapes and/or orientations that are candidates for substantially lying along a contour of the road;

b) for each candidate curve, for each of a plurality of lines of pixels in the image that image regions of the scene lying along the candidate curve or that image regions of the scene lying along a same curve homologous with and having a same orientation as the candidate curve, determines a value that is a function of gradients of pixel intensities at locations along the line;

c) determines a candidate curve that most closely lies along the road contour responsive to the values determined for the lines; and

d) uses the determined candidate curve to monitor motion of the vehicle relative to the road.

2. A method for monitoring motion of a vehicle relative to a road on which the vehicle is traveling, the method comprising:

acquiring a camera image of a scene in which the road is present;

defining a plurality of real-space curves having different shapes and/or orientations that are candidates for substantially lying along a contour of the road;

for each candidate curve, for each of a plurality of lines of pixels in the image that image regions of the scene lying along the candidate curve or that image regions of the scene lying along a same curve homologous with and having a same orientation as the candidate curve, determining a value that is a function of gradients of pixel intensities at locations along the line; and

determining a candidate curve that most closely lies along the road contour responsive to the values determined for the lines; and

using the determined candidate curve to monitor motion of the vehicle relative to the road.

3. A method according to claim 2 wherein determining a candidate curve comprises determining a value for a cost function for a candidate curve responsive to the values determined for the lines of pixels in the image associated with the candidate curve and determining the candidate curve responsive to the value of the cost function.

4. A method according to claim 3 wherein determining the value for a line comprises determining for each of a plurality of locations along the line a value for a derivative of pixel intensity at the location of the pixel as a function of displacement in the image in a direction perpendicular to the line.

5. A method according to claim 4 wherein determining a value for a line comprises determining a sum of the derivatives determined for locations lying along the line.

6. A method according to claim 5 wherein the cost function is a function responsive to the absolute value of the value determined for the line.

7. A method according to claim 6 wherein the cost function is a function of a sum of the absolute values of the values determined for the lines.

8. A method according to claim 6 wherein the cost function is a function of a sum of the squares of the values determined for the lines.

9. A method according to claim 3 wherein determining the value for a line comprises determining a sum of intensities of pixels lying along the line and determining a derivative of the sum as a function of line position.

10. A method according to claim 9 wherein the cost function is function responsive to the absolute value of the derivative determined for the line.

11. A method according to claim 10 wherein the cost function is a function of a sum of the absolute values of the derivatives determined for the lines.

12. A method according to claim 3 wherein determining a cost function comprises;

partitioning the camera image into near and far portions that respectively image regions of the scene that arc closer and farther from the camera;

determining the value for each line responsive to pixel intensities lying along the line in the near portion of the camera image;

for those lines for which the value exceeds a given threshold, determining an adjusted value responsive to pixel intensities lying along the line in the far portion of the image; and

using the adjusted values to determine the cost function.

13. A method according to claim 2 wherein determining a value for a line in the camera image comprises:

for each candidate curve, transforming the camera image to a test image having features defined relative to a rectilinear coordinate system having first and second orthogonal axes so that the lines in the camera image are transformed into straight lines parallel to the first axis in the test image; and

determining the value responsive to gradients of pixel intensities for locations along the straight line in the test image that corresponds to the line in the camera image.

14. A method according to claim 13 wherein determining a value for a line in the camera image comprises:

determining a sum of intensities of pixels that lie along the straight line in the test image that corresponds to the line in the camera image; and

determining a derivative of the sum as a function of position of the straight line along the second axis.

15. A method according to claim 2 wherein each candidate curve is an arc of a circle and wherein defining the plurality of candidate arcs comprises defining a plurality of circles, each of which defines a different one of the candidate curves.

16. A method according to claim 15 wherein defining the plurality of circles comprises determining each circle to pass through a different set of three points.

17. A method according to claim 16 wherein the first point in each set is a point in real space at which the camera is located.

18. A method according to claim 17 wherein the second and third points are points determined to lie along the contour of the road at different distances from the camera.

19. A method according to claim 3 wherein each candidate curve is a parabola.

20. A method according to claim 19 wherein each candidate parabola has its major axis perpendicular to the optic axis of the camera.

21. A method according to claim 20 wherein each candidate parabola passes through the location of the camera.

22. A method according to claim 20 wherein a coefficient of a linear term of the parabola is determined responsive to gradients at pixel locations in a first region of the image corresponding to a region of the scene close to the camera.

23. A method according to claim 21 wherein a coefficient for the quadratic term of the parabola is determined responsive to the coefficient of the linear term and to gradients at pixel locations in a second region of the image corresponding to a region of the scene relatively.

24. A method according to claim 2 wherein each candidate curve is a straight line.

25. A method according to claim 12 wherein transforming the camera image comprises determining a pitch angle for the optic axis of the camera relative to the road and using the pitch angle to transform the camera image.

26. A method according to claim 3 wherein determining a cost function comprises weighting the gradients to moderate contributions to the cost function from a bright object in the camera image.