Barrier and guardrail detection using a single camera

• US 9,280,711 B2
• Filed: 09/20/2011
• Issued: 03/08/2016
• Est. Priority Date: 09/21/2010
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Rahman, Mohammad J
• Application Number:
  US13/237,163
• Publication Number:
  US 20120069185A1
• Time in Patent Office: 1,631 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Driver assistance systems for detecting a structural barrier extending along a road. The driver assistance system may be mountable in a host vehicle. The camera may capture multiple image frames in the forward field of view of the camera. A processor may process motion of images of the barrier in the image frames. The camera may be a single camera. The motion of the images may be responsive to forward motion of the host vehicle and/or the motion of the images may be responsive to lateral motion of the host vehicle.

Claims

1. A method of detection of a structural barrier extending along a road, the method performed by a driver assistance system mountable in a host vehicle, wherein the driver assistance system includes a camera operatively connectible to a processor, the method comprising:

capturing a plurality of image frames in the forward field of view of the camera;

in the image frames, locating an image patch suspected to include an image of part of a structural barrier; and

processing image motion of said image patch of the structural barrier to compute a lateral distance between the host vehicle and said image patch using the image motion of said image patch and motion of the host vehicle, the image motion of said image patch comprising mapping coordinates of an image from image frame to image frame or a function of the mapping, thereby detecting the structural barrier while avoiding use of a second sensor other than the camera to sense the structural barrier.

2. The method of claim 1, wherein the camera is a single camera.

3. The method of claim 1, wherein the camera is configured to view in the direction of forward motion of the host vehicle.

4. The method of claim 1, wherein said image motion of said image patch is responsive to forward motion of the host vehicle.

5. The method of claim 1, wherein said image motion of said image patch is responsive to lateral motion of the host vehicle.

6. The method of claim 1, wherein the structural barrier includes a plurality of posts, the method further comprising:

hypothesizing a plurality of linear image structures in an image frame as projections of the barrier onto the road surface and thereby obtaining a plurality of vertical image coordinates respectively from said linear image structures;

computing multiple forward distances and corresponding lateral distances to the posts based on the vertical image coordinates;

based on the known forward motion of the host vehicle and horizontal image coordinates of the linear image structures, computing new horizontal image coordinates of the linear image structures; and

selecting the horizontal image coordinate in a second image frame of one of the linear image structures to align an image of one of the posts.

7. The method of claim 1, wherein the structural barrier includes a plurality of posts, the method comprising, for each of the posts:

determining forward distances from the host vehicle to the posts based on the motion of the images and the forward motion of the host vehicle;

computing lateral distances to the posts from the host vehicle from the forward distances and horizontal image coordinates of the posts;

hypothesizing road plane lines at said lateral distances, to form a plurality of hypothesized road plane lines as projections of the structural barrier onto the road surface;

projecting said hypothesized road plane lines at said lateral distances onto an image of the structural barrier in an image frame; and

selecting the correct road plane line from the hypothesized road plane lines by aligning the correct road plane line with the image of the structural barrier in the image frame.

8. The method of claim 1, further comprising:

warping said image patch based on a vertical surface model; and

detecting the structural barrier by ascertaining that the patch is an image of the structural barrier when points in columns of the patch scale vertically with host vehicle motion.

9. The method of claim 8, further comprising:

warping said image patch based on a road surface model; and

ascertaining that the patch is an image of the road surface if points in rows of the patch scale horizontally with host vehicle motion.

10. A driver assistance system for detecting a structural barrier extending along a road, wherein the driver assistance system is mountable in a host vehicle, the driver assistance system comprising:

a camera operable to capture a plurality of image frames in the forward field of view; and

a processor operatively connected to the camera, wherein the processor is operable to process in the image frames image motion of an image patch suspected to be an image of part of a barrier to compute a lateral distance between the host vehicle and said image patch using the image motion of said image patch and motion of the host vehicle, the image motion of said image patch comprising mapping coordinates of an image from image frame to image frame or a function of the mapping, to detect the barrier while avoiding use of a second sensor other than the camera to sense the structure barrier.

11. The system of claim 10, wherein the camera is a single camera.

12. The system of claim 10, wherein said image motion of the image patch is responsive to forward motion of the host vehicle.

13. The system of claim 10, wherein said image motion of the image patch is responsive to lateral motion of the host vehicle.

14. The system of claim 10, wherein said image motion of the image patch of the part of the structural barrier correlates with an image line in the direction of the vanishing point of the road, wherein said image line corresponds to a vertical projection of the structural barrier onto the road plane.

15. The system of claim 10, wherein the processor is operable to hypothesize linear image structures as projections of the structural barrier onto the road plane to produce a plurality of hypotheses;

wherein each said hypothesis gives a lateral position of the barrier relative to the host vehicle;

wherein for each said hypothesis, the lateral positions and host vehicle motion are used to predict image motion and wherein the predicted image motion is compared to the actual image motion to verify the hypothesis and to derive the actual lateral position of the structural barrier relative to the host vehicle.

16. The system of claim 15, wherein motion of the host vehicle has a lateral component relative to the road direction and the image motion is of an image line in the image that is above the linear image structure, wherein vertical motion of the image line is used to determine lateral distance between the host vehicle and the structural barrier.

17. The system of claim 10, wherein the structural barrier includes a plurality of posts, wherein the processor is operable:

to hypothesize a plurality of linear image structures in an image frame as projections of the barrier onto the road surface and thereby obtaining a plurality of vertical image coordinates respectively from said linear image structures;

to compute multiple forward distances and corresponding lateral distances to the posts based on the vertical image coordinates;

based on the known forward motion of the host vehicle and horizontal image coordinates of the linear image structures, to compute new horizontal image coordinates of the linear image structures; and

to select the horizontal image coordinate in a second image frame of one of the linear image structures to align an image of one of the posts.

18. The system of claim 10, wherein the structural barrier includes a plurality of posts, wherein for each of the posts, the processor is operable to:

determine forward distances from the host vehicle to the posts based on the motion of the images and the forward motion of the host vehicle;

compute lateral distances to the posts from the host vehicle from the forward distance and horizontal image coordinates of the posts;

hypothesize road plane lines at said lateral distances, to form a plurality of hypothesized road plane lines as projections of the structural barrier onto the road surface;

project said hypothesized road plane lines at said lateral distances onto an image of the structural barrier in an image frame; and

select the correct road plane line from the hypothesized road plane lines by aligning the correct road plane line with the image of the structural barrier in the image frame.

19. The system of claim 10, wherein the processor is operable to:

locate in an image frame said image patch above an image line intersecting the vanishing point in the image frame;

warp said image based on a vertical surface model; and

detect the structural barrier by ascertaining that the patch is an image of the structural barrier when points in columns of the patch scale vertically with the host vehicle motion.

20. The system of claim 19, wherein the processor is operable to:

ascertain that the image patch is an image of the road surface if points in rows of the patch scale horizontally with host vehicle motion.

21. The system of claim 10, wherein the camera and processor are used to serve another driver assistance system.