Hazard detection from a camera in a scene with moving shadows

• US 9,892,328 B2
• Filed: 06/12/2015
• Issued: 02/13/2018
• Est. Priority Date: 06/18/2014
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon, Dagan, Erez
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): SHIN, BRIAN D
• Application Number:
  US14/737,522
• Publication Number:
  US 20150371096A1
• Time in Patent Office: 977 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Computerized methods are performable by a driver assistance system while the host vehicle is moving. The driver assistance system includes a camera connectible to a processor. First and second image frames are captured from the field of view of the camera. Corresponding image points of the road are tracked from the first image frame to the second image frame. Image motion between the corresponding image points of the road is processed to detect a hazard in the road. The corresponding image points are determined to be of a moving shadow cast on the road to avoid a false positive detection of a hazard in the road or the corresponding image points are determined not to be of a moving shadow cast on the road to verify detection of a hazard in the road.

Claims

1. A method of processing images captured by a camera mountable in a host vehicle while the vehicle is moving, the method comprising:

capturing a first image frame and a second image frame from a field of view of the camera;

tracking corresponding image points from the first image frame and the second image frame;

detecting a plurality of corresponding suspect hazard image points, from among the tracked image points, based on a comparison of the first image frame to the second image frame;

provisionally classifying the suspect hazard image points as part of a hypothetical static object on a road;

comparing a change in a position of a bottom edge of the hypothetical static object in the first image frame and the second image frame relative to a change in a position of an image point of a road plane in the first image frame and the second image frame; and

rejecting the provisional static object classification if the change in the position of the bottom edge of the hypothetical static object is not attributable to the change in the position of the image point of the road plane or validating the provisional static object classification if the change in the position of the bottom edge of the hypothetical static object is attributable to the change in the position of the image point of the road plane.

2. The method of claim 1, further comprising:

recognizing in the image frames, images of a moving vehicle; and

associating the corresponding suspect hazard image points with the images of a moving vehicle, thereby determining the corresponding suspect hazard image points is due to a moving shadow cast from the moving vehicle.

3. The method of claim 2, wherein the associating further comprises:

hypothesizing that an image motion between the corresponding suspect hazard image points is due to a hypothetical moving shadow cast from the moving vehicle;

computing respective distances from the host vehicle to the hypothetical moving shadow and from the host vehicle to the moving vehicle; and

matching the computed distances to the hypothetical moving shadow with the computed distances to the moving vehicle, thereby confirming the hypothesis that the image motion is due to the moving shadow cast from the moving vehicle.

4. The method of claim 2, wherein the associating further comprises:

hypothesizing that an image motion between the corresponding suspect hazard image points is due to a hypothetical moving shadow cast from the moving vehicle;

determining a relative speed between the host vehicle and the moving vehicle;

calculating image motion of points on the road for the relative speed between the host vehicle and the moving vehicle;

measuring image motion of the hypothetical moving shadow;

comparing the measured image motion of the hypothetical moving shadow to the calculated image motion of points on the road for the relative speed between the host vehicle and the moving vehicle; and

confirming the hypothesis that the suspect hazard image points are of a moving shadow cast from the moving vehicle if the measured image motion matches the calculated image motion.

5. The method of claim 1, further comprising:

associating an image motion between the corresponding suspect hazard image points with a moving shadow when the image motion of the corresponding suspect hazard image points is determined to be consistent with a static object in the road of height greater than a threshold height.

6. The method of claim 1, further comprising:

recognizing images of a lane mark in the road; and

associating an image motion between the corresponding suspect hazard image points with a moving shadow when a patch associated with the suspect hazard image points is determined to be superimposed over the images of the lane mark.

7. The method of claim 6, further comprising:

associating the image motion between the corresponding suspect hazard image points with a moving shadow when the image motion is similar on both sides of the images of the lane mark.

8. The method of claim 1, further comprising:

recognizing in the image frames an image of an object suspected to cast a hypothetical moving shadow;

drawing a plurality of hypothetical lines in the image frames between image points associated with the hypothetical moving shadow and the image of the object; and

associating the hypothetical moving shadow with the object when a plurality of the hypothetical lines originate from the image of the object.

9. The method of claim 1, further comprising:

verifying that an image motion between the corresponding suspect hazard image points is due to a moving shadow when the image motion is not aligned with a focus of expansion or when a direction of the image motion changes sign.

10. The method of claim 1, further comprising:

recognizing in the image frames an object suspected to cast a hypothetical moving shadow; and

performing texture analysis on an image of the object to determine if the image of the object has at least one texture selected from a group consisting of;

branch texture and leaf texture.

11. The method of claim 1, further comprising:

recognizing in the image frames an image of a street lamp and an image of a tree suspected of casting a moving shadow; and

verifying that an image motion between the corresponding suspect hazard image points is due to a moving shadow from the light of the street lamp by constructing hypothetical lines between the image of the street lamp, the image of the tree and the corresponding suspect hazard image points.

12. The method according to claim 1, further comprising:

determining whether a bottom edge of the hypothetical static object is detected in the first and second image frames.

13. The method according to claim 12, further comprising:

rejecting the provisional static object classification if no bottom edge of the hypothetical static object is detected.

14. The method according to claim 1, wherein the bottom edge of the hypothetical static object is a hypothesized bottom edge.

15. The method according to claim 1, wherein provisionally classifying the suspect hazard image points as part of a hypothetical static object includes:

computing an estimated distance to the hypothetical static object; and

determining an image location of the bottom edge of the hypothetical static object based at least on the estimated distance to the hypothetical static object.

16. The method according to claim 15, wherein determining the bottom edge of the hypothetical static object includes:

determining an image location of the road plane at the estimated distance.

17. The method according to claim 1, wherein provisionally classifying the suspect hazard image points of a hypothetical static object includes:

computing a distance and a height of at least one of the corresponding suspect hazard image points assuming that the at least one of the corresponding suspect hazard image points corresponds to a static object on the road; and

determining an image location of the bottom edge of the hypothetical static object based on the computed image height and the computed distance of the least one of the corresponding suspect hazard image points.

18. A driver assistance system mountable in a host vehicle, the driver assistance system including a camera operatively connectible to a processor, the system configured, while the host vehicle is moving, to:

capture a first image frame and a second image frame from a field of view of the camera;

track corresponding image points from the first image frame and the second image frame;

detect a plurality of corresponding suspect hazard image points, from among the tracked image points, based on a comparison of the first image frame to the second image frame;

provisionally classify the suspect hazard image points as part of a hypothetical static object on a road;

compare a change in a position of a bottom edge of the hypothetical static object in the first image frame and the second image frame relative to a change in a position of an image point of a road plane in the first image frame and the second image frame; and

reject the provisional static object classification if the change in the position of the bottom edge of the hypothetical static object is not attributable to the change in the position of the image point of the road plane or validate the provisional static object classification if the change in the position of the bottom edge of the hypothetical static object is attributable to the change in the position of the image point of the road plane.

19. The driver assistance system of claim 18, further configured to:

recognize in the image frames, images of a moving vehicle; and

if an image motion between the corresponding suspect hazard image points of the road is associated with an image motion of the moving vehicle, determine that the image motion between the corresponding suspect hazard image points of the road is due to a moving shadow cast from the moving vehicle.

20. The driver assistance system of claim 18, further configured to:

recognize images of a lane mark in the road; and

associate an image motion between the corresponding suspect hazard image points with a moving shadow when a patch associated with the suspect hazard image points is determined to be superimposed over the images of the lane mark.

21. The driver assistance system of claim 18, further configured to:

recognize in the image frames an image of an object suspected to cast a hypothetical moving shadow;

draw a plurality of hypothetical lines in the image frames between image points associated with the hypothetical moving shadow and the image of the object; and

associate the hypothetical moving shadow with the object when a plurality of the hypothetical lines originate from the image of the object.

22. The driver assistance system of claim 18, further configured to:

verify that an image motion between the corresponding suspect hazard image points is due to a moving shadow when the image motion is not aligned with a focus of expansion or when a direction of the image motion changes sign.

23. The driver assistance system of claim 18, further configured to:

recognize in the image frames an object suspected to cast a hypothetical moving shadow;

perform texture analysis on an image of the object to determine if the image of the object has at least one texture selected from a group consisting of;

branch texture and leaf texture.

24. The driver assistance system of claim 18, further configured to:

recognize in the image frames an image of a street lamp and an image of a tree suspected of casting a moving shadow; and

verify that an image motion between the corresponding suspect hazard image points is due to a moving shadow from the light of the street lamp by constructing hypothetical lines between the image of the street lamp, the image of the tree and the corresponding suspect hazard image points.

25. The driver assistance system of claim 18, further configured to:

determine whether a bottom edge of the hypothetical static object is detected in the first and second image frames.

26. The driver assistance system of claim 25, further configured to:

reject the provisional static object classification if no bottom edge of the hypothetical static object is detected.

27. The driver assistance system of claim 18, wherein the bottom edge of the hypothetical static object is a hypothesized bottom edge.

28. The driver assistance system of claim 18, wherein the system is further configured to provisionally classify the suspect hazard image points as part of the hypothetical static object by:

computing an estimated distance to the hypothetical static object; and

determining an image location of the bottom edge of the hypothetical static object based at least on the estimated distance to the hypothetical static object.

29. The driver assistance system of claim 28, wherein the bottom edge of the hypothetical static object is determined by the processor by:

determining an image location of the road plane at the estimated distance.

30. The driver assistance system of claim 18, wherein the system is further configured to provisionally classify the suspect hazard image points as part of the hypothetical static object by:

computing a distance and a height of at least one of the corresponding suspect hazard image points assuming that the at least one of the corresponding suspect hazard image points corresponds to a static object on the toed; and

determining an image location of the bottom edge of the hypothetical static object based on the computed image height and the computed distance of the least one of the corresponding suspect hazard image points.