Hazard detection from a camera in a scene with moving shadows

• US 10,262,216 B2
• Filed: 01/10/2018
• Issued: 04/16/2019
• Est. Priority Date: 06/18/2014
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon, Dagan, Erez
• Primary Examiner(s): Bella, Matthew C
• Assistant Examiner(s): Shin, Brian D
• Application Number:
  US15/867,550
• Publication Number:
  US 20180225529A1
• Time in Patent Office: 461 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Computerized methods are performable by a driver assistance system while the host vehicle is moving. The driver assistance system includes a camera connectible to a processor. First and second image frames are captured from the field of view of the camera. Corresponding image points of the road are tracked from the first image frame to the second image frame. Image motion between the corresponding image points of the road is processed to detect a hazard in the road. The corresponding image points are determined to be of a moving shadow cast on the road to avoid a false positive detection of a hazard in the road or the corresponding image points are determined not to be of a moving shadow cast on the road to verify detection of a hazard in the road.

Claims

1. A method of processing images captured by a camera mountable in a host vehicle while the vehicle is moving, the method comprising:

capturing a first image frame and a second image frame from a field of view of the camera;

tracking corresponding image points from the first image frame and the second image frame;

detecting a plurality of corresponding suspect hazard image points, from among the tracked image points, based on a comparison of the first image frame to the second image frame;

representing a hypothetical static object corresponding to the suspect hazard image points;

comparing a change in a position of a bottom edge of the hypothetical static object in the first image frame and the second image frame relative to a change in a position of an image point of a road plane in the first image frame and the second image frame; and

determining that the suspect hazard image points correspond to an actual static object corresponding to the hypothetical static object if the change in the position of the bottom edge of the hypothetical static object is attributable to the change in the position of the image point of the road plane.

2. The method of claim 1, further comprising determining that the suspect hazard image points do not correspond to the actual static object if the change in the position of the bottom edge of the hypothetical static object is not attributable to the change in the position of the image point of the road plane.

3. The method of claim 1, wherein the bottom edge of the hypothetical static object is a hypothesized bottom edge.

4. The method of claim 1, further comprising determining whether a bottom edge of the hypothetical static object is detected in the first and second image frames.

5. The method of claim 4, further comprising determining that the suspect hazard image points do not correspond to the actual static object if no bottom edge of the hypothetical static object is detected.

6. The method of claim 1, further comprising:

recognizing, in the first and second image frames, images of a moving vehicle; and

associating the corresponding suspect hazard image points with the images of a moving vehicle, thereby determining the corresponding suspect hazard image points are due to a moving shadow cast from the moving vehicle.

7. The method of claim 6, wherein the associating further comprises:

hypothesizing that an image motion between the corresponding suspect hazard image points is due to a hypothetical moving shadow cast from the moving vehicle; and

computing respective distances from the host vehicle to the hypothetical moving shadow and from the host vehicle to the moving vehicle.

8. The method of claim 7, wherein the associating further comprises:

matching the computed distances to the hypothetical moving shadow with the computed distances to the moving vehicle, thereby confirming the hypothesis that the image motion is due to the moving shadow cast from the moving vehicle.

9. The method of claim 1, further comprising associating an image motion between the corresponding suspect hazard image points with a moving shadow when the image motion of the corresponding suspect hazard image points is determined to be consistent with a static object in the road of height greater than a threshold height.

10. A system mountable in a host vehicle, the system including a camera operatively connectible to a processor, the processor of the system configured, while the host vehicle is moving, to:

capture a first image frame and a second image frame from a field of view of the camera;

track corresponding image points from the first image frame and the second image frame;

detect a plurality of corresponding suspect hazard image points, from among the tracked image points, based on a comparison of the first image frame to the second image frame;

represent a hypothetical static object corresponding to the suspect hazard image points;

compare a change in a position of a bottom edge of the hypothetical static object in the first image frame and the second image frame relative to a change in a position of an image point of a road plane in the first image frame and the second image frame; and

determine that the suspect hazard image points correspond to an actual static object corresponding to the hypothetical static object if the change in the position of the bottom edge of the hypothetical static object is attributable to the change in the position of the image point of the road plane.

11. The system of claim 10, wherein the processor is further configured to determine that the suspect hazard image points do not correspond to the actual static object if the change in the position of the bottom edge of the hypothetical static object is not attributable to the change in the position of the image point of the road plane.

12. The system of claim 10, wherein the processor is further configured to:

recognize, in the first and second image frames, images of a moving vehicle; and

associate the corresponding suspect hazard image points with the images of a moving vehicle, thereby determining the corresponding suspect hazard image points are due to a moving shadow cast from the moving vehicle.

13. The system of claim 10, wherein the processor is further configured to perform the associating by:

hypothesizing that an image motion between the corresponding suspect hazard image points is due to a hypothetical moving shadow cast from the moving vehicle; and

computing respective distances from the host vehicle to the hypothetical moving shadow and from the host vehicle to the moving vehicle.

14. The system of claim 13, wherein the processor is further configured to perform the associating by matching the computed distances to the hypothetical moving shadow with the computed distances to the moving vehicle, thereby confirming the hypothesis that the image motion is due to the moving shadow cast from the moving vehicle.

15. The system of claim 10, wherein the processor is further configured to associate an image motion between the corresponding suspect hazard image points with a moving shadow when the image motion of the corresponding suspect hazard image points is determined to be consistent with a static object in the road of height greater than a threshold height.

16. A non-transitory computer readable medium storing computer-executable instructions that, when executed by the processor, cause the processor to perform a method comprising:

capturing a first image frame and a second image frame from a field of view of the camera;

tracking corresponding image points from the first image frame and the second image frame;

detecting a plurality of corresponding suspect hazard image points, from among the tracked image points, based on a comparison of the first image frame to the second image frame;

representing a hypothetical static object corresponding to the suspect hazard image points;

comparing a change in a position of a bottom edge of the hypothetical static object in the first image frame and the second image frame relative to a change in a position of an image point of a road plane in the first image frame and the second image frame; and

determining that the suspect hazard image points correspond to an actual static object corresponding to the hypothetical static object if the change in the position of the bottom edge of the hypothetical static object is attributable to the change in the position of the image point of the road plane.

17. The non-transitory computer readable medium of claim 16, wherein the method further comprises determining that the suspect hazard image points do not correspond to the actual static object if the change in the position of the bottom edge of the hypothetical static object is not attributable to the change in the position of the image point of the road plane.

18. The non-transitory computer readable medium of claim 16, wherein the method further comprises:

recognizing, in the first and second image frames, images of a moving vehicle; and

associating the corresponding suspect hazard image points with the images of a moving vehicle, thereby determining the corresponding suspect hazard image points are due to a moving shadow cast from the moving vehicle.

19. The non-transitory computer readable medium of claim 18, wherein the associating further comprises:

hypothesizing that an image motion between the corresponding suspect hazard image points is due to a hypothetical moving shadow cast from the moving vehicle; and

computing respective distances from the host vehicle to the hypothetical moving shadow and from the host vehicle to the moving vehicle.

20. The non-transitory computer readable medium of claim 19, wherein the associating further comprises:

matching the computed distances to the hypothetical moving shadow with the computed distances to the moving vehicle, thereby confirming the hypothesis that the image motion is due to the moving shadow cast from the moving vehicle.