Pedestrian collision warning system

• US 9,925,939 B2
• Filed: 12/29/2015
• Issued: 03/27/2018
• Est. Priority Date: 04/27/2011
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Rosenbaum, Dan, Gurman, Amiad, Samet, Yonatan, Stein, Gideon P., Aloni, David
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Duley, Janese
• Application Number:
  US14/982,198
• Publication Number:
  US 20160107595A1
• Time in Patent Office: 819 Days
• CPC Class Codes: B60R 21/013 including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A method is provided for preventing a collision between a motor vehicle and a pedestrian. The method uses a camera and a processor mountable in the motor vehicle. A candidate image is detected. Based on a change of scale of the candidate image, it may be determined that the motor vehicle and the pedestrian are expected to collide, thereby producing a potential collision warning. Further information from the image frames may be used to validate the potential collision warning. The validation may include an analysis of the optical flow of the candidate image, that lane markings prediction of a straight road, a calculation of the lateral motion of the pedestrian, if the pedestrian is crossing a lane mark or curb and/or if the vehicle is changing lanes.

Claims

1. A computerized method using a camera mountable in a motor vehicle, the method comprising:

acquiring a plurality of image frames from the camera with a known time interval;

detecting an image patch in at least one of the image frames, wherein the image patch includes a candidate image of a pedestrian in the field of view of the camera;

responsive to the motor vehicle moving forward, tracking optical flow between the image frames of an image point of the pedestrian;

from horizontal image motion of the tracked point of the pedestrian relative to the horizontal image coordinate of the focus of expansion in an image space, computing physical lateral motion of the pedestrian responsive to longitudinal distance to the pedestrian in a physical space; and

estimating the lateral position of the pedestrian at a time-to-collision from the physical lateral motion of the pedestrian.

2. The method of claim 1, further comprising

determining a first time-to-collision based on the longitudinal distance to the pedestrian and a speed of the motor vehicle and comparing the first time-to-collision to a threshold;

and

determining a second time-to-collision based on optical flow of the image patch and comparing the second time-to-collision to a threshold.

3. The method of claim 2, further comprising

when the first and second times to collision are of similar value and both less than the respective thresholds, providing a warning or issuing a control signal;

or inhibiting a warning or inhibiting a control signal if the motor vehicle when the first and second times to collision are not of similar value and not both less than the respective thresholds.

4. The method of claim 2, further comprising:

for a plurality of models selected from the group consisting of;

(i) a road surface model wherein a portion of the image points is modeled to be imaged from a road surface (ii) vertical surface model wherein a portion of the image points is modeled to be imaged from a substantially vertical object and (iii) a mixed model wherein a first portion of the image points is modeled to be imaged from a road surface and a second portion of the image points is modeled to be imaged from a substantially vertical object, fitting the tracked optical flow of at least a portion of the image points to produce a plurality of fits to the models;

using at least a portion of the image points, scoring the fits of the tracked optical flow to the respective models to produce respective scores; and

determining if a collision is expected and the second time-to-collision by selecting the model with a score which corresponds to a best fit of the image points to the tracked optical flow.

5. The method of claim 4, further comprising:

determining that a collision is expected based on the best fit of the image points to the tracked optical flow corresponding to the vertical surface model or the mixed model.

6. The method of claim 1, further comprising

providing a collision warning or a collision prevention control signal if the pedestrian is on the far side of a curb or a lane mark and is crossing the curb or the lane mark;

or

inhibiting a collision warning or a collision prevention control signal if the pedestrian is on the far side of a curb or a lane mark and the pedestrian is not crossing the curb or the lane mark.

7. The method of claim 1, further comprising:

predicting at least two most likely paths of the host vehicle;

computing a region of intersection of said at least two paths;

determining whether the pedestrian is inside or moving towards said region of intersection; and

providing a collision warning signal or a collision control signal when the pedestrian is inside or moving towards said region of intersection;

or inhibiting a collision warning signal or a collision control signal when the pedestrian is not inside and not moving towards said region of intersection.

8. The method of claim 1, further comprising:

inputting or computing from the image frames a yaw rate of the motor vehicle; and

estimating from the yaw rate the horizontal image coordinate of the focus of expansion.

9. A system comprising:

a camera mountable in a motor vehicle; and

a processor mountable in the motor vehicle, wherein the processor is operably connectable to the camera;

wherein the processor is configured to;

acquire a plurality of image frames from the camera with a known time interval;

detect an image patch in at least one of the image frames, wherein the image patch includes a candidate image of a pedestrian in the field of view of the camera;

responsive to the forward motion of the motor vehicle, track optical flow between the image frames of an image point of the pedestrian;

from horizontal image motion of the tracked point of the pedestrian relative to the horizontal image coordinate of the focus of expansion in an image space, compute physical lateral motion of the pedestrian responsive to longitudinal distance to the pedestrian in a physical space; and

estimate the lateral position of the pedestrian at a time-to-collision from the physical lateral motion of the pedestrian.

10. The system according to claim 9, wherein the processor is further configured to:

determine a first time-to-collision based on the longitudinal distance to the pedestrian and a speed of the motor vehicle and compare the first time-to-collision to a threshold; and

determine a second time-to-collision based on optical flow of the image patch and comparing the second time-to-collision to a threshold.

11. The system of claim 10, wherein the processor is further configured to:

provide a warning or issue a control signal when the first and second times to collision are of similar value and both less than the respective thresholds;

or when the first and second times to collision are not of similar value and not both less than the respective thresholds inhibit a warning or inhibit a control signal.

12. The system of claim 10, wherein the processor is further configured to:

for a plurality of models selected from the group consisting of;

(i) a road surface model wherein a portion of the image points is modeled to be imaged from a road surface (ii) vertical surface model wherein a portion of the image points is modeled to be imaged from a substantially vertical object and (iii) a mixed model wherein a first portion of the image points is modeled to be imaged from a road surface and a second portion of the image points is modeled to be imaged from a substantially vertical object, fit the tracked optical flow of at least a portion of the image points to produce a plurality of fits to the models;

using at least a portion of the image points, score the fits of the tracked optical flow to the respective models to produce respective scores; and

determine if a collision is expected and the second time-to-collision by selecting the model with a score which corresponds to a best fit of the image points to the tracked optical flow.

13. The system of claim 12, wherein the processor is further configured to:

determine that a collision is expected based on the best fit of the image points to the tracked optical flow corresponding to the vertical surface model or the mixed model.

14. The system of claim 9, wherein the processor is further configured to:

provide a collision warning or a collision prevention control signal if the pedestrian is on the far side of a curb or a lane mark and is crossing the curb or the lane mark;

or

inhibit a collision warning or a collision prevention control signal if the pedestrian is on the far side of a curb or a lane mark and the pedestrian is not crossing the curb or the lane mark.

15. The system of claim 9, wherein the processor is further configured to:

predict at least two most likely paths of the host vehicle;

compute a region of intersection of said at least two paths;

determine whether the pedestrian is inside or moving towards said region of intersection; and

provide a collision warning signal or a collision control signal when the pedestrian is inside or moving towards said region of intersection;

or when the pedestrian is not inside and not moving towards said region of intersection, inhibit a collision warning signal or a collision control signal.

16. The system of claim 9, wherein the processor is further configured to:

input or compute from the image frames a yaw rate of the motor vehicle; and

estimate from the yaw rate the horizontal image coordinate of the focus of expansion.