Pedestrian collision warning system

• US 9,233,659 B2
• Filed: 04/25/2012
• Issued: 01/12/2016
• Est. Priority Date: 04/27/2011
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Rosenbaum, Dan, Gurman, Amiad, Samet, Yonatan, Stein, Gideon P., Aloni, David
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Duley, Janese
• Application Number:
  US13/455,719
• Publication Number:
  US 20120314071A1
• Time in Patent Office: 1,357 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B60R 21/013 including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A method is provided for preventing a collision between a motor vehicle and a pedestrian. The method uses a camera and a processor mountable in the motor vehicle. A candidate image is detected. Based on a change of scale of the candidate image, it may be determined that the motor vehicle and the pedestrian are expected to collide, thereby producing a potential collision warning. Further information from the image frames may be used to validate the potential collision warning. The validation may include an analysis of the optical flow of the candidate image, that lane markings prediction of a straight road, a calculation of the lateral motion of the pedestrian, if the pedestrian is crossing a lane mark or curb and/or if the vehicle is changing lanes.

Claims

1. A method for determining a risk of collision between a host motor vehicle and a pedestrian using a camera and a processor mountable in the host motor vehicle, wherein the processor is operably connectible to the camera and the processor is configured to capture a plurality of image frames from the camera of the environment of the motor vehicle in the field of view of the camera, the method comprising:

detecting an image patch in at least one of the image frames, wherein the image patch includes a candidate image of a pedestrian in the field of view of the camera;

providing a collision warning or a collision prevention control signal if the pedestrian is on the far side of the curb or lane mark and is crossing the curb or the lane mark;

or

inhibiting a collision warning or a collision prevention control signal if the pedestrian is on the far side of the curb or lane mark and the pedestrian is not crossing the curb or the lane mark.

2. The method of claim 1, further comprising:

tracking optical flow between the image frames of a plurality of image points of said image patch to produce a tracked optical flow;

for a plurality of models selected from the group consisting of;

(i) a road surface model wherein a portion of the image points is modeled to be imaged from a road surface (ii) vertical surface model wherein a portion of the image points is modeled to be imaged from a substantially vertical object and (iii) a mixed model wherein a first portion of the image points is modeled to be imaged from a road surface and a second portion of the image points is modeled to be imaged from a substantially vertical object, fitting the tracked optical flow of at least a portion of the image points to produce a plurality of fits to the models;

using at least a portion of the image points, scoring the fits of the tracked optical flow to the respective models to produce respective scores; and

determining if a collision is expected and a time-to-collision by selecting the model with a score which corresponds to a best fit of the image points to the tracked optical flow.

3. The method of claim 2, further comprising:

verifying said candidate image as an image of said upright pedestrian and not an object in the road surface when the best fit model is the vertical surface model.

18. The method of claim 1, further comprising:

subsequent to said detecting an image patch, based on a change of scale of the candidate image, determining that the motor vehicle and the pedestrian are expected to collide, thereby producing a potential collision warning.

20. The system of claim 1 wherein the processor is configured to, based on a change of scale of the candidate image, determine that the host motor vehicle and the pedestrian are expected to collide to produce a potential collision warning and the processor is configured to analyze optical flow of the candidate image to validate the potential collision warning.

4. A method for determining a risk of collision between a host motor vehicle and a pedestrian, the method using a camera and a processor mountable in the host motor vehicle, wherein the processor is operably connectible to the camera and the processor is configured to capture image frames from the camera of the environment of the host motor vehicle in the field of view of the camera, the method comprising:

detecting an image patch in at least one of the image frames, wherein the image patch includes a candidate image of a pedestrian in the field of view of the camera;

predicting at least two most likely paths of the host vehicle;

computing a region of intersection of said at least two paths;

determining whether the pedestrian is inside or moving towards said region of intersection; and

providing a collision warning signal or a collision control signal when the pedestrian is inside or moving towards said region of intersection;

or

inhibiting a collision warning signal or a collision control signal when the pedestrian is not inside and not moving towards said region of intersection.

5. The method of claim 4, wherein said at least two most likely paths are selected from the group consisting of:

a path of the present lane of travel of the vehicle as indicated by lane markings or curb, a path that follows current steering angle or yaw rate, a path of travel to follow a preceding vehicle, a path which is straight ahead and a path which avoids the vehicle going up on a curb.

6. The method of claim 4, further comprising:

determining that lane markings predict a straight road.

7. The method of claim 4, further comprising:

calculating lateral motion of the pedestrian.

8. The method of claim 4, further comprising:

determining if the pedestrian is crossing a lane mark or curb.

9. The method of claim 4, further comprising:

determining if the vehicle is changing lanes.

10. The method of claim 4, further comprising:

selecting an image patch in an image frame corresponding to where the motor vehicle will be in a predetermined time interval;

monitoring said image patch; and

if the pedestrian is imaged in said image patch, then providing said collision warning signal or said collision prevention control signal.

11. The method of claim 4 wherein said determining that the motor vehicle and the pedestrian are expected to collide includes calculating a first time-to-collision based on a change of scale of the candidate image.

12. The method of claim 11, further comprising:

calculating a second time-to-collision based on further information from the image frames; and

comparing the first and second times to collision.

13. The method of claim 12, wherein said providing the collision warning signal or the collision control signal is performed when an absolute value difference between the first and second times-to-collision is less than a threshold.

19. The method of claim 4, further comprising:

based on a change of scale of the candidate image, determining that the motor vehicle and the pedestrian are expected to collide, thereby producing a potential collision warning;

analyzing optical flow of the candidate image thereby validating the potential collision warning.

14. A system operable to determine a risk of collision between a motor vehicle and a pedestrian, the system comprising:

a camera mountable in the motor vehicle; and

a processor mountable in the motor vehicle, wherein the processor is operably connectible to the camera and the processor is configured to capture image frames from the camera of the environment of the motor vehicle in the field of view of the camera,

wherein the processor is configured to detect an image patch, in at least one of the image frames, wherein the image patch includes a candidate image of the pedestrian in the field of view of the camera;

wherein the processor is configured to provide a collision warning signal or a collision control signal if the pedestrian is on the far side of the curb or lane mark and is crossing the curb or the lane mark;

or

wherein the processor is configured to inhibit a collision warning or a collision prevention control signal if the pedestrian is on the far side of the curb or lane mark and the pedestrian is not crossing the curb or the lane mark.

15. The system of claim 14, wherein the processor is configured to determine that lane markings predict a straight road.

16. The system of claim 14, wherein the processor is configured to calculate lateral motion of the pedestrian.

17. The system of claim 14, wherein the processor is configured to determine if the pedestrian is crossing a lane mark or curb.