Systems and methods for curb detection and pedestrian hazard assessment

• US 10,303,958 B2
• Filed: 02/26/2018
• Issued: 05/28/2019
• Est. Priority Date: 05/14/2014
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Springer, Ofer
• Primary Examiner(s): Wong, Allen C
• Application Number:
  US15/905,777
• Publication Number:
  US 20180189579A1
• Time in Patent Office: 456 Days
• Field of Search: 348148
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A detection system for a vehicle is provided. The detection system may include at least one image capture device configured to acquire a plurality of images of an area forward of the vehicle, the area including a curb separating a road surface from an off-road surface and a data interface. The detection system may also include at least one processing device programmed to receive the plurality of images via the data interface, and determine a plurality of curb edge line candidates in the plurality of images. The at least one processing device may be further programmed to identify at least one edge line candidate as an edge line of the curb.

Claims

1. A detection system, the system comprising:

at least one processing device programmed to;

receive a plurality of images of an area output from at least one camera via a data interface;

determine a plurality of road edge line candidates from and within the plurality of images of the area; and

identify, based on the road edge line candidates, a region in the plurality of images that corresponds to a road surface and a region in the plurality of images that corresponds to an off-road surface.

2. The detection system of claim 1, wherein the at least one processing device is further programmed to:

identify a pedestrian in one or more of the plurality of images;

assess whether the pedestrian presents a hazard; and

determine an action to be taken to avoid a collision with the pedestrian, based on the assessment.

3. The detection system of claim 2, wherein assessing whether the pedestrian presents a hazard includes classifying the pedestrian based on the identified regions and at least one of a location or a trajectory of the pedestrian.

4. The detection system of claim 3, wherein classifying the pedestrian includes classifying the pedestrian as on the road surface or on the off-road surface.

5. The detection system of claim 2, wherein assessing whether the pedestrian presents a hazard includes comparing a trajectory of the pedestrian to a trajectory of a vehicle.

6. The detection system of claim 2, wherein the determined action to be taken to avoid the collision includes providing an alert to a driver.

7. The detection system of claim 2, wherein the determined action to be taken to avoid the collision includes modifying at least one of a speed, acceleration, or direction of a vehicle.

8. The detection system of claim 1, wherein the at least one processing device is further programmed to:

identify an object in one or more of the plurality of images;

assess whether the object presents a hazard; and

determine an action to be taken to avoid a collision with the object, based on the assessment.

9. The detection system of claim 8, wherein assessing whether the object presents a hazard includes classifying the object based on the identified regions and at least one of a location or a trajectory of the object.

10. The detection system of claim 9, wherein classifying the object includes classifying the object as on the road surface or on the off-road surface.

11. A vehicle, comprising:

a body; and

at least one processing device programmed to;

receive a plurality of images of an area output from at least one camera via a data interface;

determine a plurality of road edge line candidates from and within the plurality of images of the area; and

identify, based on the road edge line candidates, a region in the plurality of images that corresponds to a road surface and a region in the plurality of images that corresponds to an off-road surface.

12. The vehicle of claim 11, wherein the at least one processing device is further programmed to:

identify a pedestrian in one or more of the plurality of images;

assess whether the pedestrian presents a hazard; and

determine an action to be taken to avoid a collision with the pedestrian, based on the assessment.

13. The vehicle of claim 12, wherein assessing whether the pedestrian presents a hazard includes classifying the pedestrian based on the identified regions and at least one of a location or a trajectory of the pedestrian.

14. The vehicle of claim 13, wherein classifying the pedestrian includes classifying the pedestrian as on the road surface or on the off-road surface.

15. The vehicle of claim 12, wherein assessing whether the pedestrian presents a hazard includes comparing a trajectory of the pedestrian to a trajectory of a vehicle.

16. The vehicle of claim 12, wherein the determined action to be taken to avoid the collision includes providing an alert to a driver.

17. The vehicle of claim 12, wherein the determined action to be taken to avoid the collision includes modifying at least one of a speed, acceleration, or direction of a vehicle.

18. The vehicle of claim 11, wherein the at least one processing device is further programmed to:

identify an object in one or more of the plurality of images;

assess whether the object presents a hazard; and

determine an action to be taken to avoid a collision with the object, based on the assessment.

19. The vehicle of claim 18, wherein assessing whether the object presents a hazard includes classifying the object based on the identified regions and at least one of a location or a trajectory of the object, and classifying the object includes classifying the object as on the road surface or on the off-road surface.

20. A method for hazard assessment, the method comprising the following operations performed by one or more processors:

receiving a plurality of images of an area output from at least one camera via a data interface;

determining a plurality of road edge line candidates from and within the plurality of images of the area; and

identifying, based on the road edge line candidates, a region in the plurality of images that corresponds to a road surface and a region in the plurality of images that corresponds to an off-road surface.