Systems and methods for detecting low-height objects in a roadway

• US 10,317,910 B2
• Filed: 11/06/2015
• Issued: 06/11/2019
• Est. Priority Date: 01/30/2014
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon, Shashua, Amnon
• Primary Examiner(s): Whittington, Jess
• Application Number:
  US14/934,304
• Publication Number:
  US 20160167704A1
• Time in Patent Office: 1,313 Days
• CPC Class Codes: B60K 31/00 Vehicle fittings, acting on a single sub-unit only, for automatically controlling vehicle speed, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator fittings acting on two or more sub-units B60W30/14; speedometers G01P; systems or devices for controlling speed in general G05D13/00 ; in traffic anti-collision system for road vehicles G08G1/16
• CPC Class Codes: B60T 7/12 for automatic initiation; for initiation not subject to will of driver or passenger limiting speed of vehicles other than rail vehicles B60K31/00
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2552/15 Road slope, i.e. the inclination of a road segment in the longitudinal direction
• CPC Class Codes: B60W 2552/30 Road curve radius
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/4042 Longitudinal speed
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/806 Relative heading
• CPC Class Codes: B60W 2555/60 Traffic rules, e.g. speed limits or right of way
• CPC Class Codes: B60W 2710/207 Steering angle of wheels
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60W 30/14 Adaptive cruise control
• CPC Class Codes: B60W 30/18163 Lane change; Overtaking manoeuvres
• CPC Class Codes: B60W 60/00276 : for two or more other traffic participants
• CPC Class Codes: B62D 6/00 : Arrangements for automatically controlling steering depending on driving conditions sensed and responded to, e.g. control circuits means for initiating a change in direction B62D1/00; steering valves B62D5/06; combined with means for inclining the vehicle body or wheels on bends B62D9/00
• CPC Class Codes: G01S 19/42 : Determining position
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0276 : using signals provided by a source external to the vehicle involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058
• CPC Class Codes: G06T 2207/30256 : Lane; Road marking
• CPC Class Codes: G06T 2207/30261 : Obstacle
• CPC Class Codes: G06T 7/62 : of area, perimeter, diameter or volume
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 : of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/095 : Traffic lights
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

Systems and methods use cameras to provide autonomous navigation features. In one implementation, a driver-assist object detection system is provided for a vehicle. One or more processing devices associated with the system receive at least two images from a plurality of captured images via a data interface. The device(s) analyze the first image and at least a second image to determine a reference plane corresponding to the roadway the vehicle is traveling on. The processing device(s) locate a target object in the first two images, and determine a difference in a size of at least one dimension of the target object between the two images. The system may use the difference in size to determine a height of the object. Further, the system may cause a change in at least a directional course of the vehicle if the determined height exceeds a predetermined threshold.

Claims

1. A driver-assist object detection system for a vehicle, the system comprising:

at least one image sensor configured to capture a plurality of images representative of an area surrounding the vehicle;

a data interface; and

at least one processor configured to;

receive, via the data interface, a first image from among the plurality of images and at least a second image from among the plurality of images;

analyze the first image and at least the second image to determine a reference plane corresponding to a road plane;

locate a target object in the first image and in the second image;

determine a difference in a size of at least one spatial dimension of the target object between the first image and the second image;

use the determined difference in the size of the at least one spatial dimension and a detected velocity of the vehicle to determine at least a height of the target object relative to the road plane; and

cause a change in at least a directional course of the vehicle if the determined height of the target object exceeds a predetermined threshold.

2. The driver-assist object detection system of claim 1, wherein the at least one processor is further configured to determine a road clearance height of the vehicle.

3. The driver-assist object detection system of claim 2, wherein if the determined height of the target object is less than the road clearance height of the vehicle, the at least one processor is further configured to:

determine that the vehicle can change directional course to drive over the target object with the target object oriented between wheels of the vehicle; and

cause a steering system of the vehicle to change the directional course of the vehicle such that the vehicle drives over the target object with the target object oriented between the wheels of the vehicle.

4. The driver-assist object detection system of claim 1, wherein the image sensor has a field of view of between 35 degrees to 55 degrees in a width direction of the vehicle.

5. The driver-assist object detection system of claim 1, wherein the predetermined threshold is at least 2 cm from a surface of the road.

6. The driver-assist object detection system of claim 1, wherein the at least one processor is further configured to determine the road plane as an average surface level associated with a road surface appearing in the plurality of images.

7. The driver-assist object detection system of claim 6, wherein the at least one processor is further configured to detect a change in composition of the road surface.

8. The driver-assist object detection system of claim 1, wherein the determined height of the target object is less than 10 cm relative to the road plane.

9. The driver-assist object detection system of claim 1, wherein the first image and the second image are captured by the same image sensor.

10. The driver-assist object detection system of claim 1, wherein using the detected velocity of the vehicle to determine the height of the target object relative to the road plane comprises determining an optical flow using the detected velocity and using the optical flow to determine the height.

11. A vehicle, comprising:

a body;

at least one image sensor configured to capture a plurality of images representative of an area surrounding the vehicle;

a data interface; and

at least one processor configured to;

receive, via the data interface, a first image from among the plurality of images and at least a second image from among the plurality of images;

analyze the first image and at least the second image to determine a reference plane corresponding to a road plane;

locate a target object in the first image and in the second image;

determine a difference in a size of at least one spatial dimension of the target object between the first image and the second image;

use the determined difference in the size of the at least one spatial dimension and a detected velocity of the vehicle to determine at least a height of the target object relative to the road plane; and

cause a change in at least a directional course of the vehicle if the determined height of the target object exceeds a predetermined threshold.

12. The vehicle of claim 11, wherein the at least one processor is further configured to determine a road clearance height of the vehicle.

13. The vehicle of claim 12, wherein if the determined height of the target object is less than the road clearance of the vehicle, the at least one processor is further configured to:

determine that the vehicle can change directional course to drive over the target object with the target object oriented between wheels of the vehicle; and

cause a steering system of the vehicle to change the directional course of the vehicle such that the vehicle drives over the target object with the target object oriented between the wheels of the vehicle.

14. The vehicle of claim 11, wherein the image sensor has a field of view of between 35 degrees to 55 degrees in a width direction of the vehicle.

15. The vehicle of claim 11, wherein the predetermined threshold is at least 2 cm from the road surface.

16. The vehicle of claim 11, wherein the at least one processor is further configured to determine the road plane as an average surface level associated with a road surface appearing in the plurality of images.

17. The vehicle of claim 16, wherein the at least one processor is further configured to detect a change in composition of the road surface.

18. The vehicle of claim 11, wherein the determined height of the target object is less than 10 cm relative to the road plane.

19. A computer-implemented method for detecting an object in a roadway, the method comprising:

capturing, via at least one image sensor, a plurality of images representative of an area surrounding a vehicle;

receiving, via at least one processor, a first image from among the plurality of images and at least a second image from among the plurality of images;

analyzing, via the at least one processor, the first image and at least the second image to determine a reference plane corresponding to a road plane;

locating, via the at least one processor, a target object in the first image and in the second image;

determining, via the at least one processor, a difference in a size of at least one spatial dimension of the target object between the first image and the second image;

using, via the at least one processor, the determined difference in the size of the at least one spatial dimension and a detected velocity of the vehicle to determine at least a height of the target object relative to the road plane; and

causing, via the at least one processor, a change in at least a directional course of the vehicle if the determined height of the target object exceeds a predetermined threshold.

20. The method of claim 19, wherein the predetermined threshold is at least 2 cm from a surface of the roadway.