Systems and methods for mimicking a leading vehicle

• US 9,150,220 B2
• Filed: 12/04/2014
• Issued: 10/06/2015
• Est. Priority Date: 12/04/2013
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Clarke, Anna, Bagon, Eyal
• Primary Examiner(s): ARTHUR JEANGLAUD, GERTRUDE
• Application Number:
  US14/560,938
• Publication Number:
  US 20150151742A1
• Time in Patent Office: 306 Days
• Field of Search: 701/45, 701/93, 701/301, 340/435, 340/436, 340/903, 340/905, 340/937
• US Class Current: 1/1
• CPC Class Codes: B60K 2031/0025 Detecting position of target vehicle, e.g. vehicle driving ahead from host vehicle
• CPC Class Codes: B60K 31/0066 responsive to vehicle path curvature
• CPC Class Codes: B60T 13/662 characterised by specified functions of the control system components
• CPC Class Codes: B60T 2201/022 Collision avoidance systems
• CPC Class Codes: B60T 2201/03 Brake assistants
• CPC Class Codes: B60T 2201/08 Lane monitoring; Lane Keeping Systems
• CPC Class Codes: B60T 2210/34 Blind spots
• CPC Class Codes: B60T 7/12 for automatic initiation; for initiation not subject to will of driver or passenger limiting speed of vehicles other than rail vehicles B60K31/00
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60T 8/32 responsive to a speed condition, e.g. acceleration or deceleration using electrical circuitry or regulation means B60T8/17 ; B60T8/28 takes precedence; electric devices on electrically propelled vehicles indicating the wheel slip B60L3/10; measuring linear or angular speed per se G01P3/00
• CPC Class Codes: B60T 8/321 deceleration B60T8/34, B60T8/52, B60T8/54, B60T8/56, B60T8/58, B60T8/72, B60T8/86, B60T8/88 take precedence
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/0075 Automatic parameter input, automatic initialising or calibrating means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2510/0604 Throttle position
• CPC Class Codes: B60W 2510/18 Braking system
• CPC Class Codes: B60W 2510/20 Steering systems
• CPC Class Codes: B60W 2520/14 Yaw
• CPC Class Codes: B60W 2552/30 Road curve radius
• CPC Class Codes: B60W 2552/53 : Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/4041 : Position
• CPC Class Codes: B60W 2554/80 : Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/806 : Relative heading
• CPC Class Codes: B60W 2556/10 : Historical data
• CPC Class Codes: B60W 2556/50 : of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 2710/18 : Braking system
• CPC Class Codes: B60W 2710/20 : Steering systems
• CPC Class Codes: B60W 30/00 : Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 30/08 : Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 30/09 : Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0953 : the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/12 : Lane keeping
• CPC Class Codes: B60W 30/14 : Adaptive cruise control
• CPC Class Codes: B60W 30/143 : Speed control B60W30/16 takes precedence
• CPC Class Codes: B60W 30/165 : Automatically following the path of a preceding lead vehicle, e.g. "electronic tow-bar"
• CPC Class Codes: B60W 30/18154 : Approaching an intersection
• CPC Class Codes: B60W 30/18163 : Lane change; Overtaking manoeuvres
• CPC Class Codes: B60W 40/06 : Road conditions
• CPC Class Codes: B60W 40/072 : Curvature of the road
• CPC Class Codes: B60W 40/076 : Slope angle of the road
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0027 : using trajectory prediction for other traffic participants
• CPC Class Codes: B62D 15/025 : Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: B62D 15/0255 : Automatic changing of lane, e.g. for passing another vehicle
• CPC Class Codes: B62D 15/0265 : Automatic obstacle avoidance by steering
• CPC Class Codes: B62D 6/00 : Arrangements for automatically controlling steering depending on driving conditions sensed and responded to, e.g. control circuits means for initiating a change in direction B62D1/00; steering valves B62D5/06; combined with means for inclining the vehicle body or wheels on bends B62D9/00
• CPC Class Codes: G01C 11/04 : Interpretation of pictures
• CPC Class Codes: G01C 21/10 : by using measurements of speed or acceleration G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/26 : specially adapted for navigation in a road network
• CPC Class Codes: G01C 21/30 : Map- or contour-matching
• CPC Class Codes: G01S 5/0027 : of actual mobile position, i.e. position determined on mobile
• CPC Class Codes: G05D 1/0088 : characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0214 : in accordance with safety or protection criteria, e.g. avoiding hazardous areas monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0253 : extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G05D 1/0278 : using satellite positioning signals, e.g. GPS
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30256 : Lane; Road marking
• CPC Class Codes: G06T 2207/30261 : Obstacle
• CPC Class Codes: G06T 7/285 : using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/292 : Multi-camera tracking
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/97 : Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 : of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/09 : Arrangements for giving variable traffic instructions railroad crossing signals B61L; reflectors E01F, G08B; indicating arrangements for variable information by selection or combination of individual elements G09F9/00
• CPC Class Codes: G08G 1/09623 : Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle G08G1/0967 takes precedence
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: G08G 1/22 : Platooning, i.e. convoy of communicating vehicles
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Systems and methods use cameras to provide autonomous navigation features. In one implementation, a driver assist navigation system is provided for a primary vehicle. The system may include at least one image capture device configured to acquire a plurality of images of an area in a vicinity of the primary vehicle; a data interface; and at least one processing device. The at least one processing device may be configured to: locate in the plurality of images a leading vehicle; determine, based on the plurality of images, at least one action taken by the leading vehicle; and cause the primary vehicle to mimic the at least one action of the leading vehicle.

Claims

1. A driver assist navigation system for a primary vehicle, the system comprising:

at least one image capture device configured to acquire a plurality of images of an area in a vicinity of the primary vehicle;

a data interface; and

at least one processing device configured to;

receive the plurality of images via the data interface;

determine, from at least some of the plurality of images, a first lane constraint on a first side of the primary vehicle;

determine, from at least some of the plurality of images, a second lane constraint on a second side of the primary vehicle opposite to the first side of the primary vehicle, wherein the first and second lane constraints define a lane within which the primary vehicle travels;

cause the primary vehicle to travel within the first and second lane constraints;

locate in the plurality of images a leading vehicle;

determine, based on the plurality of images, at least one action taken by the leading vehicle; and

cause the primary vehicle to mimic the at least one action of the leading vehicle,

wherein the at least one action taken by the leading vehicle includes providing a first offset distance on a side of the leading vehicle adjacent to the first lane constraint that is different from a second offset distance on a side of the leading vehicle adjacent to the second lane constraint.

2. The system of claim 1, wherein at least one of the first and second lane constraints includes a line marked on a road surface.

3. The system of claim 1, wherein at least one of the first and second lane constraints includes an edge of a road surface.

4. The system of claim 1, wherein at least one of the first and second lane constraints includes a determined midpoint of a road surface width.

5. The system of claim 1, wherein the at least one action taken by the leading vehicle includes a turn, and the at least one processing device causes the primary vehicle to execute a similar turn beginning at a location at which the leading vehicle initiated its turn.

6. The system of claim 1, wherein the at least one action taken by the leading vehicle includes an acceleration.

7. The system of claim 1, wherein the at least one action taken by the leading vehicle includes braking.

8. A driver assist navigation system for a primary vehicle, the system comprising:

at least one image capture device configured to acquire a plurality of images of an area in a vicinity of the primary vehicle;

a data interface; and

at least one processing device configured to;

receive the plurality of images via the data interface;

determine, from at least some of the plurality of images, a first lane constraint on a first side of the primary vehicle;

determine, from at least some of the plurality of images, a second lane constraint on a second side of the primary vehicle opposite to the first side of the primary vehicle, wherein the first and second lane constraints define a lane within which the primary vehicle travels;

cause the primary vehicle to travel within the first and second lane constraints;

locate in the plurality of images a leading vehicle;

determine, based on the plurality of images, at least one action taken by the leading vehicle;

cause the primary vehicle to mimic the at least one action of the leading vehicle;

determine position information for the leading vehicle and compare the position information to predetermined map data;

create a navigation history for the leading vehicle by tracking movements of the leading vehicle relative to the predetermined map data; and

determine whether to cause the primary vehicle to mimic the at least one action of the leading vehicle based on the navigation history of the leading vehicle.

25. The system of claim 8, wherein the at least one processing device is configured to:

cause the primary vehicle to mimic a turn of the leading vehicle, even where no turn is detected based on the plurality of images, when the navigation history indicates that the leading vehicle has successfully navigated at least one prior turn.

9. A primary vehicle, comprising:

a body;

at least one image capture device configured to acquire a plurality of images of an area in a vicinity of the primary vehicle;

a data interface; and

at least one processing device configured to;

receive the plurality of images via the data interface;

determine, from at least some of the plurality of images, a first lane constraint on a first side of the primary vehicle;

determine, from at least some of the plurality of images, a second lane constraint on a second side of the primary vehicle opposite to the first side of the primary vehicle, wherein the first and second lane constraints define a lane within which the primary vehicle travels;

cause the primary vehicle to travel within the first and second lane constraints;

locate in the plurality of images a leading vehicle;

determine, based on the plurality of images, at least one action taken by the leading vehicle; and

cause the primary vehicle to mimic the at least one action of the leading vehicle,

wherein the at least one action taken by the leading vehicle includes providing a first offset distance on a side of the leading vehicle adjacent to the first lane constraint that is different from a second offset distance on a side of the leading vehicle adjacent to the second lane constraint.

10. The primary vehicle of claim 9, wherein at least one of the first and second lane constraints includes a line marked on a road surface.

11. The primary vehicle of claim 9, wherein at least one of the first and second lane constraints includes an edge of a road surface.

12. The primary vehicle of claim 9, wherein at least one of the first and second lane constraints includes a determined midpoint of a road surface width.

13. The primary vehicle of claim 9, wherein the at least one action taken by the leading vehicle includes a turn, and the at least one processing device causes the primary vehicle to execute a similar turn beginning at a location at which the leading vehicle initiated its turn.

14. The primary vehicle of claim 9, wherein the at least one action taken by the leading vehicle includes an acceleration.

15. The primary vehicle of claim 9, wherein the at least one action taken by the leading vehicle includes braking.

16. The primary vehicle of claim 9, wherein the at least one processing device is further configured to:

determine, based on the plurality of images or based on map data and position information, whether the leading vehicle is turning at an intersection; and

if the leading vehicle is determined to be turning at an intersection, decline to mimic the turn of the leading vehicle.

17. The primary vehicle of claim 9, wherein the at least one processing device is further configured to:

determine, based on the plurality of images or based on map data and position information, whether the leading vehicle is turning at an intersection;

determine whether the leading vehicle is changing lanes within the intersection;

mimic the turn of the leading vehicle if the leading vehicle is determined to be turning at an intersection and not changing lanes within the intersection; and

decline to mimic the turn of the leading vehicle if the leading vehicle is determined to be turning at an intersection, but changing lanes within the intersection.

18. The primary vehicle of claim 9, wherein the at least one processing device is further configured to:

acquire a first leading vehicle based on the plurality of images;

acquire a second leading vehicle based on the plurality of images; and

cause the primary vehicle to mimic the first leading vehicle if, based on the e plurality of images, the first leading vehicle is determined to be continuing on a path having less of a turning radius than a path followed by the second leading vehicle.

19. The primary vehicle of claim 9, wherein the at least one processing device is further configured to:

determine position information for the leading vehicle and compare the position information to predetermined map data;

create a navigation history for the leading vehicle by tracking movements of the leading vehicle relative to the predetermined map data; and

determine whether to cause the primary vehicle to mimic the at least one action of the leading vehicle based on the navigation history of the leading vehicle.

20. The primary vehicle of claim 19, wherein the at least one processing device is configured to:

cause the primary vehicle to mimic a turn of the leading vehicle, even where no turn is detected based on the plurality of images, when the navigation history indicates that the leading vehicle has successfully navigated at least one prior turn.

21. A method for navigating a primary vehicle, the method comprising:

acquiring, using at least one image capture device, a plurality of images of an area in a vicinity of the primary vehicle;

determining, from at least some of the plurality of images, a first lane constraint on a first side of the primary vehicle;

determining, from at least some of the plurality of images, a second lane constraint on a second side of the primary vehicle opposite to the first side of the primary vehicle, wherein the first and second lane constraints define a lane within which the primary vehicle travels;

causing the primary vehicle to travel within the first and second lane constraints;

locating in the plurality of images a leading vehicle;

determining, based on the plurality of images, at least one action taken by the leading vehicle; and

causing the primary vehicle to mimic the at least one action of the leading vehicle,

wherein the at least one action taken by the leading vehicle includes providing a first offset distance on a side of the leading vehicle adjacent to the first lane constraint that is different from a second offset distance on a side of the leading vehicle adjacent to the second lane constraint.

22. A driver assist navigation system for a primary vehicle, the system comprising:

at least one image capture device configured to acquire a plurality of images of an area in a vicinity of the primary vehicle;

a data interface; and

at least one processing device configured to;

receive the plurality of images via the data interface;

determine, from at least some of the plurality of images, a first lane constraint on a first side of the primary vehicle;

determine, from at least some of the plurality of images, a second lane constraint on a second side of the primary vehicle opposite to the first side of the primary vehicle, wherein the first and second lane constraints define a lane within which the primary vehicle travels;

cause the primary vehicle to travel within the first and second lane constraints;

locate in the plurality of images a leading vehicle;

determine, based on the plurality of images, at least one action taken by the leading vehicle;

cause the primary vehicle to mimic the at least one action of the leading vehicle;

determine, based on the plurality of images or based on map data and position information, whether the leading vehicle is turning at an intersection; and

if the leading vehicle is determined to be turning at an intersection, decline to mimic the turn of the leading vehicle.

23. A driver assist navigation system for a primary vehicle, the system comprising;

at least one image capture device configured to acquire a plurality of images of an area in a vicinity of the primary vehicle;

a data interface; and

at least one processing device configured to;

receive the plurality of images via the data interface;

determine, from at least some of the plurality of images, a first lane constraint on a first side of the primary vehicle;

determine, from at least some of the plurality of images, a second lane constraint on a second side of the primary vehicle opposite to the first side of the primary vehicle, wherein the first and second lane constraints define a lane within which the primary vehicle travels;

cause the primary vehicle to travel within the first and second lane constraints;

locate in the plurality of images a leading vehicle;

determine, based on the plurality of images, at least one action taken by the leading vehicle;

cause the primary vehicle to mimic the at least one action of the leading vehicle;

determine, based on the plurality of images or based on map data and position information, whether the leading vehicle is turning at an intersection;

determine whether the leading vehicle is changing lanes within the intersection;

mimic the turn of the leading vehicle if the leading vehicle is determined to be turning at an intersection and not changing lanes within the intersection; and

decline to mimic the turn of the leading vehicle if the leading vehicle is determined to be turning at an intersection, but changing lanes within the intersection.

24. A driver assist navigation system for a primary vehicle, the system comprising:

at least one image capture device configured to acquire a plurality of images of an area in a vicinity of the primary vehicle;

a data interface; and

at least one processing device configured to;

receive the plurality of images via the data interface;

determine, from at least some of the plurality of images, a first lane constraint on a first side of the primary vehicle;

determine, from at least some of the plurality of images, a second lane constraint on a second side of the primary vehicle opposite to the first side of the primary vehicle, wherein the first and second lane constraints define a lane within which the primary vehicle travels;

cause the primary vehicle to travel within the first and second lane constraints;

locate in the plurality of images a leading vehicle;

determine, based on the plurality of images, at least one action taken by the leading vehicle;

cause the primary vehicle to mimic the at least one action of the leading vehicle;

acquire a first leading vehicle based on the plurality of images;

acquire a second leading vehicle based on the plurality of images; and

cause the primary vehicle to mimic the first leading vehicle if, based on the plurality of images, the first leading vehicle is determined to be continuing on a path having less of a turning radius than a path followed by the second leading vehicle.