Forward-facing multi-imaging system for navigating a vehicle

• US 10,318,823 B2
• Filed: 10/14/2014
• Issued: 06/11/2019
• Est. Priority Date: 10/14/2013
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Shashua, Amnon, Hayon, Gaby, Hadad, Yossi, Belman, Efim, Bagon, Eyal
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US14/513,542
• Publication Number:
  US 20150103159A1
• Time in Patent Office: 1,701 Days
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Systems and methods use cameras to provide autonomous navigation features. In one implementation, a driver-assist system is provided for a vehicle. The system may include one or more image capture devices configured to acquire images of an area forward of the vehicle. The system may also include at least one processing device configured to receive, via one or more data interfaces, the images. The at least one processing device may be further configured to analyze the images acquired by the one or more image capture devices and cause at least one navigational response in the vehicle based on monocular and/or stereo image analysis of the images.

Claims

1. A driver-assist system for a vehicle, the system comprising:

a first image capture device configured to acquire a first plurality of images of an area forward of the vehicle, wherein the first image capture device has a first angular field of view;

a second image capture device configured to acquire a second plurality of images of an area forward of the vehicle, wherein the second image capture device has a second angular field of view greater than the first angular field of view, the second angular field of view having at least a portion overlapping with the first angular field of view;

a third image capture device configured to acquire a third plurality of images of an area forward of the vehicle, wherein the third image capture device has a third angular field of view greater than the second angular field of view, the third angular field of view having at least a portion overlapping with the first angular field of view and with the second angular field of view;

a data interface; and

at least one processing device configured to;

receive, via the data interface, the first, second, and third plurality of images;

analyze two or more images of the first, second, and third plurality of images to detect a feature associated with an environment of the vehicle and track a position of the detected feature relative to the vehicle; and

transmit a control signal generated in response to the detected and tracked feature to at least one of a plurality of vehicle control systems to cause the vehicle to take at least one navigational response based on image information derived from any two of the first, second, and third plurality of images.

2. The system of claim 1, wherein the first angular field of view is selected from within a range of 20 degrees to 40 degrees.

3. The system of claim 1, wherein the second angular field of view is selected from within a range of 40 degrees to 60 degrees.

4. The system of claim 1, wherein the third angular field of view is selected from within a range of 100 degrees to 190 degrees.

5. The system of claim 1, wherein the at least one processing device is further configured to:

cause the vehicle to take the at least one navigational response based at least partially on map data and a determined position of the vehicle.

6. The system of claim 1, wherein the at least one processing device is further configured to:

cause the vehicle to take the at least one navigational response based on a relative velocity between the vehicle and an object detected within any of the first, second, and third plurality of images, wherein the at least one processing device is configured to determine the relative velocity based on any of the first, second, and third plurality of images.

7. The system of claim 1, wherein the at least one processing device is further configured to:

cause the vehicle to take the at least one navigational response based on a relative acceleration between the vehicle and an object detected within any of the first, second, and third plurality of images, wherein the at least one processing device is configured to determine the relative acceleration based on any of the first, second, and third plurality of images.

8. The system of claim 1, wherein the derived image information identifies one or more lane marks.

9. The system of claim 1, wherein analyzing two or more images of the first, second, and third plurality of images to detect a feature associated with the environment of the vehicle includes:

calculating a disparity of pixels between images obtained by at least two different ones from the first image capture device, the second image capture device, and the third image capture device.

10. The system of claim 9, wherein analyzing two or more images of the first, second, and third plurality of images to detect a feature associated with the environment of the vehicle includes:

generating a three-dimensional reconstruction of the environment of the vehicle based on the calculated disparity.

11. The system of claim 10, wherein analyzing two or more images of the first, second, and third plurality of images to detect a feature associated with the environment of the vehicle includes:

combining the three-dimensional reconstruction of the environment with three-dimensional map data or three-dimensional information calculated based on images from one of the first, second, or third plurality of images.

12. The system of claim 9, wherein analyzing two or more images of the first, second, and third plurality of images to detect a feature associated with the environment of the vehicle includes:

calculating a camera displacement relating to at least one of the first image capture device, the second image capture device, and the third image capture device; and

calculating the disparity of pixels between successive images.

13. The system of claim 1, wherein the at least one processing device is further configured to select the two or more images of the first, second, and third plurality of images for analysis based on one or more objects detected in the first, second, and third plurality of images.

14. The system of claim 1, wherein the at least one processing device is further configured to select the two or more images of the first, second, and third plurality of images for analysis based on at least one of the following:

image quality, image resolution, effective field of view reflected in the images, the number of captured frames, a percentage of frames in which an object of interest appears, or a proportion of the object that appears in the frames.

15. A vehicle, comprising:

a body;

a first image capture device configured to acquire a first plurality of images of an area forward of the body of the vehicle, wherein the first image capture device has a first angular field of view;

a second image capture device configured to acquire a second plurality of images of an area forward of the body of the vehicle, wherein the second image capture device has a second angular field of view greater than the first angular field of view, the second angular field of view having at least a portion overlapping with the first angular field of view;

a third image capture device configured to acquire a third plurality of images of an area forward of the body of the vehicle, wherein the third image capture device has a third angular field of view greater than the second angular field of view, the third angular field of view having at least a portion overlapping with the first angular field of view and with the second angular field of view;

a plurality of vehicle control systems;

a data interface; and

at least one processing device configured to;

receive, via the data interface, the first, second, and third plurality of images;

analyze two or more images of the first, second, and third plurality of images to detect a feature associated with an environment of the vehicle and track a position of the detected feature relative to the vehicle; and

transmit a control signal generated in response to the detected and tracked feature to at least one of the plurality of vehicle control systems to cause the vehicle to take at least one navigational response based on image information derived from any two of the first, second, and third plurality of images.

16. The vehicle of claim 15, wherein the at least one processing device is further configured to:

cause the vehicle to take the at least one navigational response based at least partially on map data and a determined position of the vehicle.

17. The vehicle of claim 15, wherein the at least one processing device is further configured to:

cause the vehicle to take the at least one navigational response based on a relative velocity between the vehicle and an object detected within any of the first, second, and third plurality of images, wherein the at least one processing device is configured to determine the relative velocity based on any of the first, second, and third plurality of images.

18. The vehicle of claim 15, wherein the at least one processing device is further configured to:

cause the vehicle to take the at least one navigational response based on a relative acceleration between the vehicle and an object detected within any of the first, second, and third plurality of images, wherein the at least one processing device is configured to determine the relative acceleration based on any of the first, second, and third plurality of images.

19. The vehicle of claim 15, wherein the derived image information identifies one or more lane marks.