Fusion of images in enhanced obstacle detection

• US 8,378,851 B2
• Filed: 07/26/2010
• Issued: 02/19/2013
• Est. Priority Date: 05/31/2006
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: Mobileye Technologies Limited (Intel Corporation)
• Inventors: Stein, Gideon, Shashua, Amnon, Gdalyahu, Yoram, Livyatan, Harel
• Primary Examiner(s): TRIEU, VAN THANH
• Application Number:
  US12/843,368
• Publication Number:
  US 20110018700A1
• Time in Patent Office: 939 Days
• Field of Search: 340/435, 340/436, 340/437, 340/903, 340/904, 340/907, 340/937, 348/148, 348/42, 348/E7.087, 250/330, 250/336.1, 250/338, 180/167, 180/170, 701/301, 382/104, 382/291, 396/301
• US Class Current: 340/937
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/304 using merged images, e.g. merging camera image with stored images
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A vehicle warning system that includes: a first camera mounted on the vehicle at a first longitudinal position along the length of the vehicle, which acquires consecutively in real time a plurality of first camera image frames; a second camera mounted on the vehicle at a second longitudinal position along the length of the vehicle, which acquires second camera image frames; and a processor configured to use information in both first and second camera image frames to determine whether there is a danger of a collision between the vehicle and an object shown in both the first and second camera images. The first and second longitudinal positions may be at different positions along the length of the vehicle.

Claims

1. A system comprising:

a first camera mountable on a vehicle;

a second camera mountable on the vehicle behind the first camera; and

a processor operatively connected to said first and second cameras, wherein said processor is adapted to acquire a first image frame from said first camera and to acquire a second image frame from said second camera;

wherein said processor is configured to locate a first feature of a first image of an object in the first image frame and a second feature of a second image of said object in the second image frame and determine alignment thereby of the first and second image frames,

wherein respective locations of the first feature in the first image frame and the second feature in the second image frame are responsive to respective positions of the second camera behind the first camera, and

wherein said processor is configured to compute a distance between said vehicle and said object based on said alignment.

2. The system of claim 1, wherein the second camera is mounted in a cabin of the vehicle.

3. The system of claim 1, wherein the first camera is mounted in front of the engine of the vehicle.

4. The system of claim 1, wherein the first camera is mounted between 1 and 2 meters forward of the second camera such that the second camera is mounted between 1 and 2 meters behind the first camera.

5. The system of claim 1, wherein the first and second cameras are further mounted on the vehicle at substantially different heights above road surface.

6. The system of claim 5, wherein the first and second cameras are mounted on the vehicle such that said first and second image frames provide a wide baseline for stereo analysis.

7. The system of claim 1, wherein the first and second cameras are further mounted on the vehicle at substantially different lateral locations in a dimension perpendicular to the forward direction of movement of the vehicle.

8. The system of claim 1, wherein the first and second cameras are mounted on the vehicle such that said first and second image frames provide a wide baseline for stereo analysis.

9. The system of claim 1, wherein the processor is configured to determine whether there is a danger of a collision between the vehicle and the object responsive to the computed distance.

10. The system of claim 1, wherein the processor is configured to select an area in a second camera image frame, and determine a corresponding area in a first camera image frame.

11. The system of claim 1, wherein the first and second cameras are directed forwards to acquire images in the direction of travel of the vehicle.

12. The system of claim 1, wherein the first and second cameras are directed backwards to acquire images in the direction opposite the direction of travel of the vehicle.

13. The system of claim 1, wherein the processor is adapted to locate the object in the second camera image frame using information from the one or more first camera image frames.

14. The system of claim 1, wherein the first camera is sensitive to far infrared electromagnetic radiation.

15. The system of claim 1, wherein said first camera and said second camera have different fields of view.

16. The system of claim 1, wherein said first camera is sensitive to far infrared electromagnetic radiation to enable detection of a pedestrian at night.

17. The system of claim 1, wherein said first camera and said second camera are monocular cameras.

18. The system of claim 1, wherein said first and second features are selected from the group consisting of:

gray levels, histograms and edges.

19. A method performable by a system mountable on a vehicle, the system including a first camera, a second camera mountable behind the first camera and a processor operatively connected to the first and second cameras, the method comprising:

acquiring a first image frame from the first camera;

acquiring a second image frame from the second camera;

locating a first feature of a first image of an object in the first image frame and a second feature of a second image of said object in the second image frame, thereby aligning the first and second image frames, wherein respective locations of the first feature in the first image frame and the second feature in the second image frame are responsive to respective positions of the second camera behind the first camera; and

comprising a distance between said vehicle and said object based on said aligning.

20. The method of claim 19, wherein the second camera is mounted a distance of 1-2 meters to the rear of the first camera.

21. The method of claim 19, further comprising:

locating the second feature of the object in said second image frame using information from said first image frame.

22. The method of claim 19, wherein said aligning is performed by measuring similarity between respective histograms of gray levels of said first image and said second image.

23. The method of claim 19, wherein said aligning is performed by correlating between sub patches between said first image and said second image.

24. The method of claim 19, wherein said aligning is performed by minimizing Hausdorf distance between edge points in said first image and edge points in said second image.