Fusion of Images in Enhanced Obstacle Detection

• US 20110018700A1
• Filed: 07/26/2010
• Published: 01/27/2011
• Est. Priority Date: 05/31/2006
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: Mobileye Technologies Limited (Intel Corporation)
• Inventors: Livyatan, Harel, Shashua, Amnon, Gdalyahu, Yoram, Stein, Gideon
• Granted Patent: US 8,378,851 B2
• Time in Patent Office: Days
• US Class Current: 340/436
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/304 using merged images, e.g. merging camera image with stored images
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A vehicle warning system that includes: a first camera mounted on the vehicle at a first longitudinal position along the length of the vehicle, which acquires consecutively in real time a plurality of first camera image frames; a second camera mounted on the vehicle at a second longitudinal position along the length of the vehicle, which acquires second camera image frames; and a processor configured to use information in both first and second camera image frames to determine whether there is a danger of a collision between the vehicle and an object shown in both the first and second camera images. The first and second longitudinal positions may be at different positions along the length of the vehicle.

Claims

1. A vehicle warning system, comprising:

a first camera mounted on the vehicle at a first longitudinal position along the length of the vehicle, which acquires consecutively in real time a plurality of first camera image frames;

a second camera mounted on the vehicle at a second longitudinal position along the length of the vehicle, which acquires second camera image frames; and

a processor configured to use information in both first and second camera image frames to determine whether there is a danger of a collision between the vehicle and an object shown in both the first and second camera images,

wherein the first and second longitudinal positions are at different positions along the length of the vehicle.

2. The system of claim 1, wherein the first longitudinal position is in a cabin of the vehicle.

3. The system of claim 1, wherein the second longitudinal position is in front of the engine of the vehicle.

4. The system of claim 1, wherein the first and second longitudinal positions are separated by between 1-2 meters from each other.

5. The system of claim 1, wherein the first and second cameras are mounted on the vehicle at substantially different heights.

6. The system of claim 1, wherein the first and second cameras are mounted on the vehicle at substantially different lateral locations.

7. The system of claim 1, wherein the second camera is directed to acquire images in the same direction as the first camera.

8. The system of claim 1, wherein the first and second cameras are mounted on the vehicle such that their acquired images have a wide baseline for stereo analysis.

9. The system of claim 1, wherein the optical axes of the first and second camera are aligned.

10. The system of claim 1, wherein the processor is configured to locate an object in a second camera image frame, to determine the distance between the vehicle and the object responsive to the location of the detected object in both the first and second camera image frame and to determine whether there is a danger of a collision between the vehicle and the object responsive to the determined distance.

11. The system of claim 1, wherein the processor is configured to select an area in a second camera image frame, and determine a corresponding area in a first camera image frame.

12. The system of claim 1, wherein the first and second cameras are directed forwards to acquire images in the direction of travel of the vehicle.

13. The system of claim 1, wherein the first and second cameras are directed backwards to acquire images in the direction opposite the direction of travel of the vehicle.

14. The system of claim 1, wherein the processor is adapted to locate the object in the second camera image frame using information from the one or more first camera image frames.

15. A method of detecting a danger of a collision of a vehicle, comprising:

acquiring by a first camera mounted on the vehicle at a first longitudinal position along the length of the vehicle, a plurality of first camera image frames, consecutively in real time;

acquiring by a second camera mounted on the vehicle at a second longitudinal position along the length of the vehicle, different from the first longitudinal position, second camera image frames; and

determining, by a processor coupled to the first and second cameras, using information in both the first and second camera image frames, whether there is a danger of a collision between the vehicle and an object shown in both the first and second camera images.

16. The method of claim 15, wherein the first and second longitudinal positions are separated by between 1-2 meters from each other.

17. The method of claim 15, wherein determining whether there is a danger of a collision comprises determining a distance to the object responsive to both first and second camera images.

18. The method of claim 15, wherein the optical axes of the first and second cameras are aligned.

19. The method of claim 15, wherein determining whether there is a danger of a collision comprises locating the object in the second camera image frame using information from the one or more first camera image frames.

20. The method of claim 15, wherein second camera comprises an infrared camera.