Method and device for visualizing the surroundings of a vehicle

• US 7,593,573 B2
• Filed: 04/14/2006
• Issued: 09/22/2009
• Est. Priority Date: 10/16/2003
• Status: Active Grant

• Current Assignee: Bayerische Motoren Werke AG
• Original Assignee: Bayerische Motoren Werke AG
• Inventors: Hahn, Wolfgang, Weidner, Thomas
• Primary Examiner(s): Bali; Vikkram
• Assistant Examiner(s): Bitar; Nancy
• Application Number:
  US11/404,079
• Publication Number:
  US 20060257024A1
• Time in Patent Office: 1,257 Days
• Field of Search: 382/181, 382/104, 382/115, 382/190, 382/118, 382/103, 348/164, 345/619, 250/330, 250/338.7, 250/332, 131/81, 131/04, 131/15
• US Class Current: 382/181
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A method and device for visualizing the surroundings of a vehicle, in particular for detecting and representing the traffic situation in front of a driver are provided. A night vision information system fuses video images from essentially the same location and time but from different spectral regions are fused using weighting factors. The video images are weighted with a globally settable preliminary weighting factor, and an object analysis of the video images is performed. Based on the object analysis the weighting factors are locally adapted in an object-selective manner.

Claims

1. A method for visualizing the surroundings of a vehicle by a night vision information system, the method comprising the acts of:

detecting video images, of different spectral regions, from essentially the same location and time;

weighting the video images with a globally settable preliminary weighting factor;

performing object analysis, using an object recognition system, of the video images;

locally adapting the weighting factors based on the object analysis in an object-selective manner to generate locally adapted weighting factors; and

fusing the video images based, at least in part, on the locally adapted weighting factors.

2. The method according to claim 1, wherein using the object recognition system includes performing an object evaluation for the detected objects according to their relevance, and the local weighting factors are adapted to the associated relevance of the particular object based on the object evaluation, so that specified detected objects are represented primarily in the spectral region in which they can be most clearly perceived on the fused video image by a driver of the vehicle.

3. The method according to claim 2, further comprising the act of:

dividing the detected objects for the object evaluation into object classes.

4. The method according to claim 1, wherein image regions in which no relevant objects are detected or whose relevance cannot be unambiguously assigned are weighted using the globally set weighting factor.

5. The method according to claim 1, wherein a continuous transition between adjacent relevant and irrelevant image regions is produced in the fused video image using a variable characteristic curve of the ratio of the weighting factors for different spectral regions.

6. The method according to claim 1, wherein fusing of the video images includes accounting for radiation portions having wavelengths from the visible spectral region and radiation portions having wavelengths from the infrared spectral region are taken into account.

7. The method according to claim 1, wherein color information is processed in the visual video images.

8. The method according to claim 1, further comprising suppressing, at least partially, masking and/or occlusion effects from detected objects by the adaptation of the local weighting factors.

9. A device for visualizing the surroundings of a vehicle, the device comprising:

a camera system that records video images from different spectral regions essentially at the same location and time;

a fusion device that fuses the video images in a weighted manner, wherein the fusion device globally applies a preliminary weighting factor to the video images;

an object recognition system, located upstream from the fusion device, which locates objects in the surroundings of a vehicle using images provided by the camera system; and

a control device which in operative connection with the object recognition system actuates the fusion device with an object-selective local spectral region weighting, wherein the fusion device locally adapts weighting factors based on a result of the object recognition system in an object-selective manner.

10. The device according to claim 9, wherein the camera system has at least one sensor that is sensitive in the visible spectral region, and at least one sensor that is sensitive in the infrared spectral region.

11. The device according to claim 9, further comprising:

a heads-up display on which the fused video images are presented to the driver.