Method and device for visualizing the surroundings of a vehicle by fusing an infrared image and a visual image

• US 20080024608A1
• Filed: 08/07/2007
• Published: 01/31/2008
• Est. Priority Date: 02/11/2005
• Status: Active Grant

• Current Assignee: Bayerische Motoren Werke AG
• Original Assignee: Bayerische Motoren Werke AG
• Inventors: Weidner, Thomas, Hahn, Wolfgang
• Granted Patent: US 9,088,737 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60R 1/22 for viewing an area outside the vehicle, e.g. the exterior of the vehicle
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/304 using merged images, e.g. merging camera image with stored images
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/20 for generating image signals from infrared radiation only
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A method for visualizing surroundings of a vehicle, especially in the dark is provided. A visual image, containing the digital data of the surroundings shows the visually perceptible objects. An infrared image, containing the digital data of the surroundings, shows the infrared radiation, emitted by the visually perceptible and/or other objects. An image fusion takes place that fuses the visual image and the infrared image into a target image, which can be represented in an image display unit, in order to simplify the allocation of the infrared radiation-emitting objects in the recorded environment. The image fusion is interrupted as a function of one environment parameter in order to represent only one of the two images, or none of the images, as the target image.

Claims

1. A method for visualizing surroundings of a vehicle, the method comprising the acts of:

obtaining a visual image, containing digital data of the surroundings, in order to show visually perceptible objects;

obtaining an infrared image, containing digital data of the surroundings, in order to show infrared radiation emitted by the visually perceptible objects and/or other objects;

fusing the visual image and the infrared image into a target image, which target image is representable in a display unit, in order to simplify an allocation of the infrared radiation emitting objects in a recorded environment, and

interrupting the fusing of the visual image and the infrared image as a function of an environment parameter in order to represent only one of the visual and infrared images, or none of the images, as the target image.

2. The method according to claim 1, wherein the interrupting act further comprises the act of blanking-out all of the digital data of the surroundings of the respective image in order to avoid image distortion.

3. The method according to claim 1, wherein the environment parameter is determined from at least one of a driving dynamic variable sensed via a sensor and at least one other parameter.

4. The method according to claim 3, wherein a vehicle speed related parameter is processed as the driving dynamic variable.

5. The method according to claim 4, wherein the vehicle speed related variable is one of an undershooting and overshooting of a predefined speed.

6. The method according to claim 3, wherein a distance related variable between the vehicle and an object is processed as the driving dynamic variable.

7. The method according to claim 6, wherein the distance related variable is one of an undershooting and overshooting of a predefined distance between the vehicle and the object.

8. The method according to claim 3, wherein a current position of the vehicle is processed as the other parameter.

9. The method according to claim 3, wherein at least one of a topology and a current weather condition is processed as the at least one other parameter.

10. The method according to claim 3, wherein a user selected parameter is processed as the other parameter.

11. The method according to claim 3, further comprising the acts of:

determining entropy values of the visual image and the infrared image;

comparing the entropy values with a predefined entropy value; and

determining whether the visual image, the infrared image, and/or both the visual and infrared images, is to be blanked-out based on results of the comparing act.

12. The method according to claim 1, wherein the interrupting act further comprises the act of ceasing the fusing act based upon a time hysteresis.

13. A system for visualizing surroundings of a vehicle, comprising:

a visual camera providing a visual image, containing digital data of the surroundings, the visual image showing visually perceptible objects;

an infrared camera providing an infrared image, containing digital data of the surroundings, the infrared image showing infrared radiation emitted by the visually perceptible objects and/or other objects;

an image processing unit for processing the visual image and the infrared image, the image processing unit fusing the visual image and the infrared image into a target image;

an image display unit which displays the target image from the image processing unit; and

wherein the image processing unit is operatively configured to interrupt the fusing of the visual image and the infrared image as a function of an environment parameter.

14. The system according to claim 13, further comprising at least one sensor, coupled to the image processing unit, for supplying a driving dynamic variable utilized as the environment parameter.

15. The system according to claim 14, wherein an additional parameter is supplied to the image processing unit for utilization in interrupting the fusing of the visual image and the infrared image.

16. The system according to claim 13, wherein the image processing unit is operatively configured to determine the environment parameter based on a driving dynamic variable and/or an additional parameter.

17. The system according to claim 13, wherein the visual camera and the infrared camera form a camera system calibrated with respect to a fixed distance range.

18. A method of operating an image fusion system utilized for visualizing surroundings of a vehicle, the image fusion system operating to fuse a visual image containing digital data of the surroundings showing visually perceptible objects and an infrared image containing digital data of the surroundings showing infrared radiation emitted by the visually perceptible objects and other objects, the operating method comprising the acts of:

receiving at least one environment parameter of the vehicle; and

interrupting the fusion of the visual image and the infrared image as a function of the at least one environment parameter such that either only the visual image or the infrared image, or neither image, is displayed to a user of the vehicle.

19. The operating method according to claim 18, wherein the interrupting is performed by blanking-out all of the digital data of the respective image or images.

20. The method according to claim 18, wherein the at least one environment parameter is one of a vehicle speed related variable and a vehicle distance related variable.