Apparatus for determining the presence of fog using image obtained by vehicle-mounted device

• US 20080169912A1
• Filed: 01/10/2008
• Published: 07/17/2008
• Est. Priority Date: 01/11/2007
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Kawasaki, Naoki
• Granted Patent: US 8,786,697 B2
• Time in Patent Office: Days
• US Class Current: 340/425.5
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

An apparatus that detects the presence of a constituent such as fog affecting a view of space which is present ahead of a vehicle, the constituent resulting from a natural phenomenon in the space is provided. In this apparatus, the determination of whether the presence of the element is determined based on an image that captured by a vehicle-mounted camera and in which a picture of a high-luminance obstacle located on a road over which the vehicle is traveling is masked.

Claims

1. An apparatus that determines the presence of a constituent affecting a view of space which is present ahead of a vehicle, the constituent in the space, comprising:

means for capturing the image of space, the image being viewed from the vehicle and consisting of arrays of pixels having luminance values;

means for extracting a reference image from the image captured by the means for capturing the image, the reference image including a portion of the image of the space located a predetermined distance apart from the vehicle;

means for estimating the values of luminance of the arrays of the pixels in the reference image;

means for detecting a high-luminance obstacle from the reference image, the high-luminance obstacle being higher in luminance than a predetermined luminance value;

means for masking the area including the picture of the high-luminance obstacle determined by the means for detecting the high-luminance obstacle in the reference image to generate a masked reference image to neglect an effect of existence of the obstacle in determining the presence of the element based on the image captured by the means for capturing the image of space;

means for calculating a total luminance intensity as a function of the values of the luminance of the arrays of the pixels in the masked reference image as a total luminance intensity; and

means for determining of whether or not the constituent is present in the environmental atmosphere around the vehicle in accordance with both the total luminance intensity.

2. The apparatus according to claim 1, further comprising:

means for extracting a region of sky in the reference image based on the values of luminance of the pixels thereof and estimates a value of luminance of sky,

wherein the means for detecting the high-luminance obstacle performs to detect the high-luminance obstacle defined as an obstacle having higher luminance than that of the sky.

3. The apparatus according to claim 1, wherein

the means for calculating the total luminance intensity performs to transform luminance data including sequential values of luminance of the pixels in each individual array as a sum of base functions with respective coefficients and calculates a total spatial power spectrum of the image based on the coefficients calculated in the transformation of the luminance data, the masked reference image being expressed by sequential luminance values of the unmasked pixels in each individual array such that a length of the luminance data becomes shorter as the number of values of luminance of the unmasked pixels becomes smaller, and

the means for determining the presence of the element in the environmental atmosphere around the vehicle performs to determine the presence of the element in the environmental atmosphere around the vehicle in accordance with both the total spatial power spectrum using a fact that the element in the atmosphere scatters a light from the sun which results a smaller value of the total spatial power spectrum than that obtained in the case where the element is absent in the environmental atmosphere around the vehicle.

4. The apparatus according to claim 3, wherein

the constituent that affects the view of the space which is present ahead of the vehicle is fog.

5. The method according to claim 1, wherein

the means for calculating the total luminance intensity performs to apply an edge operator to each of the pixels in the masked reference image to result each individual edge intensity of the corresponding the pixel and calculates a total edge intensity as a sum of the individual edge intensities of the pixel in the masked reference image, and

the means for determining the presence of the element in the environmental atmosphere around the vehicle performs to determine the presence of the element in the environmental atmosphere around the vehicle in accordance with both the total edge intensity using a fact that the element in the atmosphere scatters a light from the sun which results a smaller value of the total edge intensity than that obtained in the case where the element is absent in the environmental atmosphere around the vehicle.

7. The method according to claim 1, further comprising:

means for detecting an obstacle which is located in ahead of the vehicle and whose picture is included in the reference image to obtain a contour of the obstacle in the reference image,

wherein the means for masking performs to mask not only the area including the picture of the high-luminance obstacle determined by the means for detecting the high-luminance obstacle but also a further area including the picture of the obstacle in the reference image in accordance with the contour of the obstacle detected by the means for detecting the obstacle in the reference image to generate a masked reference image to neglect an effect of existence of both the high-luminance obstacle and the obstacle in determining the presence of the element based on the image captured by the means for capturing the image.

8. The apparatus according to claim 2, further comprising:

means for detecting an obstacle which is located in ahead of the vehicle and whose picture is included in the reference image to obtain a contour of the obstacle in the reference image,

wherein the means for masking performs to mask not only the area including the picture of the high-luminance obstacle determined by the means for detecting the high-luminance obstacle but also a further area including the picture of the obstacle in the reference image in accordance with the contour of the obstacle detected by the means for detecting the obstacle in the reference image to generate a masked reference image to neglect an effect of existence of both the high-luminance obstacle and the obstacle in determining the presence of the element based on the image.

9. The apparatus according to claim 1, further comprising:

means for detecting a value of illumination of the vehicle,

wherein the means for estimating the values of luminance performs to estimate an actual luminance of the space in ahead of the vehicle based on the values of luminance of the arrays of the pixels in the reference image with reference to the value of illumination of the vehicle, and

the means for detecting a high-luminance obstacle performs to detect the high-luminance obstacle which is located in ahead of the vehicle and whose picture is included in the reference image to determine an area including the picture of the high-luminance object in the reference image, the high-luminance object being defined as an obstacle having the actual luminance higher than a predetermined value.

16. The method according to claim 1, further comprising a step of:

detecting an obstacle which is located in ahead of the vehicle and whose picture is included in the reference image to obtain a contour of the obstacle in the reference image,

wherein the step of masking performs to mask not only the area including the picture of the high-luminance obstacle determined by the means for detecting the high-luminance obstacle but also a further area including the picture of the obstacle in the reference image in accordance with the contour of the obstacle detected by the means for detecting the obstacle in the reference image to generate a masked reference image to neglect an effect of existence of both the high-luminance obstacle and the obstacle in determining the presence of the element based on the image captured in the step of capturing the image.

6. The apparatus according to claim S, wherein

the constituent that affects the view of the space which is present ahead of the vehicle is fog.

10. A method for determining the presence of a constituent affecting a view of space which is present ahead of a vehicle, the constituent in the space, comprising steps of:

capturing the image of space, the image being viewed from the vehicle and consisting of arrays of pixels having luminance values;

extracting a reference image from the image captured in the step of capturing the image, the reference image including a portion of the image of the space located a predetermined distance apart from the vehicle;

estimating the values of luminance of the arrays of the pixels in the reference image;

detecting a high-luminance obstacle from the reference image, the high-luminance obstacle being higher in luminance than a predetermined luminance value;

masking the area including the picture of the high-luminance obstacle determined in the step of detecting the high-luminance obstacle in the reference image to generate a masked reference image to neglect an effect of existence of the obstacle in determining the presence of the element based on the image captured in the step of capturing the image of space;

calculating a total luminance intensity as a function of the values of the luminance of the arrays of the pixels in the masked reference image as a total luminance intensity; and

determining of whether or not the constituent is present in the environmental atmosphere around the vehicle in accordance with both the total luminance intensity.

11. The method according to claim 10, further comprising a step of:

extracting a region of sky in the reference image based on the values of luminance of the pixels thereof and estimates a value of luminance of the sky,

wherein the step of detecting the high-luminance obstacle performs to detect the high-luminance obstacle defined as an obstacle having higher luminance than that of sky.

12. The method according to claim 10, wherein

the step of calculating the total luminance intensity performs to transform luminance data including sequential values of luminance of the pixels in each individual array as a sum of base functions with respective coefficients and calculates a total spatial power spectrum of the image based on the coefficients calculated in the transformation of the luminance data, the masked reference image being expressed by sequential luminance values of the unmasked pixels in each individual array such that a length of the luminance data becomes shorter as the number of values of luminance of the unmasked pixels becomes smaller, and

the step of determining the presence of the element in the environmental atmosphere around the vehicle performs to determine the presence of the element in the environmental atmosphere around the vehicle in accordance with both the total spatial power spectrum using a fact that the element in the atmosphere scatters a light from the sun which results a smaller value of the total spatial power spectrum than that obtained in the case where the element is absent in the environmental atmosphere around the vehicle.

13. The apparatus according to claim 12, wherein

the constituent that affects the view of the space which is present ahead of the vehicle is fog.

14. The method according to claim 10, wherein

the step of calculating the total luminance intensity performs to apply an edge operator to each of the pixels in the masked reference image to result each individual edge intensity of the corresponding the pixel and calculates a total edge intensity as a sum of the individual edge intensities of the pixel in the masked reference image, and

the step of determining the presence of the element in the environmental atmosphere around the vehicle performs to determine the presence of the element in the environmental atmosphere around the vehicle in accordance with both the total edge intensity using a fact that the element in the atmosphere scatters a light from the sun which results a smaller value of the total edge intensity than that obtained in the case where the element is absent in the environmental atmosphere around the vehicle.

15. The method according to claim 14, wherein

the constituent that affects the view of the space which is present ahead of the vehicle is fog.

17. The method according to claim 11, further comprising:

detecting an obstacle which is located in ahead of the vehicle and whose picture is included in the reference image to obtain a contour of the obstacle in the reference image,

wherein the step of masking performs to mask not only the area including the picture of the high-luminance obstacle determined by the means for detecting the high-luminance obstacle but also a further area including the picture of the obstacle in the reference image in accordance with the contour of the obstacle detected in the step of detecting the obstacle in the reference image to generate a masked reference image to neglect an effect of existence of both the high-luminance obstacle and the obstacle in determining the presence of the element based on the image.

18. The method according to claim 10, further comprising a step of:

detecting a value of illumination of the vehicle,

wherein the means for estimating the values of luminance performs to estimate an actual luminance of the space in ahead of the vehicle based on the values of luminance of the arrays of the pixels in the reference image with reference to the value of illumination of the vehicle, and

the step of detecting a high-luminance obstacle performs to detect the high-luminance obstacle which is located in ahead of the vehicle and whose picture is included in the reference image to determine an area including the picture of the high-luminance object in the reference image, the high-luminance object being defined as an obstacle having the actual luminance higher than a predetermined value.

19. A program, for use with a computer system, the program being tangibly embodied in a computer readable medium and being provided for detecting the presence of a constituent affecting a view of space which is present ahead of a vehicle, the constituent in the space, the program enabling the computer to functionally realizing:

capturing the image of space, the image being viewed from the vehicle and consisting of arrays of pixels having luminance values;

extracting a reference image from the image captured by the image capturing device, the reference image including a portion of the image of the space located a predetermined distance apart from the vehicle;

estimating the values of luminance of the arrays of the pixels in the reference image;

detecting a high-luminance obstacle from the reference image, the high-luminance obstacle being higher in luminance than a predetermined luminance value;

masking the area including the picture of the high-luminance obstacle determined by the high-luminance obstacle detecting device in the reference image to generate a masked reference image to neglect an effect of existence of the obstacle in determining the presence of the element based on the image captured in the step of capturing the image of the space;

calculating a total luminance intensity as a function of the values of the luminance of the arrays of the pixels in the masked reference image as a total luminance intensity; and

determining of whether or not the constituent is present in the environmental atmosphere around the vehicle in accordance with both the total luminance intensity.

20. The program according to claim 19, wherein

the step of calculating the total luminance intensity performs to transform luminance data including sequential values of luminance of the pixels in each individual array as a sum of base functions with respective coefficients and calculates a total spatial power spectrum of the image based on the coefficients calculated in the transformation of the luminance data, the masked reference image being expressed by sequential values of luminance of the unmasked pixels in each individual array such that a length of the luminance data becomes shorter as the number of values of luminance of the unmasked pixels becomes smaller, and

the step of determining the presence of the element in the environmental atmosphere around the vehicle performs to determine the presence of the element in the environmental atmosphere around the vehicle in accordance with both the total spatial power spectrum calculated in the step of calculating the total luminance intensity using a fact that the element in the atmosphere scatters a light from the sun which results a smaller value of the total spatial power spectrum than that obtained in the case where the element is absent in the environmental atmosphere around the vehicle.