Lens Dirtiness Detection Apparatus and Lens Dirtiness Detection Method

• US 20160165101A1
• Filed: 06/05/2014
• Published: 06/09/2016
• Est. Priority Date: 07/26/2013
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA)
• Original Assignee: Clarion Co., Ltd. (Faurecia SA)
• Inventors: Akiyama, Yasuhiro, Nakamura, Katsuyuki, Irie, Kota
• Granted Patent: US 9,813,593 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: G06T 2207/20048 Transform domain processing
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/42 using transform domain methods
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

It is possible to consider an image as a composite wave that is the result of layering waves having differing periods and amplitudes. An image captured via a lens having a water droplet attached thereto has a higher occurrence of changes such as image blur than an image that does not have a water droplet attached thereto, which means that this type of change in an image results in changes in the composite wave thereof as well. Provided is a lens dirtiness detection device that suitably determines whether a lens is dirty without being affected by a background image by: focusing on the occurrence of large changes in the composite wave of an image that accompany an increase in the dirtiness of a camera lens; extracting each of the image frequency components that constitute the composite wave; and analyzing changes in the magnitude relation between the frequency components.

Claims

1. A lens dirtiness detection apparatus configured to detect a lens dirtiness of a camera, comprising:

a control unit; and

a memory coupled to the control unit,

wherein the memory stores therein pixel value data of an image captured by the camera, and

wherein the control unit is configured to;

calculate image frequency power based on pixel values of a first region of the image;

determine presence or absence of a lens dirtiness of the camera based on the image frequency power of the first region; and

output the determination result on presence or absence of a lens dirtiness.

2. The lens dirtiness detection apparatus according to claim 1,

wherein the control unit is further configured to:

calculate image frequency powers of a plurality of frequency regions based on pixel values of the first region of the image; and

determine presence or absence of the lens dirtiness in the first region based on the image frequency powers of the plurality of frequency areas.

3. The lens dirtiness detection apparatus according to claim 2,

wherein the control unit is further configured to:

determine whether each of the image frequency powers of the plurality of frequency regions of the first region exceeds a predetermined first threshold; and

determine that the lens dirtiness is present in the first region if the number of the frequency regions in which the image frequency power does not exceed the first threshold value exceeds a predetermined second threshold.

4. The lens dirtiness detection apparatus according to claim 3,

wherein the control unit is further configured to:

calculate image frequency powers of a plurality of second regions each made of a plurality of the first regions, each of the second regions being smaller than the entire image;

calculate image frequency powers of the plurality of first regions based on the pixel value data; and

determine presence or absence of the lens dirtiness based on the image frequency powers of the respective second regions and the image frequency powers of the respective first regions.

5. The lens dirtiness detection apparatus according to claim 4,

wherein the control unit is further configured to:

calculate image frequency powers of the plurality of second regions before calculating image frequency powers of the plurality of first regions;

determine presence or absence of the lens dirtiness based on the image frequency powers of the respective second regions;

determine presence or absence of the lens dirtiness based on the image frequency powers of the respective first regions included in the respective second regions determined to have the lens dirtiness; and

output the determination result on presence or absence of the lens dirtiness based on the image frequency powers of the respective first regions.

6. The lens dirtiness detection apparatus according to claim 5,

wherein the control unit is further configured to determine presence or absence of a lens dirtiness in the respective second regions based on image frequency powers obtained through weighed calculation on the image frequency power of each of the second regions and image frequency powers of a plurality of the second regions adjacent to said second region.

7. The lens dirtiness detection apparatus according to claim 6,

wherein the control unit is further configured to:

compare an image of at least one stationary subject section in which the same subject always appear in the image based on the pixel value data with a template image captured in advance, thereby detecting image distortion of the at least one stationary subject section; and

determine presence or absence of the lens dirtiness based on the image frequency powers of the respective first regions and the detected image distortion of the at least one stationary subject section.

8. The lens dirtiness detection apparatus according to claim 7,

wherein the control unit is further configured to:

determine whether a normalized sum of absolute difference between a plurality of images of the stationary subject sections and the template images for the plurality of stationary subject sections exceeds a predetermined third threshold or not; and

determine presence or absence of the lent dirtiness in the image based on the number of the first regions determined to have the lens dirtiness, and the number of the stationary subject sections in which the normalized sum of absolute differences did not exceed the third threshold.

9. The lens dirtiness detection apparatus according to claim 3,

wherein the control unit is further configured to output the determination result on presence or absence of lens dirtiness for each of the first regions.

10. The lens dirtiness detection apparatus according to claim 3,

wherein the control unit is further configured to:

determine whether the number of the first regions determined to have the lens dirtiness in the image exceeds a predetermined fourth threshold or not;

determine presence of the lens dirtiness in the image if the number of the first regions determined to have the lens dirtiness exceeds the fourth threshold; and

output the determination result on presence or absence of the lens dirtiness in the image.

11. The lens dirtiness detection apparatus according to claim 3,

wherein the control unit is further configured to:

determine whether or not the number of the first regions determined to have the lens dirtiness in each third region that is smaller than the entire image and that is greater than the first region exceeds a predetermined fifth threshold;

determine presence of the lens dirtiness in the third region if the number of the first regions determined to have the lens dirtiness exceeds the fifth threshold; and

output the determination result on presence or absence of the lens dirtiness for each third region.

12. The lens dirtiness detection apparatus according to claim 1, further comprising:

the camera; and

at least one infrared light source configured to radiate infrared light to a lens surface of the camera in the case where an image capturing environment of the camera meets predetermined conditions.

13. The lens dirtiness detection apparatus according to claim 1, further comprising at least one of a display device and an audio output device, the display device being configured to output visual information indicating a determination result on presence or absence of the lens dirtiness, the audio output device being configured to output audio information indicating a determination result on presence or absence of the lens dirtiness.

14. A lens dirtiness detection method conducted by a lens dirtiness detection apparatus to detect a lens dirtiness on a camera, the lens dirtiness detection apparatus including a control unit and a memory coupled to the control unit, the memory therein having pixel value data of an image captured by the camera, the detection method comprising:

a calculation process in which the control unit calculates image frequency powers based on pixel values of a first region of the image,

a determination process in which the control unit determines presence or absence of a lens dirtiness of the camera, based on the image frequency power of the first region, and

an output process in which the control unit outputs the determination result on presence or absence of the lens dirtiness.