Outside recognition apparatus and lens dirtiness detection method

• US 9,813,593 B2
• Filed: 06/05/2014
• Issued: 11/07/2017
• Est. Priority Date: 07/26/2013
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA)
• Original Assignee: Clarion Co., Ltd. (Faurecia SA)
• Inventors: Akiyama, Yasuhiro, Nakamura, Katsuyuki, Irie, Kota
• Primary Examiner(s): Misleh, Justin P
• Application Number:
  US14/906,969
• Publication Number:
  US 20160165101A1
• Time in Patent Office: 1,251 Days
• Field of Search: None
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: G06T 2207/20048 Transform domain processing
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/42 using transform domain methods
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

It is possible to consider an image as a composite wave that is the result of layering waves having differing periods and amplitudes. An image captured via a lens having a water droplet attached thereto has a higher occurrence of changes such as image blur than an image that does not have a water droplet attached thereto, which means that this type of change in an image results in changes in the composite wave thereof as well. Provided is a lens dirtiness detection device that suitably determines whether a lens is dirty without being affected by a background image by: focusing on the occurrence of large changes in the composite wave of an image that accompany an increase in the dirtiness of a camera lens; extracting each of the image frequency components that constitute the composite wave; and analyzing changes in the magnitude relation between the frequency components.

Claims

1. An outside recognition apparatus configured to detect a raindrop on a lens of a camera, comprising:

a control unit; and

a memory coupled to the control unit,

wherein the memory stores therein pixel value data of an image captured by the camera, and

wherein the control unit is configured to;

calculate image frequency powers of a plurality of frequency regions based on pixel values of a first region of a plurality of first regions of the image;

determine whether each of the calculated image frequency powers of the plurality of frequency regions of the first region exceeds a predetermined first threshold;

determine that the raindrop is in the first region if a number of the plurality of frequency regions in which the calculated image frequency powers does not exceed the first threshold value exceeds a predetermined second threshold;

calculate image frequency powers based on pixel values of a plurality of second regions of the image each made of the plurality of the first regions, each of the plurality of second regions being smaller than the entire image;

calculate image frequency powers based on pixel values of the plurality of first regions;

determine presence or absence of the raindrop based on the calculated image frequency powers of the plurality of second regions and the calculated image frequency powers of the plurality of first regions; and

output the determination that the raindrop is present or absent.

2. The outside recognition apparatus according to claim 1,

wherein the control unit is further configured to:

calculate the image frequency powers of the plurality of second regions before calculating the image frequency powers of the plurality of first regions;

determine presence or absence of the raindrop based on the calculated image frequency powers of the plurality of first regions included in a number of the plurality of second regions determined to have the raindrop.

3. The outside recognition apparatus according to claim 2,

wherein the calculated image frequency powers of the plurality of second regions are obtained through weighed calculation on an image frequency power of each of the plurality of second regions.

4. The outside recognition apparatus according to claim 3,

wherein the control unit is further configured to:

compare an image of at least one stationary subject in which a subject always appears with a template image captured in advance to detect image distortion of the at least one stationary subject; and

determine presence or absence of the raindrop based the detected image distortion of the at least one stationary subject.

5. The outside recognition apparatus according to claim 4,

wherein the control unit is further configured to:

determine whether a normalized sum of absolute difference between the image of the at least one of the stationary subject and the template image exceeds a predetermined third threshold; and

determine presence or absence of the raindrop based on a number of the plurality of first regions determined to have the raindrop, and that the normalized sum of absolute differences does not exceed the third threshold.

6. The outside recognition apparatus according to claim 1,

wherein the control unit is further configured to:

determine whether a number of the plurality of first regions determined to have the raindrop exceeds a predetermined fourth threshold; and

determine presence of the raindrop if the number of the plurality of first regions exceeds the predetermined fourth threshold.

7. The outside recognition apparatus according to claim 1,

wherein the control unit is further configured to:

determine whether a number of the plurality of first regions determined to have the raindrop in each of a plurality of third regions, each third region being smaller than the entire image and greater than the first region, exceeds a predetermined fifth threshold; and

determine that the raindrop is a third region of the plurality of third regions if the number of the plurality of first regions determined to have the raindrop exceeds the fifth threshold.

8. The outside recognition apparatus according to claim 1, further comprising:

the camera; and

at least one infrared light source configured to radiate infrared light to the lens of the camera in the case where an image capturing environment of the camera meets predetermined conditions.

9. The outside recognition apparatus according to claim 1, further comprising at least one of a display device and an audio output device, the display device being configured to output visual information indicating the determination that the raindrop is present or absent, the audio output device being configured to output audio information indicating the determination that the raindrop is present or absent.

10. A method for detecting a raindrop on a lens of a camera using an outside recognition apparatus that includes a control unit, and a memory coupled to the control unit, the memory storing therein pixel value data of an image captured by the camera, the method comprising:

calculating image frequency powers of a plurality of frequency regions based on pixel values of a first region of a plurality of first regions of the image;

determining whether each of the calculated image frequency powers of the plurality of frequency regions of the first region exceeds a predetermined first threshold;

determining that the raindrop is in the first region if a number of the plurality of frequency regions in which the calculated image frequency powers does not exceed the first threshold value exceeds a predetermined second threshold;

calculating image frequency powers based on pixel values of a plurality of second regions of the image each made of the plurality of the first regions, each of the plurality of second regions being smaller than the entire image;

calculating image frequency powers based on pixel values of the plurality of first regions;

determining presence or absence of the raindrop based on the calculated image frequency powers of the plurality of second regions and the calculated image frequency powers of the plurality of first regions; and

determining that the raindrop is present or absent.