Estimating rainfall precipitation amounts by applying computer vision in cameras

• US 9,436,997 B2
• Filed: 06/23/2015
• Issued: 09/06/2016
• Est. Priority Date: 11/24/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cerqueira, Renato F., Mantripragada, Kiran
• Primary Examiner(s): Moyer, Andrew
• Application Number:
  US14/748,125
• Publication Number:
  US 20160148383A1
• Time in Patent Office: 441 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01W 1/14 Rainfall or precipitation gauges measuring volume in general G01F
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 20/38 Outdoor scenes
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

A method and system are provided. The method includes storing a set of references images without rain and spanning a plurality of different light conditions. The method further includes capturing, using a camera, an image of a scene with rain. The method also includes selecting a reference image from the set of reference images based on the light condition of the captured image. The method additionally includes performing an arithmetic subtraction image processing operation between the captured image and the reference image to generate a subtraction image. The method further includes estimating an amount of rain in the subtraction image based on previously calibrated values.

Claims

1. A method, comprising:

storing a set of references images without rain and spanning a plurality of different light conditions;

capturing, using a camera, an image of a scene with rain;

selecting a reference image from the set of reference images based on the light condition of the captured image;

performing an arithmetic subtraction image processing operation between the captured image and the reference image to generate a subtraction image;

cutting off corner pixels in the subtraction image; and

estimating an amount of rain in the subtraction image based on previously calibrated values.

2. The method of claim 1, further comprising re-purposing the camera from another use.

3. The method of claim 1, further comprising:

performing image border detection processing on the subtraction image to detect the rain;

generating a binary mask responsive to a result of the border detection image processing, the binary mask having a first pixel value indicative of an absence of rain and having a second pixel value indicative of a presence of rain; and

performing a count of pixels having rain based on a total number of occurrences of at least one of the first pixel value and the second pixel value.

4. The method of claim 3, wherein said estimating step generates an estimate of the amount of rain responsive to the count of pixels.

5. The method of claim 3, wherein the image border detection processing comprises border detection, line detection, and at least one of Fourier Transformation and wavelet transformation.

6. The method of claim 3, wherein the image border detection processing comprises applying at least one of a Fourier transform and a wavelet transform to the subtraction image to at least one of identify and enhance frequencies which describe textures of raindrops.

7. The method of claim 3, wherein the image border detection processing comprises performing line detection to detect lines of one or more particular orientations.

8. The method of claim 7, wherein the line detection comprises applying at least one Hough transform to the subtraction image.

9. The method of claim 1, wherein the reference image is selected from the set based upon a time correspondence between the reference image and the captured image.

10. The method of claim 1, wherein the arithmetic subtraction image processing operation comprises performing:

C(x,y)−

T(x,y)=O(x,y),

where C denotes the captured image, T denotes the reference image, O denotes the subtraction image without a background and with at least one moving object, x denotes a first spatial orientation, and y denotes a second spatial orientation orthogonal with respect to the first spatial orientation.

11. The method of claim 1, further comprising performing a threshold filtering operation to cut off pixels in the subtraction image having an intensity below a threshold intensity.

12. The method of claim 1, wherein said step of estimating the amount of rain comprises setting a predetermined granularity and precision for at least one rain estimate.

13. The method of claim 1, further comprising generating a user-perceptible indication of the amount of rain.

14. The method of claim 1, further comprising training a set of calibrated values that include the previously calibrated values to improve a precision of a rain amount estimate generated by said estimating step.

15. The method of claim 1, wherein the captured image includes a plate glass from which the amount of rain is estimated.