Estimating rainfall precipitation amounts by applying computer vision in cameras

• US 9,582,730 B2
• Filed: 11/24/2014
• Issued: 02/28/2017
• Est. Priority Date: 11/24/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cerqueira, Renato F., Mantripragada, Kiran
• Primary Examiner(s): MOYER, ANDREW M
• Application Number:
  US14/551,238
• Publication Number:
  US 20160148382A1
• Time in Patent Office: 827 Days
• Field of Search: 382/217, 382/218, 382/221
• US Class Current: 1/1
• CPC Class Codes: G01W 1/14 Rainfall or precipitation gauges measuring volume in general G01F
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 20/38 Outdoor scenes
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

A method and system are provided. The method includes storing a set of references images without rain and spanning a plurality of different light conditions. The method further includes capturing, using a camera, an image of a scene with rain. The method also includes selecting a reference image from the set of reference images based on the light condition of the captured image. The method additionally includes performing an arithmetic subtraction image processing operation between the captured image and the reference image to generate a subtraction image. The method further includes estimating an amount of rain in the subtraction image based on previously calibrated values.

Claims

1. A system, comprising:

memory for storing a set of reference images without rain and spanning a plurality of different light conditions;

a camera for capturing an image of a scene with rain; and

a processor for selecting a reference image from the set of reference images based on the light condition of the captured image, performing an arithmetic subtraction image processing operation between the captured image and the reference image to generate a subtraction image, cutting off corner pixels in the subtraction image, and estimating an amount of rain in the subtraction image based on previously calibrated values, wherein the previously calibrated values include actual measurements of rainfall.

2. The system of claim 1, wherein the processor performs image border detection processing on the subtraction image to detect the rain and generates a binary mask responsive to a result of the subtraction image processing operation, the binary mask having a first pixel value indicative of an absence of rain and having a second pixel value indicative of a presence of rain, and wherein the processor performs a direct count of pixels having rain by counting a total number of occurrences of the second pixel value, and generates an estimate of the amount of rain responsive to the direct count of pixels.

3. The system of claim 1, wherein the set of reference images is captured at least one day prior to capturing the image of the scene with rain.

4. The system of claim 1, wherein the processor obtains the set of reference images from a remote source with respect to the camera.

5. The system of claim 1, wherein the reference image is selected from the set of reference images based on the light condition of the captured image using a time of day as a selection basis.

6. The system of claim 1, wherein the reference image is selected from the set of reference images based on the light condition of the captured image using a day of the year as a selection basis.

7. The system of claim 1, wherein the arithmetic subtraction image processing operation comprises performing:

C(x,y)−

T(x,y)=O(x,y),

where C denotes the captured image, T denotes the reference image, O denotes the subtraction image without a background and with at least one moving object, x denotes a first spatial orientation, and y denotes a second spatial orientation orthogonal with respect to the first spatial orientation.

8. The system of claim 1, wherein a predetermined granularity and precision are set for at least one rain estimate.

9. The system of claim 1, wherein estimating the amount of rain includes a numerical value of rainfall displayed on a display device.

10. The system of claim 1, wherein the processor is further configured to recalibrate the previously calibrated values by comparing the previously calibrated values and measurements obtained by external instruments.

11. A computer program product for estimating rainfall precipitation, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:

storing a set of reference images without rain and spanning a plurality of different light conditions;

capturing, using a camera, an image of a scene with rain;

selecting a reference image from the set of reference images based on the light condition of the captured image;

performing an arithmetic subtraction image processing operation between the captured image and the reference image to generate a subtraction image;

cutting off corner pixels in the subtraction image; and

estimating an amount of rain in the subtraction image based on previously calibrated values, wherein the previously calibrated values include actual measurements of rainfall.

12. The computer program product of claim 11, wherein the set of reference images is captured at least one day prior to capturing the image of the scene with rain.

13. The computer program product of claim 11, further comprising obtaining the set of reference images from a remote source with respect to the camera.

14. The computer program product of claim 11, wherein the reference image is selected from the set of reference images based on the light condition of the captured image using a time of day as a selection basis.

15. The computer program product of claim 11, wherein the reference image is selected from the set of reference images based on the light condition of the captured image using a day of the year as a selection basis.

16. The computer program product of claim 11, wherein the arithmetic subtraction image processing operation comprises performing:

C(x,y)−

T(x,y)=O(x,y),

where C denotes the captured image, T denotes the reference image, O denotes the subtraction image without a background and with at least one moving object, x denotes a first spatial orientation, and y denotes a second spatial orientation orthogonal with respect to the first spatial orientation.

17. The computer program product of claim 11, wherein said step of estimating the amount of rain comprises setting a predetermined granularity and precision for at least one rain estimate.

18. The computer program product of claim 11, wherein estimating the amount of rain includes a numerical value of rainfall displayed on a display device.

19. The computer program product of claim 11, wherein the set of reference images spanning the plurality of different light conditions includes images obtained from a surface region upon which rain is detected.

20. The computer program product of claim 11, further comprising recalibrating the previously calibrated values by comparing the previously calibrated values and measurements obtained by external instruments.