Hydrometeor identification methods and systems

• US 10,026,163 B2
• Filed: 02/25/2016
• Issued: 07/17/2018
• Est. Priority Date: 02/25/2015
• Status: Active Grant

• Current Assignee: Cale Fallgatter, University of Utah Research Foundation (State of Utah)
• Original Assignee: Cale Fallgatter, University of Utah Research Foundation (State of Utah)
• Inventors: Fallgatter, Cale, Garrett, Tim, Shkurko, Konstantin
• Primary Examiner(s): Rao, Anand S
• Assistant Examiner(s): Edwards, Tyler B
• Application Number:
  US15/053,951
• Publication Number:
  US 20160247273A1
• Time in Patent Office: 873 Days
• Field of Search: None
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules

Abstract

A technology is described for identifying hydrometeors. A method includes receiving an image of a hydrometeor captured using a camera. The hydrometeor in the image can be identified and analyzed to determine characteristics associated with the hydrometeor. Environmental measurements recorded substantially contemporaneously with the image can be obtained from environmental sensors located in proximity to the camera. A feature vector can be constructed using the hydrometeor characteristics and the environmental measurements. The feature vector can be input to a classification model used to classify the hydrometeor, and the classification model can output a classification for the hydrometeor using the feature vector.

Claims

1. A computer implemented method, comprising:

receiving an image of a hydrometeor captured using a camera included in a hydrometeor classification device;

identifying the hydrometeor in the image, using a hydrometeor classification device processor;

analyzing the hydrometeor represented in the image to determine hydrometeor characteristics associated with the hydrometeor, using the hydrometeor classification device processor;

obtaining environmental measurements from environmental sensors located in proximity to the camera in the hydrometeor classification device and recorded substantially contemporaneously with the image, using the hydrometeor classification device processor;

constructing a feature vector using the hydrometeor characteristics and the environmental measurements, using the hydrometeor classification device processor; and

inputting the feature vector to a classification model used to classify the hydrometeor, using the hydrometeor classification device processor, wherein the classification model outputs a classification for the hydrometeor using the feature vector.

2. A method as in claim 1, wherein identifying the hydrometeor in the image further comprises,

determining a threshold for the image using a thresholding technique;

detecting contours of the hydrometeor in the image using an edge detection technique that uses the threshold for the image; and

identifying the contours of the hydrometeor that are in focus in the image.

3. A method as in claim 2, further comprising determining regions of the image having inter-pixel brightness variability that is greater than the threshold for the image to identify the contours of the hydrometeor that are in focus.

4. A method as in claim 1, wherein characteristics associated with the hydrometeor includes one or more of:

hydrometeor shape, hydrometeor size, hydrometeor fall speed, hydrometeor brightness, and hydrometeor internal variability.

5. A method as in claim 1, wherein the environmental measurements obtained from the environmental sensors includes at least one of:

environmental temperature, environmental relative humidity, environmental dew point temperature, wind direction, atmospheric pressure, or precipitation accumulation.

6. A method as in claim 1, wherein the feature vector includes at least one of:

mean intensity of the hydrometeor, variance of the hydrometeor, inter-pixel brightness variability, perimeter of the hydrometeor, slope of an azimuthally averaged power spectrum of the hydrometeor, fall speed of the hydrometeor, number of separate regions in the image that are in focus, environmental relative humidity, or environmental temperature.

7. A method as in claim 1, wherein the classification model is a machine learning model or fuzzy logic model.

8. A method as in claim 1, wherein classifications output by the classification model includes at least one of:

drizzle, rain, freezing rain, freezing drizzle, freezing fog, freezing rain, ice crystals, hail graupel, snow, snow grains, ice pellets, snow pellets, blowing snow, or mixed precipitation.

9. A method as in claim 1, further comprising determining visibility using a contrasting black and white image captured with near-infrared light using the camera.

10. A method as in claim 9, wherein determining visibility further comprises analyzing the contrasting black and white image to determine a presence of fog or haze.

11. A method as in claim 1, wherein receiving the image of the hydrometeor captured using a camera further includes determining a presence of the hydrometeor using motion sensing to determine presence of the hydrometeor located at a predetermined location in order to trigger the camera.

12. A method as in claim 1, wherein the hydrometeor includes at least one of:

a snowflake, raindrop, hail, or dust.

13. An apparatus comprising:

a motion sensor controller configured to control a motion sensor and to generate a camera activation signal when the motion sensor senses motion;

a camera activation module in communication with the motion sensor controller, wherein the camera activation module is configured to receive the camera activation signal and activate a camera to capture an image of a hydrometeor;

an environmental measurement module configured to obtain environmental measurements from a plurality of environmental sensors that are located in proximity to the camera and are recorded substantially contemporaneous with the image of the hydrometeor using the camera; and

a hydrometeor classification module configured with instructions that, when executed by a processor, output a classification for the hydrometeor using a classification model, wherein a feature vector constructed using hydrometeor characteristics obtained from the image of the hydrometeor and the environmental measurements obtained from the environmental measurement module is inputted to the classification model.

14. An apparatus as in claim 13, further comprising a light source activation module in communication with the motion sensor controller and the camera activation module, wherein the light source activation module is configured to receive the camera activation signal and in response activate a flashing light source to provide a flash of light in synchronization with image capture by the camera to illuminate the hydrometeor when images of the hydrometeor are captured.

15. An apparatus as in claim 14, wherein the flashing light source that provides the flash of light further comprises, a pair of infrared light emitters and a pair of corresponding infrared light detectors that detect hydrometeors within a field of view of the camera.

16. An apparatus as in claim 14, wherein the flashing light source that provides the flash of light further comprises, a pair of infrared laser diode emitters and a pair of corresponding infrared laser detectors that detect hydrometeors within a field of view of the camera.

17. An apparatus as in claim 14, wherein the light source activation module is further configured to receive the camera activation signal as a result of the hydrometeor passing through a detection region comprising an upper trigger area and a lower trigger area formed using motion sensor arrays, wherein triggering the upper trigger area starts a timer and triggering the lower trigger area stops the timer and causes the camera activation signal to be sent.

18. A system comprising:

a processor;

a memory device including instructions that, when executed by the processor, cause the system to;

receive an image of a hydrometeor within an area defined by motion sensors that is captured using a camera;

identify the hydrometeor in the image;

analyze the hydrometeor represented in the image to determine hydrometeor characteristics associated with the hydrometeor;

obtain environmental measurements from environmental sensors located in proximity to the camera and recorded substantially contemporaneously with the image;

construct a feature vector using the hydrometeor characteristics and the environmental measurements; and

input the feature vector to a classification model used to classify the hydrometeor, wherein the classification model outputs a classification for the hydrometeor using the feature vector.

19. A system as in claim 18, wherein the memory device includes instructions that, when executed by the processor, causes the system to calculate a size of the hydrometeor based in part on a distance of the camera to the area defined by the motion sensors.

20. A system as in claim 18, wherein the classification model is a machine learning model or fuzzy logic model.