Information extraction using image data

• US 10,489,674 B2
• Filed: 09/08/2017
• Issued: 11/26/2019
• Est. Priority Date: 12/01/2015
• Status: Active Grant

• Current Assignee: Tianjin Kantian Technology Co., Ltd.
• Original Assignee: Weather Intelligence Technology, Inc.
• Inventors: Shen, Xinning, Chu, Wei
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US15/699,898
• Publication Number:
  US 20170372167A1
• Time in Patent Office: 809 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30192 Weather; Meteorology
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

Techniques for weather information extraction using image data are described herein. In accordance with the present disclosure, economical weather data collection devices may be installed in any places on the earth, and form a dense network of weather data collection devices. Preferably, these weather data collection devices include image capture components that may capture series of sequential images over time (i.e., image streams) from any particular locations. The disclosed techniques include extracting a sky portion from a series of time-sequenced images. The extracted sky portion may be used for determining cloud coverage, a sun track, and so on, thereby enabling to provide accurate weather forecast as well as other weather data mining abilities.

Claims

1. A method of determining a cloudiness value, comprising:

identifying, based on color values, a first type of data elements among a plurality of data elements indicative of a sky portion of an image;

detecting, among the first type of data elements, a set of neighbouring data elements indicative of a sun portion of the image;

removing the set of neighbouring data elements indicative of the sun portion from the first type of data elements among the plurality of data elements indicative of the sky portion; and

determining a cloudiness value based at least in part on a proportion of a number of remaining first type of data elements after the removing to a number of the plurality of data elements indicative of the sky portion.

2. The method of claim 1, before the identifying a first type of data elements among a plurality of data elements indicative of a sky portion of an image, further comprising:

using a mask image to obtain the sky portion of the image, the sky portion of the image comprising the plurality of data elements indicative of the sky portion.

3. The method of claim 1, wherein the color values of the first type of data elements satisfy criteria of |R−

B| less than 30 and |B−

G| less than 30.

4. The method of claim 1, wherein the set of neighbouring data elements indicative of a sun portion have hue values less than 10.

5. The method of claim 1, wherein the detecting, among the first type of data elements, a set of neighbouring data elements indicative of a sun portion further comprises:

identifying a largest area among a plurality of areas defined by a plurality of corresponding sets of neighbouring data elements among the first type of data elements, the plurality of corresponding sets of neighbouring data elements having hue values less than 10;

determining that an area ratio of the largest area to a minimum geometric shape enclosing the largest area is greater than a threshold;

determining that a radius of the minimum geometric shape is within a predetermined range; and

identifying a set of neighbouring data elements defining the largest area as the set of neighbouring data elements indicative of the sun portion.

6. The method of claim 1, further comprising

verifying a detected set of neighbouring data elements is indicative of the sun portion of the image in response to a determination that the detected set of neighbouring data elements and a plurality of data elements indicative of a sun track share at least a data element based on position information.

7. The method of claim 6, to obtain the plurality of data elements indicative of the sun track, further comprising:

identifying, based on image capture times, at least a pair of corresponding images among at least two series of images, the at least two series of images each comprising a plurality of time-sequenced images;

detecting that the at least a pair of corresponding images comprise respective sets of neighbouring data elements indicative of respective sun portions;

determining that the respective sets of neighbouring data elements indicative of respective sun portions share at least a subset of data elements based on position information;

determining a centroid of an area defined by the at least a subset of data elements; and

identifying a data element corresponding to the centroid as one of the plurality of data elements indicative of the sun track.

8. The method of claim 7, further comprising:

generating a curve denoting the sun truck based on the plurality of data elements indicative of the sun track.

9. The method of claim 7, wherein the at least two series of images comprises three series of images that are captured in three successive days, respectively.

10. The method of claim 7, wherein the identifying, based on image capture times, at least a pair of corresponding images among the at least two series of images further comprises:

selecting a first image captured at time t from a first series of images among the at least two series of images; and

searching a second series of images among the at least two series of images to identify a corresponding image captured within a time period of t+/−

2 minutes.

11. A system of determining a cloudiness value, comprising:

at least a processor; and

at least a memory communicatively coupled to the at least a processor to configure the at least a processor to;

identify, based on color values, a first type of data elements among a plurality of data elements indicative of a sky portion of an image,

detect, among the first type of data elements, a set of neighbouring data elements indicative of a sun portion of the image,

remove the set of neighbouring data elements indicative of the sun portion from the first type of data elements among the plurality of data elements indicative of the sky portion, and

determine a cloudiness value based at least in part on a proportion of a number of remaining first type of data elements after the removing to a number of the plurality of data elements indicative of the sky portion.

12. The system of claim 11, the at least a memory further configuring the at least a processor to:

use a mask image to obtain the sky portion of the image, the sky portion of the image comprising the plurality of data elements indicative of the sky portion.

13. The system of claim 11, wherein the color values of the first type of data elements satisfy criteria of |R−

B| less than 30 and |B−

G| less than 30.

14. The system of claim 11, the at least a memory further configuring the at least a processor to:

identify a largest area among a plurality of areas defined by a plurality of corresponding sets of neighbouring data elements among the first type of data elements, the plurality of corresponding sets of neighbouring data elements having hue values less than 10;

determine that an area ratio of the largest area to a minimum geometric shape enclosing the largest area is greater than a threshold;

determine that a radius of the minimum geometric shape is within a predetermined range; and

detect a set of neighbouring data elements defining the largest area as the set of neighbouring data elements indicative of the sun portion.

15. The system of claim 11, the at least a memory further configuring the at least a processor to:

verify a detected set of neighbouring data elements is indicative of the sun portion of the image in response to a determination that the detected set of neighbouring data elements and a plurality of data elements indicative of the sun track share at least a data element based on position information.

16. The system of claim 15, the at least a memory further configuring the at least a processor to:

identify, based on image capture times, at least a pair of corresponding images among at least two series of images, the at least two series of images each comprising a plurality of time-sequenced images;

detect that the at least a pair of corresponding images comprise respective sets of neighbouring data elements indicative of respective sun portions;

determine that the respective sets of neighbouring data elements indicative of respective sun portions share at least a subset of data elements based on position information;

determine a centroid of an area defined by the at least a subset of data elements; and

identify a data element corresponding to the centroid as one of the plurality of data elements indicative of the sun track.

17. A non-transitory computer-readable storage medium bearing computer-readable instructions that upon execution on a computing device cause the computing device at least to:

identify, based on color values, a first type of data elements among a plurality of data elements indicative of a sky portion of an image,

detect, among the first type of data elements, a set of neighbouring data elements indicative of a sun portion of the image,

remove the set of neighbouring data elements indicative of the sun portion from the first type of data elements among the plurality of data elements indicative of the sky portion, and

determine a cloudiness value based at least in part on a proportion of a number of remaining first type of data elements after the removing to a number of the plurality of data elements indicative of the sky portion.

18. The non-transitory computer-readable storage medium of claim 17, further comprising computer-readable instructions that upon execution on the computing device cause the computing device at least to:

identify a largest area among a plurality of areas defined by a plurality of corresponding sets of neighbouring data elements among the first type of data elements, the plurality of corresponding sets of neighbouring data elements having hue values less than 10;

determine that an area ratio of the largest area to a minimum geometric shape enclosing the largest area is greater than a threshold;

determine that a radius of the minimum geometric shape is within a predetermined range; and

detect a set of neighbouring data elements defining the largest area as the set of neighbouring data elements indicative of the sun portion.

19. The non-transitory computer-readable storage medium of claim 17, further comprising computer-readable instructions that upon execution on the computing device cause the computing device at least to:

verify a detected set of neighbouring data elements is indicative of the sun portion of the image in response to a determination that the detected set of neighbouring data elements and a plurality of data elements indicative of the sun track share at least a data element based on position information.

20. The non-transitory computer-readable storage medium of claim 19, further comprising computer-readable instructions that upon execution on the computing device cause the computing device at least to:

identify, based on image capture times, at least a pair of corresponding images among at least two series of images, the at least two series of images each comprising a plurality of time-sequenced images;

detect that the at least a pair of corresponding images comprise respective sets of neighbouring data elements indicative of respective sun portions;

determine that the respective sets of neighbouring data elements indicative of respective sun portions share at least a subset of data elements based on position information;

determine a centroid of an area defined by the at least a subset of data elements; and

identify a data element corresponding to the centroid as one of the plurality of data elements indicative of the sun track.