Adaptive image acquisition and processing with image analysis feedback

• US 8,379,913 B1
• Filed: 09/13/2012
• Issued: 02/19/2013
• Est. Priority Date: 08/26/2011
• Status: Active Grant

• Current Assignee: Planet Labs PBC
• Original Assignee: Skybox Imaging, Inc. (Planet Labs PBC)
• Inventors: Robinson, M. Dirk, Robertson, Mark, Isaac, Hadar, Berkenstock, Daniel, Guinan, Oliver, Melendez, Thomas Joseph, Mann, Julian
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US13/614,050
• Publication Number:
  US 20130051621A1
• Time in Patent Office: 159 Days
• Field of Search: 382/100, 382/325
• US Class Current: 382/100
• CPC Class Codes: G06F 18/41 Interactive pattern learning with a human teacher
• CPC Class Codes: G06V 10/7788 the supervisor being a human, e.g. interactive learning with a human teacher
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

Described are systems, methods, computer programs, and user interfaces for image location, acquisition, analysis, and data correlation that uses human-in-the-loop processing, Human Intelligence Tasks (HIT), and/or or automated image processing. Results obtained using image analysis are correlated to non-spatial information useful for commerce and trade. For example, images of regions of interest of the earth are used to count items (e.g., cars in a store parking lot to predict store revenues), detect events (e.g., unloading of a container ship, or evaluating the completion of a construction project), or quantify items (e.g., the water level in a reservoir, the area of a farming plot).

Claims

1. A method of correlating features in satellite imagery with economic data, the method comprising:

under control of an automated imaging system comprising computer hardware;

receiving input control parameters comprising;

a textual description of an area of interest, and

a time range;

analyzing the received input control parameters to obtain location information of the area of interest;

retrieving a geographical location corresponding to the obtained location information;

determining one or more polygons of interest corresponding to the geographical location, wherein a polygon of interest comprises coordinates delineating a boundary containing at least a portion of the area of interest;

obtaining, from at least one of an image archive or an image collection subsystem, satellite imagery of the area of interest, the satellite imagery comprising a plurality of images, such that;

a surface area of each of the one or more polygons of interest is covered by the satellite imagery, and

an acquisition time of the satellite imagery is within the time range;

automatically processing the satellite imagery, wherein automatically processing the satellite imagery comprises at least one of;

combining multiple images from the satellite imagery to form a single image,

selecting a preferred image from the satellite imagery, or

modifying the satellite imagery to visually enhance portions of the satellite imagery;

obtaining image analysis information related to the processed satellite imagery, wherein the image analysis information comprises at least one of;

a quantity of an object of interest in the processed satellite imagery,

a boundary of an object of interest in the processed satellite imagery,

a location of an object of interest in the processed satellite imagery,

a size of an object of interest in the processed satellite imagery, or

an indication of an occurrence of an event in the processed satellite imagery; and

automatically determining a correlation between the image analysis information and the economic data.

2. The method of claim 1, the received input control parameters further comprising at least one of:

a textual description of an object of interest to be analyzed in the area of interest,

a textual description of the image analysis information to be obtained,

a frequency with which to obtain the image analysis information,

a frequency with which to obtain the satellite imagery, or

a confidence score of the image analysis information.

3. The method of claim 1, wherein the location information comprises an address of the area of interest and the geographical location comprises latitude and longitude coordinates.

4. The method of claim 1, wherein determining a polygon of interest comprises:

creating a polygon template having a border;

placing the polygon template at the geographical location; and

adjusting a size of the polygon template to contain at least a portion of the area of interest, wherein the border of the adjusted polygon template defines the polygon of interest.

5. The method of claim 1, wherein combining multiple images from the satellite imagery to form a single image comprises at least one of:

using the multiple images from the satellite imagery to create a single image having a higher resolution than any individual image of the multiple images, or

stitching together proximate, adjacent, or overlapping images covering an area to create a single image of the area.

6. The method of claim 1, wherein modifying the satellite imagery to visually enhance portions of an image comprises at least one of:

reducing noise in the satellite imagery,

enhancing contrast in the satellite imagery, or

balancing color in the satellite imagery.

7. The method of claim 1, wherein obtaining image analysis information comprises:

sending the processed satellite imagery to a worker computer;

receiving an object indicator from the worker computer; and

associating the object indicator with a property of an object of interest in the satellite imagery, wherein the property comprises a size, a quantity, a location, or a presence.

8. The method of claim 1, wherein determining a correlation between the image analysis information and the economic data comprises deriving a relationship between a value of the economic data with a property of an object of interest in the satellite imagery, wherein the property comprises a size, a boundary, a quantity, a location, or a presence.

9. The method of claim 1, further comprising:

generating a correlation history between prior economic data and prior image analysis information; and

predicting future economic data or future image analysis information based at least partly on the correlation history.

10. The method of claim 1, further comprising:

receiving updated input control parameters;

obtaining updated satellite imagery corresponding to the updated input control parameters;

obtaining updated image analysis information; and

determining a correlation between the updated image analysis information and the economic data.

11. A system for correlating features in satellite imagery with economic data, the system comprising:

computer hardware comprising a physical processor, the computer hardware programmed to execute;

a location search subsystem configured to;

receive input control parameters comprising a textual description of an area of interest and a time range,

analyze the received input control parameters to obtain location information of the area of interest, and

retrieve a geographical location from a data source corresponding to the obtained location information;

an image acquisition subsystem configured to;

determine one or more polygons of interest corresponding to the geographical location, wherein a polygon of interest comprises coordinates delineating a boundary containing at least a portion of the area of interest; and

obtain satellite imagery of the area of interest, the satellite imagery comprising a plurality of images, such that;

a surface area of each of the one or more polygons of interest is covered by the satellite imagery, and

an acquisition time of the satellite imagery is within the time range;

an image analysis subsystem configured to;

process the satellite imagery by at least one of;

combining multiple images from the satellite imagery to form a single image, selecting a preferred image from the satellite imagery, or modifying the satellite imagery to visually enhance portions of an image; and

obtain image analysis information related to the processed satellite imagery, wherein the image analysis information comprises at least one of;

a quantity of an object of interest in the processed satellite imagery,

a boundary of an object of interest in the processed satellite imagery,

a location of an object of interest in the processed satellite imagery,

a size of an object of interest in the processed satellite imagery, or

an indication of an occurrence of an event in the processed satellite imagery; and

an economic data subsystem configured to determine a correlation between the image analysis information and the economic data.

12. The system of claim 11, wherein the location search subsystem is configured to receive feedback from the image acquisition subsystem, the feedback comprising updated input control parameters.

13. The system of claim 12, wherein the feedback is provided responsive to a determination that the one or more polygons of interest extends outside of the obtained satellite imagery.

14. The system of claim 11, wherein the location search subsystem is configured to receive feedback from the economic data subsystem, the feedback comprising updated input control parameters.

15. The system of claim 14, wherein the feedback is provided in response to a determination that the image analysis information is insufficiently correlated with the economic data.

16. The system of claim 11, wherein to determine the one or more polygons of interest, the image acquisition subsystem is configured to:

create a polygon template having a border;

place the polygon template at the geographical location; and

adjust a size of the polygon template to contain at least a portion of the area of interest, wherein the border of the adjusted polygon template defines the polygon of interest.

17. The system of claim 11, wherein the image analysis subsystem is further configured to:

send the processed satellite imagery to a worker computer,

receive an object indicator from the worker computer, and

associate the object indicator with a property of an object of interest in the processed satellite imagery, wherein the property comprises a size, a boundary, a quantity, a location, or a presence.

18. The system of claim 11, wherein the economic data subsystem is further configured to determine predicted economic data based on the image analysis information and a correlation between prior economic data and prior image analysis information.