Adaptive image acquisition and processing with image analysis feedback

• US 9,105,128 B2
• Filed: 08/26/2011
• Issued: 08/11/2015
• Est. Priority Date: 08/26/2011
• Status: Active Grant

• Current Assignee: Planet Labs PBC
• Original Assignee: Skybox Imaging, Inc. (Planet Labs PBC)
• Inventors: Robinson, M. Dirk, Robertson, Mark, Isaac, Hadar, Guinan, Oliver, Melendez, Thomas Joseph, Berkenstock, Daniel, Mann, Julian
• Primary Examiner(s): Drennan, Barry
• Application Number:
  US13/219,461
• Publication Number:
  US 20130051672A1
• Time in Patent Office: 1,446 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06T 17/05 Geographic models

Abstract

Described are systems, methods, computer programs, and user interfaces for image location, acquisition, analysis, and data correlation that uses human-in-the-loop processing, Human Intelligence Tasks (HIT), and/or or automated image processing. Results obtained using image analysis are correlated to non-spatial information useful for commerce and trade. For example, images of regions of interest of the earth are used to count items (e.g., cars in a store parking lot to predict store revenues), detect events (e.g., unloading of a container ship, or evaluating the completion of a construction project), or quantify items (e.g., the water level in a reservoir, the area of a farming plot).

Claims

1. A computer-implemented method for selecting and processing imagery for use in analysis tasks, comprising:

receiving, by one or more computing devices, a search location;

automatically determining, by the one or more computing devices, one or more derived coordinates based on the received search location;

automatically receiving, by the one or more computing devices, instructions that determine one or more polygons of interest based on the derived coordinates;

automatically obtaining, by the one or more computing devices, imagery of the one or more polygons of interest;

automatically processing, by the one or more computing devices, the obtained imagery based on one or more image processing instructions, wherein the image processing instructions process the obtained imagery to help discriminate objects of interest from background within the obtained imagery;

automatically receiving, by the one or more computing devices, instructions that select multiple objects of interest within the obtained imagery from background of the imagery;

automatically overlaying, by the one or more computing devices, multiple visual marks on respective areas of the obtained imagery, each visual mark overlayed on one of the selected multiple objects of interest and

automatically determining, by the one or more computing devices, a correlation between a count of the selected multiple objects of interest represented by the overlayed multiple visual marks and economic data, wherein the correlation is based at least in part on previously collected correlations between image analysis data and economic data.

2. The computer-implemented method of claim 1, wherein the one or more polygons of interest are also based on the received search location.

3. The computer-implemented method of claim 1, further comprising receiving, by the one or more computing devices, feedback consisting of an updated search location;

and wherein automatically obtaining, by the one or more computing devices, imagery of one or more polygons of interest is modified based on the feedback.

4. The computer-implemented method of claim 3, wherein the feedback is provided responsive to a determination that the one or more polygons of interest extends outside of the obtained imagery.

5. The computer-implemented method of claim 3, wherein the feedback is provided responsive to a determination that the correlation between the count of the selected multiple objects of interest represented by the overlayed multiple visual marks and economic data is inconsistent.

6. The computer-implemented method of claim 1, wherein the imagery is obtained from a database.

7. The computer-implemented method of claim 1, wherein the imagery is obtained from an overhead imaging satellite.

8. The computer-implemented method of claim 1, wherein new imagery is obtained responsive to a determination that the obtained imagery is insufficient with respect to one or more of:

a timeliness constraint, a resolution, and a quality.

9. The computer-implemented method of claim 1, further orthorectifying, by the one or more computing devices, obtained imagery and correcting obtained imagery for angular distortion.

10. The computer-implemented method of claim 1, wherein obtaining imagery of the polygons of interest comprises obtaining imagery until an area of all polygons of interest is covered by the obtained imagery.

11. The computer-implemented method of claim 1, wherein determination of polygons of interest is based on an expected task polygon shape.

12. The computer-implemented method of claim 1, wherein the instructions that determine one or more polygons of interest based on the derived coordinates comprise:

centering a polygon template around the derived coordinates, the polygon template comprising a border; and

estimating a boundary of the polygon based on a location of the border of the template.

13. The computer-implemented method of claim 1, wherein automatically receiving, by the one or more computing devices, instructions that determine one or more polygons of interest based on the derived coordinates comprises:

presenting a human intelligence task for creating polygons; and

receiving the polygon as a result of the task.

14. The computer-implemented method of claim 1, further comprising receiving feedback consisting of at least one of:

a frequency with which to obtain imagery, and a time to obtain imagery; and

wherein automatically obtaining, by the one or more computing devices, imagery of the one or more polygons of interest is modified based on the feedback.

15. The computer-implemented method of claim 1, further comprising receiving feedback consisting of at least one of an alignment for an overhead imaging satellite, and a capture resolution for the overhead imaging satellite;

and wherein automatically obtaining, by the one or more computing devices, imagery of the one or more polygons of interest is modified based on the feedback.

16. The computer-implemented method of claim 1, wherein automatically processing, by the one or more computing devices, the obtained imagery based on one or more image processing instructions comprises modifying one or more of:

a shape of the objects of interest, a color of the objects of interest, and a context of the objects of interest.

17. The computer-implemented method of claim 1, wherein automatically processing, by the one or more computing devices, the obtained imagery based on one or more image processing instructions comprises performing one or more of:

multi-image stitching, spatial image enhancement, and pixel image enhancement.

18. The computer-implemented method of claim 1, further comprising receiving feedback consisting of updated image processing instructions.

19. The computer-implemented method of claim 18, wherein the feedback is provided responsive to a determination that the existing image processing instructions are negatively affecting the imagery of the one or more polygons of interest.

20. The computer-implemented method of claim 18, wherein the feedback consisting of updated image processing instructions is received from a worker interface.

21. The computer-implemented method of claim 20, wherein the updated image processing instructions are based on at least one of:

panning, zooming, color enhancement, and contrast enhancement.

22. The computer-implemented method of claim 20, further comprising:

receiving updated image processing instructions from a plurality of worker interfaces;

aggregating the updated image processing instructions; and

applying the aggregated image processing instructions to the plurality of worker interfaces.

23. The computer-implemented method of claim 20, wherein the updated image processing instructions are applied to a specific image, to imagery from a specific location, to imagery from a specific image source, or to imagery of a specific image type.

24. The computer-implemented method of claim 18, wherein the updated image processing instructions are generated responsive to a determination that existing image processing instructions insufficiently distinguish the objects of interest from the background of the imagery.

25. The computer-implemented method of claim 1, further comprising generating a prediction of future economic metrics based on previously collected correlations between image analysis data and the economic data.

26. The computer-implemented method of claim 25, further comprising predicting, by the one or more computing devices, future counts of the multiple objects of interest based on future non-spatial data.

27. The computer-implemented method of claim 1, further comprising performing, by the one or more computing devices, statistical analysis on the count of the selected multiple objects of interest represented by the overlayed multiple visual marks to ensure a required confidence score.

28. The computer-implemented method of claim 27, wherein the performing statistical analysis on the count of the selected objects of interest comprises receiving multiple counts of the selected objects of interest, aggregating the multiple counts together in a click cloud, and determining the standard deviation of overlayed visual marks corresponding to the respective multiple counts.