Using Human Intelligence Tasks for Precise Image Analysis

• US 20130051661A1
• Filed: 08/26/2011
• Published: 02/28/2013
• Est. Priority Date: 08/26/2011
• Status: Active Grant

• Current Assignee: Planet Labs PBC
• Original Assignee: Skybox Imaging, Inc. (Planet Labs PBC)
• Inventors: Robinson, M. Dirk, Robertson, Mark, Isaac, Hadar, Guinan, Oliver, Melendez, Thomas Joseph, Berkenstock, Daniel, Mann, Julian
• Granted Patent: US 8,873,842 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

Described are systems, methods, computer programs, and user interfaces for image location, acquisition, analysis, and data correlation that uses human-in-the-loop processing, Human Intelligence Tasks (HIT), and/or or automated image processing. Results obtained using image analysis are correlated to non-spatial information useful for commerce and trade. For example, images of regions of interest of the earth are used to count items (e.g., cars in a store parking lot to predict store revenues), detect events (e.g., unloading of a container ship, or evaluating the completion of a construction project), or quantify items (e.g., the water level in a reservoir, the area of a farming plot).

Claims

1. A system for obtaining information from imagery, the system comprising:

an image analysis subsystem configured to;

present imagery to a plurality of worker interfaces;

receive a plurality of user responses regarding the imagery; and

determine a count of one or more object of interests present in the imagery based on the user responses; and

a non-spatial correlation subsystem configured to;

correlate the count of the objects with a non-spatial data; and

predict a future non-spatial data based on a future count and the correlation between the non-spatial data and the count.

2. The system of claim 1 wherein the worker interface comprises a user interface configured to:

display the imagery;

allow a user to click using a pointer device moving an indicator on a display;

receive a user click at a click location;

place a mark on the imagery at the click location;

remove the mark responsive to a second click at the click location; and

submit the mark as the user response.

3. The system of claim 2 wherein the user interface is configured to allow a user to place a plurality of marks and requires a minimum distance between marks.

4. The system of claim 2 wherein the user interface is configured to measure a time between clicks.

5. The system of claim 1 wherein the user responses are aggregated into at least one two-dimensional click cloud indicating potential locations of objects of interest located in the imagery.

6. The system of claim 5 wherein the image analysis subsystem is configured to

fit each two dimensional click cloud to a two-dimensional probability distribution function to create at least one peak comprising a center;

and

determine a count of the objects of interest based on a count of the peaks.

7. The system of claim 6 wherein the probability density function is a Gaussian type exponential function, and wherein a width of the function is proportional to an expected size of the object of interest.

8. The system of claim 6 wherein the peak comprises a minimum percentage of clicks within a maximum distance from the center.

9. The system of claim 8 wherein the maximum distance is at least one of:

half of an expected width of the object of interest, half of a median width of the object of interest, and half of an average width of an object of interest.

10. The system of claim 6 wherein image analysis subsystem is configured to use the click cloud, peak, and count as training classifiers for computer learning program.

11. The system of claim 6 wherein the image analysis subsystem is further configured to determine a confidence score for the count.

12. The system of claim 11 wherein the confidence score is determined based on a measurement of a time between clicks from the user responses that make up the two-dimensional click cloud.

13. The system of claim 11 wherein each user response comprises a user count, and the confidence score is determined based on a variance between the user counts across all user responses.

14. The system of claim 11 wherein the confidence score is determined based on an amount of time taken to create the user responses.

15. The system of claim 11 wherein the image analysis subsystem is further configured to alter the imagery if the confidence score is below a threshold.

16. The system of claim 6 wherein the image analysis subsystem is further configured to use a density of clicks in the peak to alter the imagery presented to the worker interfaces.

17. The system of claim 6 wherein the image analysis subsystem is further configured to correlate a time between clicks to a density of clicks in the peak .

18. The system of claim 6 wherein image analysis subsystem is configured to use the user responses, and either the count or the two-dimensional click cloud to determine an overcount or an undercount of a user providing one of the user responses.

19. The system of claim 1 wherein the non-spatial correlation subsystem is further configured to predict a future count based on a future non-spatial data and the correlation between the non-spatial data and the count.

20. A system for estimating non-spatial data from overhead imagery, the system comprising:

a location search subsystem configured to concert a received location query into one or more derived coordinates;

an image acquisition subsystem configured to obtain imagery of a polygon of interest based on the derived coordinates;

an image analysis subsystem configured to;

process the obtained imagery to distinguish one or more objects of interest from a background of the imagery;

receive results determined from the objects of interest; and

a non-spatial correlation subsystem configured to determine a correlation between a non-spatial data and the results.