Moving vehicle detection and analysis using low resolution remote sensing imagery

• US 10,255,523 B2
• Filed: 11/14/2016
• Issued: 04/09/2019
• Est. Priority Date: 11/16/2015
• Status: Active Grant

• Current Assignee: Orbital Insight, Inc.
• Original Assignee: Orbital Insight, Inc.
• Inventors: Kraft, Adam Wiggen, Babenko, Boris Aleksandrovich, Baxter, Michael Alan, Bickerton, Steven Jeffrey
• Primary Examiner(s): Entezari, Michelle M
• Application Number:
  US15/351,373
• Publication Number:
  US 20170140245A1
• Time in Patent Office: 876 Days
• Field of Search: None
• CPC Class Codes: G06F 18/2132 based on discrimination criteria, e.g. discriminant analysis
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/762 using clustering, e.g. of similar faces in social networks
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

Disclosed is a method and system for processing images from an aerial imaging device. A moving vehicle analysis system receives images from an aerial imaging device. The system may perform edge analysis in the images to identify a pairs of edges corresponding to a road. The system may identify pixel blobs in the images including adjacent pixels matching each other based on a pixel attribute. The system uses a machine learning model for generating an output identifying moving vehicles in the images. The system determines a count of the moving vehicles captured by the images, where each moving vehicle is associated with corresponding pixel blobs.

Claims

1. A method for processing images from an aerial imaging device, the method comprising:

receiving a plurality of images of a geographical area, each image of the plurality of images captured at a distinct time;

generating, by a machine learning model based on the plurality of images, an output comprising;

a plurality of blobs, wherein each blob comprises one or more adjacent pixels in a distinct image of the plurality of images, the adjacent pixels matching each other based on a pixel attribute;

a score indicative of a likelihood that the plurality of blobs corresponds to a moving vehicle;

storing an association between the plurality of blobs and the moving vehicle responsive to the generated score exceeding a threshold;

determining a count of one or more moving vehicles in the plurality of images, each moving vehicle associated with a corresponding plurality of blobs

receiving a pixel resolution of the plurality of images;

determining a number of pixels in each blob of the plurality of blobs;

determining a size of each vehicle of the one or more moving vehicles based on the pixel resolution and the number of pixels in each blob of the corresponding plurality of blobs associated with the moving vehicle;

determining a number of pixels associated with a length of a vehicular path;

determining the length of the vehicular path based on the pixel resolution and the number of pixels associated with the length of the vehicular path;

for each of the one or more moving vehicles, determining a directional vector for the corresponding plurality of blobs associated with the moving vehicle by determining a difference between a centroid of each blob of the corresponding plurality of blobs, the directional vector corresponding to a speed and a direction of the moving vehicle;

for each of the one or more moving vehicles, determining a length of the directional vector based on the pixel resolution; and

for each of the one or more moving vehicles, determining a speed as the length of the directional vector divided by a difference in time between the plurality of images.

2. The method of claim 1, wherein each image of the plurality of images corresponds to a spectral band, wherein a spectral band corresponds to a range of wavelengths of light.

3. The method of claim 1, further comprising:

for each image of the plurality of images, performing pixel clustering to identify a blob.

4. The method of claim 1, wherein the output further comprises at least one of:

one or more pixels locations corresponding to a blob and a score indicative of a likelihood that the one or more pixels locations correspond to a blob;

a plurality of pixel locations corresponding to a vehicular path and a score indicative of a likelihood that the plurality of pixel locations correspond to a vehicular path;

a number of pixels in each blob of the plurality of blobs;

or

an association between the plurality of blobs and a moving vehicle.

5. The method of claim 1, further comprising:

receiving a plurality of training sets, each training set comprising a labeled plurality of blobs corresponding to a moving vehicle; and

training the machine learning model using the training sets, the machine learning model configured to generate an output comprising a plurality of blobs and a score indicative of a likelihood that the plurality of blobs corresponds to a moving vehicle.

6. The method of claim 1, further comprising:

determining a density of moving vehicles based on the count of the one or more moving vehicles and the length of the vehicular path.

7. The method of claim 1, further comprising:

for each of the one or more moving vehicles, determining a speed as the length of the directional vector divided by a difference in time between the plurality of images.

8. The method of claim 7, further comprising:

determining an average speed as the sum of the speeds of the one or more moving vehicles divided by the count of the one or more moving vehicles.

9. The method of claim 8, further comprising:

analyzing a traffic pattern comprising at least one of;

a time of capture of an image of the plurality of images;

the count of the one or more moving vehicles;

the size of each vehicle of the one or more moving vehicles;

the average speed;

the density of moving vehicles;

or

the length of the vehicular path; and

sending re-routing information to the one or more moving vehicles responsive to the analyzed traffic pattern exceeding a threshold.

10. The method of claim 1, further comprising:

performing edge analysis in an image of the plurality of images to identify a pair of edges, the pair of edges corresponding to a vehicular path.

11. The method of claim 10, further comprising:

extracting a feature vector from the plurality of images, the feature vector comprising at least a feature representing whether a blob is located between the pair of edges corresponding to the vehicular path.

12. The method of claim 11, wherein a feature represents whether two blobs intersect each other.

13. The method of claim 11, wherein a feature represents whether a difference between a centroid of each blob of the plurality of blobs matches a labeled direction of the vehicular path.

14. The method of claim 11, wherein a feature represents an association between a plurality of pixel locations and the pixel attribute.

15. A non-transitory computer-readable storage medium comprising instructions executable by a processor, the instructions when executed by the processor causes the processor to:

receive a plurality of images of a geographical area, each image of the plurality of images captured at a distinct time;

generate, by a machine learning model based on the plurality of images, an output comprising;

a plurality of blobs, wherein each blob comprises one or more adjacent pixels in a distinct image of the plurality of images, the adjacent pixels matching each other based on a pixel attribute; and

a score indicative of a likelihood that the plurality of blobs corresponds to a moving vehicle;

store an association between the plurality of blobs and the moving vehicle responsive to the generated score exceeding a threshold;

determine a count of one or more moving vehicles in the plurality of images, each moving vehicle associated with a corresponding plurality of blobs;

receive a pixel resolution of the plurality of images;

determine a number of pixels in each blob of the plurality of blobs;

determine a size of each vehicle of the one or more moving vehicles based on the pixel resolution and the number of pixels in each blob of the corresponding plurality of blobs associated with the moving vehicle;

determine a number of pixels associated with a length of a vehicular path;

determine the length of the vehicular path based on the pixel resolution and the number of pixels associated with the length of the vehicular path;

for each of the one or more moving vehicles, determine a directional vector for the corresponding plurality of blobs associated with the moving vehicle by determining a difference between a centroid of each blob of the corresponding plurality of blobs, the directional vector corresponding to a speed and a direction of the moving vehicle;

for each of the one or more moving vehicles, determine a length of the directional vector based on the pixel resolution; and

for each of the one or more moving vehicles, determine a speed as the length of the directional vector divided by a difference in time between the plurality of images.

16. The non-transitory computer-readable storage medium of claim 15, wherein each image of the plurality of images corresponds to a spectral band, wherein a spectral band corresponds to a range of wavelengths of light.

17. The non-transitory computer-readable storage medium of claim 15, comprising further instructions that when executed by the processor causes the processor to:

for each image of the plurality of images, perform pixel clustering to identify a blob.

18. The non-transitory computer-readable storage medium of claim 15, wherein the output further comprises at least one of:

one or more pixels locations corresponding to a blob and a score indicative of a likelihood that the one or more pixels locations correspond to a blob;

a plurality of pixel locations corresponding to a vehicular path and a score indicative of a likelihood that the plurality of pixel locations correspond to a vehicular path;

a number of pixels in each blob of the plurality of blobs;

or

an association between the plurality of blobs and a moving vehicle.

19. The non-transitory computer-readable storage medium of claim 15, comprising further instructions that when executed by the processor causes the processor to:

receive a plurality of training sets, each training set comprising a labeled plurality of blobs corresponding to a moving vehicle; and

train the machine learning model using the training sets, the machine learning model configured to generate an output comprising a plurality of blobs and a score indicative of a likelihood that the plurality of blobs corresponds to a moving vehicle.

20. A system, comprising:

a processor;

a non-transitory computer-readable storage medium comprising computer-readable instructions, that when executed by the processor, cause the processor to;

receive a plurality of images of a geographical area, each image of the plurality of images captured at a distinct time;

generate, by a machine learning model based on the plurality of images, an output comprising;

a plurality of blobs, wherein each blob comprises one or more adjacent pixels in a distinct image of the plurality of images, the adjacent pixels matching each other based on a pixel attribute; and

a score indicative of a likelihood that the plurality of blobs corresponds to a moving vehicle;

store an association between the plurality of blobs and the moving vehicle responsive to the generated score exceeding a threshold;

determine a count of one or more moving vehicles in the plurality of images, each moving vehicle associated with a corresponding plurality of blobs;

receive a pixel resolution of the plurality of images;

determine a number of pixels in each blob of the plurality of blobs;

determine a size of each vehicle of the one or more moving vehicles based on the pixel resolution and the number of pixels in each blob of the corresponding plurality of blobs associated with the moving vehicle;

determine a number of pixels associated with a length of a vehicular path;

determine the length of the vehicular path based on the pixel resolution and the number of pixels associated with the length of the vehicular path;

for each of the one or more moving vehicles, determine a directional vector for the corresponding plurality of blobs associated with the moving vehicle by determining a difference between a centroid of each blob of the corresponding plurality of blobs, the directional vector corresponding to a speed and a direction of the moving vehicle;

for each of the one or more moving vehicles, determine a length of the directional vector based on the pixel resolution; and

for each of the one or more moving vehicles, determine a speed as the length of the directional vector divided by a difference in time between the plurality of images.