Systems and method for identifying locations of infrastructure assets using aerial imagery

• US 9,495,596 B2
• Filed: 07/01/2015
• Issued: 11/15/2016
• Est. Priority Date: 04/16/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Stewart, Benjamin Peter
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US14/788,941
• Publication Number:
  US 20150302253A1
• Time in Patent Office: 503 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 20/182 Network patterns, e.g. roads or rivers

Abstract

Embodiments include a computer-implemented method for identifying locations of infrastructure assets. The method including identifying a location of a first infrastructure asset and a location of a second infrastructure asset, identifying a region of interest extending at least partially between the locations of the first and second infrastructure assets, obtaining an aerial image of the region of interest, determining that the aerial image of the region of interest comprises a third infrastructure asset, determining a location of the third infrastructure asset based at least in part on the aerial image, and storing the location of the secondary infrastructure asset in an infrastructure asset datastore.

Claims

1. A computer-implemented method for identifying locations of infrastructure assets, the method comprising:

determining, by one or more processors, geographic locations of a plurality of infrastructure assets, including;

querying, by the one or more processors, an asset information database for data for known asset locations, and

receiving, by the one or more processors from the asset information database, a respective dataset for each of the plurality of infrastructure assets, including receiving an asset identifier, an indication of an asset type and an indication of an asset location as part of the dataset, wherein the indication of the asset location includes at least one of (i) geographic coordinates indicating a point where the asset is located, (ii) an identifier of a cell in which a center of the asset is located, or a (iii) an indication of a boundary of the asset;

determining, by one or more processors, a plurality of geographic regions that do not contain any infrastructure assets;

obtaining, by the one or more processors, first images of the geographic locations of the plurality of infrastructure assets and second images of the plurality of geographic regions that do not include any infrastructure assets;

generating, by the one or more processors, an image processing model using the first images and the second images, including identifying first features of the first images and second features of the second images, and training the image processing model using the first features and the second features;

receiving, by one or more processors, a candidate image; and

using the image processing model, automatically locating an infrastructure asset within the candidate image.

2. The computer-implemented method of claim 1, wherein generating the image processing model includes applying context data related to the first and second images, the context data indicating, for each image, at least one of (i) a time of day when the image was acquired, (ii) a date when the image was acquired, (iii) a source of the image, or (iv) a position and angle of a camera used to acquire the image.

3. The computer-implemented method of claim 1, wherein first images and the second images define a training corpus of images;

the method further comprising;

obtaining, by the one or more processors, a verification corpus of images including third images of geographic locations including infrastructure assets and fourth images of geographic regions that do not include any infrastructure assets;

applying, by the one or more processors, the verification corpus to the image processing model to generate a metric; and

comparing, by the one or more processors, the metric to a predetermined confidence threshold to determine whether the image processing model can be used for subsequent image processing.

4. The computer-implemented method of claim 1, wherein generating the image processing model includes generating the image processing model for a single specific type of infrastructure assets, wherein obtaining the first images includes querying an asset information database for the specific type of infrastructure assets.

5. The computer-implemented method of claim 1, wherein identifying the first features and the second features includes using one of FAST, SURF or SIFT feature detecting techniques.

6. A system comprising:

one or more processors;

an asset information database storing information regarding infrastructure assets;

one or more memory devices storing program instructions that, when executed by the one or more processors, cause the system to perform the following;

determining geographic locations of a plurality of infrastructure assets, including;

querying the asset information database for data for known asset locations, and

receiving, from the asset information database, a respective dataset for each of the plurality of infrastructure assets, including receiving an asset identifier, an indication of an asset type and an indication of an asset location as part of the dataset, wherein the indication of the asset location includes at least one of (i) geographic coordinates indicating a point where the asset is located, (ii) an identifier of a cell in which a center of the asset is located, or a (iii) an indication of a boundary of the asset,

determining a plurality of geographic regions that do not contain any infrastructure assets,

obtaining first images of the geographic locations of the plurality of infrastructure assets and second images of the plurality of geographic regions that do not include any infrastructure assets,

generating an image processing model using the first images and the second images, including identifying first features of the first images and second features of the second images, and training the image processing model using the first features and the second features,

receiving a candidate image, and

using the image processing model, automatically locating an infrastructure asset within the candidate image.

7. The system of claim 6, wherein generating the image processing model includes applying context data related to the first and second images, the context data indicating, for each image, at least one of (i) a time of day when the image was acquired, (ii) a date when the image was acquired, (iii) a source of the image, or (iv) a position and angle of a camera used to acquire the image.

8. The system of claim 6, wherein first images and the second images define a training corpus of images;

wherein the instructions further cause the system to perform the following;

obtaining a verification corpus of images including third images of geographic locations including infrastructure assets and fourth images of geographic regions that do not include any infrastructure assets;

applying the verification corpus to the image processing model to generate a metric; and

comparing the metric to a predetermined confidence threshold to determine whether the image processing model can be used for subsequent image processing.

9. The system of claim 6, wherein generating the image processing model includes generating the image processing model for a single specific type of infrastructure assets, wherein obtaining the first images includes querying an asset information database for the specific type of infrastructure assets.

10. The system of claim 6, wherein identifying the first features and the second features includes using one of FAST, SURF or SIFT feature detecting techniques.

13. The computer-readable medium of claim 10, wherein first images and the second images define a training corpus of images, the steps further comprising:

obtaining a verification corpus of images including third images of geographic locations including infrastructure assets and fourth images of geographic regions that do not include any infrastructure assets;

applying the verification corpus to the image processing model to generate a metric; and

comparing the metric to a predetermined confidence threshold to determine whether the image processing model can be used for subsequent image processing.

14. The computer-readable medium of claim 10, wherein generating the image processing model includes generating the image processing model for a single specific type of infrastructure assets, wherein obtaining the first images includes querying an asset information database for the specific type of infrastructure assets.

11. A non-transitory computer-readable medium comprising program instructions stored thereon that are executable by a processors to cause the following steps operable to identify locations of infrastructure assets:

determining geographic locations of a plurality of infrastructure assets, including;

querying an asset information database for data for known asset locations, and

receiving, from the asset information database, a respective dataset for each of the plurality of infrastructure assets, including receiving an asset identifier, an indication of an asset type and an indication of an asset location as part of the dataset, wherein the indication of the asset location includes at least one of (i) geographic coordinates indicating a point where the asset is located, (ii) an identifier of a cell in which a center of the asset is located, or a (iii) an indication of a boundary of the asset;

determining a plurality of geographic regions that do not contain any infrastructure assets;

obtaining first images of the geographic locations of the plurality of infrastructure assets and second images of the plurality of geographic regions that do not include any infrastructure assets;

generating an image processing model using the first images and the second images, including identifying first features of the first images and second features of the second images, and training the image processing model using the first features and the second features;

receiving a candidate image; and

using the image processing model, automatically locating an infrastructure asset within the candidate image.

12. The computer-readable medium of claim 11, wherein generating the image processing model includes applying context data related to the first and second images, the context data indicating, for each image, at least one of (i) a time of day when the image was acquired, (ii) a date when the image was acquired, (iii) a source of the image, or (iv) a position and angle of a camera used to acquire the image.