Processing a satellite image of a region to determine a terrain boundary included therein

• US 8,774,470 B1
• Filed: 04/03/2012
• Issued: 07/08/2014
• Est. Priority Date: 04/03/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Heiler, Matthias, Schmidt, Karl Johann, Kehl, Roland
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US13/438,788
• Time in Patent Office: 826 Days
• Field of Search: None
• US Class Current: 382/113
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

A system and method for processing a satellite image to determine a terrain boundary. Pixel data corresponding to a satellite image is obtained. For each pixel within a subset of the pixel data, multiple image features are determined corresponding to texture and color statistics for the pixel and neighboring pixels, based on a feature selection process for selecting discriminative features. An initial set of rules is generated for determining a terrain boundary. Plural image models for applying to the pixel data are created based on the initial set of rules. The plural image models are applied to the pixel data to generate at least one probability map, which represents output of the plural image models and corresponds to different terrains within the satellite image. A binary label mask is generated for the satellite image based on the probability map, and mask indicates segmentations corresponding to the different terrains.

Claims

1. A machine-implemented method of processing a satellite image of a region to determine a terrain boundary included therein, the method comprising:

obtaining pixel data corresponding to the satellite image of the region;

determining, for each pixel within a subset of the pixel data, multiple image features corresponding to texture and color statistics for the pixel and neighboring pixels of the pixel, based on a feature selection process for selecting discriminative features from the pixel data;

generating an initial set of rules for determining a terrain boundary based on the obtained multiple image features;

creating plural image models for applying to the pixel data based on the initial set of rules;

applying the plural image models to the pixel data to generate at least one probability map, wherein the at least one probability map represents output of the plural image models and corresponds to different terrains within the satellite image; and

generating a binary label mask for the satellite image based on the generated probability map, wherein the binary label mask indicates segmentations corresponding to the different terrains.

2. The method of claim 1, wherein the satellite image corresponds to a single satellite image or a blend of multiple satellite images.

3. The method of claim 1, wherein the terrain boundary corresponds to a boundary for a body of water.

4. The method of claim 1, wherein the determining comprises creating at least one histogram for each pixel within the subset of the pixel data.

5. The method of claim 4, wherein the at least one histogram corresponds to at least one of a color histogram, a texton histogram based on a Gabor filter response, or a texton histogram based on small image patches.

6. The method of claim 1, wherein the feature selection process corresponds to at least one of a linear discriminant analysis (LDA) with diagonal and with full covariance matrices or a boosting algorithm.

7. The method of claim 1, wherein the plural image models correspond to at least one of a model based on k-means clustering, a boosted decision stumps model, or a boosted linear classifier model.

8. The method of claim 7, wherein the model based on k-means clustering comprises a Gaussian mixture model with unit or with full covariance matrices.

9. The method of claim 1, further comprising:

performing a smoothing process on the probability map to enforce consistency between neighboring pixels and overlapping images.

10. The method of claim 9, wherein the overlapping images are used to remove obstacles corresponding to at least one of cloud coverage or snow coverage.

11. The method of claim 1, wherein the obtaining, determining, generating the initial set of rules, creating, applying and generating the binary label mask steps are repeatedly performed using image data of higher resolution for the satellite image.

12. The method of claim 1, wherein one or more satellite images are divided into small tiles, and wherein the obtaining, determining, generating the initial set of rules, creating, applying and generating the binary label mask steps are performed in parallel on the small tiles on a cluster of computers.

13. A system for processing a satellite image of a region to determine a terrain boundary included therein, the system comprising:

one or more processors; and

a machine-readable medium comprising instructions stored therein, which when executed by the processors, cause the processors to perform operations comprising;

obtaining pixel data corresponding to the satellite image of the region;

determining, for each pixel within a subset of the pixel data, multiple image features corresponding to texture and color statistics for the pixel and neighboring pixels of the pixel, based on a feature selection process for selecting discriminative features from the pixel data, wherein the determining comprises creating at least one histogram for each pixel within the subset of the pixel data;

generating an initial set of rules for determining a terrain boundary based on the obtained multiple image features;

creating plural image models for applying to the pixel data based on the initial set of rules;

applying the plural image models to the pixel data to generate at least one probability map, wherein the at least one probability map represents output of the plural image models and corresponds to different terrains within the satellite image; and

generating a binary label mask for the satellite image based on the generated probability map, wherein the binary label mask indicates segmentations corresponding to the different terrains.

14. The system of claim 13, wherein the satellite image corresponds to a single satellite image or a blend of multiple satellite images.

15. The system of claim 13, wherein the terrain boundary corresponds to a boundary for a body of water.

16. The system of claim 13, wherein the at least one histogram corresponds to at least one of a color histogram, a texton histogram based on a Gabor filter response, or a texton histogram based on small image patches.

17. The system of claim 13, wherein the feature selection process corresponds to at least one of a linear discriminant analysis (LDA) with diagonal and with full covariance matrices or a boosting algorithm.

18. The system of claim 13, wherein the plural image models correspond to at least one of a model based on k-means clustering, a boosted decision stumps model, or a boosted linear classifier model.

19. The system of claim 18, wherein the model based on k-means clustering comprises a Gaussian mixture model with unit or with full covariance matrices.

20. The system of claim 13, the operations further comprising:

performing a smoothing process on the probability map to enforce consistency between neighboring pixels and overlapping images.

21. The system of claim 20, wherein the overlapping images are used to remove obstacles corresponding to at least one of cloud coverage or snow coverage.

22. A non-transitory machine-readable medium comprising instructions stored therein, which when executed by a system, cause the system to perform operations comprising:

obtaining pixel data corresponding to a satellite image of a region;

determining, for each pixel within a subset of the pixel data, multiple image features corresponding to texture and color statistics for the pixel and neighboring pixels of the pixel, based on a feature selection process for selecting discriminative features from the pixel data;

generating an initial set of rules for determining a terrain boundary based on the obtained multiple image features;

creating plural image models for applying to the pixel data based on the initial set of rules;

applying the plural image models to the pixel data to generate at least one probability map, wherein the at least one probability map represents output of the plural image models and corresponds to different terrains within the satellite image;

performing a smoothing process on the probability map to enforce consistency between neighboring pixels and overlapping images, wherein the overlapping images are used to remove obstacles corresponding to at least one of cloud coverage or snow coverage; and

generating a binary label mask for the satellite image based on the generated probability map, wherein the binary label mask indicates segmentations corresponding to the different terrains.