Determine heights/shapes of buildings from images with specific types of metadata

• US 9,317,966 B1
• Filed: 02/15/2012
• Issued: 04/19/2016
• Est. Priority Date: 02/15/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Deephanphongs, David
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US13/396,819
• Time in Patent Office: 1,525 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01B 11/02 for measuring length, width or thickness G01B11/08 takes precedence
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 13/20 Image signal generators

Abstract

Embodiments combine at least two photographic images of a building as well as camera parameters associated with the images with information about the legal boundaries of a lot associated with the building in order to construct a three-dimensional model of the building.

Claims

1. A computer-implemented method for constructing a three-dimensional model of a building using a plurality of two-dimensional photographic images in an image database and a building lot database, the method comprising:

identifying, by one or more computing devices, a first photographic image of the building from the image database, the first photographic image taken from a first perspective specified by a first set of camera parameters;

identifying, by the one or more computing devices, a second photographic image of the building from the image database, the second photographic image taken from a second perspective specified by a second set of camera parameters, the second perspective different from the first perspective;

identifying, by the one or more computing devices, an obstructed view of at least a portion of the building in the first or second photographic images;

identifying, by the one or more computing devices, a third-photographic image of the building from the image database, the third photographic image taken from a third perspective specified by a third set of camera parameters, wherein the third photographic image is chosen from the Image database such that the third photographic image provides an unobstructed view of the at least a portion of the building that is obstructed in the first or second photographic image; and

determining, by the one or more computing devices, a first feature in the first photographic image corresponding to at least a portion of a roof of the building;

determining, by the one or more computing devices, a second feature in the second photographic image corresponding to at least a portion of the roof of the building;

determining, by the one or more computing devices, a third feature in the third photographic image corresponding to at least a portion of the roof of the building;

determining, by the one or more computing devices, a height of the building based on the positions of first, second, and third features within the first, second, and third photographic images respectively, and the first, second, and third sets of camera parameters;

determining, by the one or more computing devices, an identity of the building using reverse geocoding based at least in part on the first and second set of camera parameters;

accessing, by the one or more computing devices, predetermined legal boundaries from the building lot database based at least in part on the determined identity of the building, the legal boundaries defining a lot associated with the building;

constructing, by the one or more computing devices, a three-dimensional model of the building based on the height and the legal boundaries, wherein the legal boundaries specify, at least in part, a perimeter of the model and the height of the building specifies, at least in part, a height of the model; and

improving, by the one or more computing devices, the three-dimensional model based on information provided by the third photographic image and the third set of camera parameters by modeling aspects of the building that are obstructed in the first photographic image and the second photographic image.

2. The method of claim 1, wherein the second photographic image is chosen from the image database with a second perspective wherein the first feature and second feature correspond to the same portion of the roof of the building.

3. The method of claim 1, wherein the three-dimensional model is created using photogrammetry, wherein the first photographic image and the second photographic image are treated as binocular views of the same building.

4. The method of claim 1, wherein one or both of the first photographic image and the second photographic image are aerial photographs or satellite photographs.

5. The method of claim 1, further comprising receiving, by the one or more computing devices, information about the building using remote sensing techniques, and incorporating the information derived from the remote sensing into constructing the three-dimensional model.

6. The method of claim 1, wherein the legal boundaries define the lot by providing one or more of:

corners of the lot, edges of the lot, or dimensions of the lot.

7. The method of claim 1, further comprising:

identifying, by the one or more computing devices, a plurality of buildings in a physical region; and

repeating, by the one or more computing devices, the identifying a first photographic image, identifying a second photographic image, determining a first feature, determining a second feature, determining a height, identifying legal boundaries, and constructing a three-dimensional model for each of the plurality of buildings in the physical region, such the method constructs a three-dimensional model of the plurality of buildings in the physical region.

8. A system for constructing a three-dimension model of a building using a plurality of two-dimensional photographic images and a building lot database, the system comprising:

one or more memory devices;

one or more processors;

an image database, comprising a plurality of photographic images, each taken from a perspective specified by a set of camera parameters;

a building lot database, comprising a plurality of buildings, each associated with legal boundaries;

a building selection module, configured to receive a selection of a building from the building lot database for which a three-dimensional model is to be constructed;

a photographic image identification module, configured to;

identify a first photographic image of the building from the image database, the first photographic image taken from a first perspective specified by a first set of camera parameters,

identify a second photographic image of the building from the image database, the second photographic image taken from a second perspective specified by a second set of camera parameters, the second perspective different from the first perspective, and

identify a third photographic image of the building from the image database when the first or second photographic images depict an obstructed view of at least a portion of the building, the third photographic image taken from a third perspective specified by a third set of camera parameters, the third photographic image depicting a clear view of the at least a portion of the building that is obstructed in the first or second photographic images;

a height determination module, configured to;

determine a first feature in the first photographic image corresponding to at least a portion of a roof of the building;

determine a second feature in the second photographic image corresponding to at least a portion of the roof of the building;

determine a third feature in the third photographic image corresponding to at least a portion of the roof of the building;

determine a height of the building based on the positions of first, second, and third features within the first, second, and third photographic images respectively, and the first, second, and third sets of camera parameters;

a boundary identification module, configured to identify predetermined legal boundaries in the building lot database that define a lot associated with the building, the legal boundaries being identified based at least in part on an identity of the building determined using reverse geocoding based at least in part on the first and second set of camera parameters;

a three-dimensional modeling module, configured to construct a three-dimensional model of the building based on the height and the legal boundaries, wherein the legal boundaries specify, at least in part, a perimeter of the model and the height of the building specifies, at least in part, a height of the model; and

improve the three-dimensional model based on information provided by the third photographic image and the third set of camera parameters by modeling aspects of the building that are obstructed in the first photographic image and the second photographic image.

9. The system of claim 8, wherein the second photographic image is chosen from the image database with a second perspective wherein the first feature and second feature correspond to the same portion of the roof of the building.

10. The system of claim 8, wherein the three-dimensional model is created using photogrammetry, wherein the first photographic image and the second photographic image are treated as binocular views of the same building.

11. The system of claim 8, wherein one or both of the first photographic image and the second photographic image are aerial photographs or satellite photographs.

12. The system of claim 8, the three-dimensional modeling module is further configured to receive information about the building using remote sensing techniques, and to incorporate the information derived from the remote sensing into constructing the three-dimensional model.

13. The system of claim 8, wherein the legal boundaries define the lot by providing one or more of:

corners of the lot, edges of the lot, or dimensions of the lot.

14. The system of claim 8, further comprising:

a regional building identifier, configured to;

identify a plurality of buildings in a physical region; and

wherein the image database, the building lot database, the building selection module, the photographic image identification module, the height determination module, the boundary identification module, and the three-dimensional modeling module are further configured to repeat the identifying a first photographic image, identifying a second photographic image, determining a first feature, determining a second feature, determining a height, identifying legal boundaries, and constructing a three-dimensional model for each of the plurality of buildings in the physical region identified by the regional building identifier, such the system is configured to construct a three-dimensional model of the buildings in the physical region.

15. One or more tangible, non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processing devices cause the one or more processing devices to perform operations, the operation comprising:

identifying a first photographic image of a building from the image database, the first photographic image taken from a first perspective specified by a first set of camera parameters;

identifying a second photographic image of the building from the image database, the second photographic image taken from a second perspective specified by a second set of camera parameters, the second perspective different from the first perspective;

identifying an obstructed view of at least a portion of the building in the first or second photographic images;

identifying a third photographic image of the building from the image database, the third photographic image taken from a third perspective specified by a third set of camera parameters, wherein the third photographic image is chosen from the image database such that the third photographic image provides an unobstructed view of the at least a portion of the building that is obstructed in the first or second photographic image;

determining a first feature in the first photographic image corresponding to at least a portion of a roof of the building;

determining a second feature in the second photographic image corresponding to at least a portion of the roof of the building;

determining a third feature in the third photographic image corresponding to at least a portion of the roof of the building;

determining a height of the building based on the positions of first, second, and third features within the first, second, and third photographic images respectively, and the first, second, and third sets of camera parameters;

determining an identity of the building using reverse geocoding based at least in part, on the first and second set of camera parameters;

accessing predetermined legal boundaries that define a lot associated with the building in the building lot database based at least in part on the determined identity of the building;

constructing a three-dimensional model of the building based on the height and the legal boundaries, wherein the legal boundaries specify, at least in part, a perimeter of the model and the height of the building specifies, at least in part, a height of the model; and

improving the three-dimensional model based on information provided by the third photographic image and the third set of camera parameters by modeling aspects of the building that are obstructed in the first photographic image and the second photographic image.