Image-based procedural remodeling of buildings

• US 9,129,432 B2
• Filed: 01/20/2011
• Issued: 09/08/2015
• Est. Priority Date: 01/28/2010
• Status: Active Grant

• Current Assignee: Hong Kong University of Science and Technology
• Original Assignee: Hong Kong University of Science and Technology
• Inventors: Quan, Long, Zhao, Peng, Fang, Tian, Zhang, Honghui
• Primary Examiner(s): Chauhan, Ulka
• Assistant Examiner(s): HILL, SCHILLER D
• Application Number:
  US13/010,193
• Publication Number:
  US 20110181589A1
• Time in Patent Office: 1,692 Days
• Field of Search: 345/420
• US Class Current: 1/1
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2210/04 Architectural design, interior design

Abstract

Systems and methods are provided to facilitate architectural modeling. In one aspect, a modeling system is provided. This includes a processor configured to generate remodeled images associated with one or more architectural structures. A procedural model is configured to integrate an approximate model from aerial images of the architectural structures and a three-dimensional (3D) reconstruction model from ground-level images of the architectural structures.

Claims

1. A modeling system, comprising:

a procedural model configured to receive an approximate three-dimensional (3D) model generated from one or more aerial images of an architectural structure and generate a 3D reconstruction model from one or more ground-level images of the architectural structure; and

a processor configured to generate a remodeled image associated with the architectural structure as a function of a set of contain rules that defines a hierarchical relationship for a set of structure elements contained in a region of the architectural structure, a set of shape grammar rules determined based on structural information associated with the approximate 3D model and at least one different approximate 3D model, textural information associated with the approximate 3D model and different textural information associated with the 3D reconstruction model, wherein the textural information and the different textural information are synthesized with the set of shape grammar rules to generate the remodeled image, wherein a structure element from the set of structure elements is associated with the region of the architectural structure and a position with respect to one or more other structure elements related to the region, wherein the region and the position are defined by the set of contain rules, wherein the structure element is extracted from an automated bottom-up and top-down analysis of the one or more aerial images and the one or more ground-level images, and wherein descending structure elements in the hierarchical relationship with respect to the structure element are subdivided without a set of split grammar rules.

2. The system of claim 1, wherein the approximate 3D model is a 3D mesh model.

3. The system of claim 1, wherein the 3D reconstruction model is generated based at least in part on structure from motion (SFM) data that includes one or more 3D point clouds and one or more camera angles from the one or more ground-level images.

4. The system of claim 1, further comprising a registration component configured to associate the one or more ground-level images with the approximate 3D model.

5. The system of claim 1, further comprising a decomposition component configured to isolate one or more structure elements included in the set of structure elements.

6. The system of claim 1, further comprising a first rules component configured to derive one or more mesh rules from the approximate 3D model, wherein the one or more mesh rules include one or more building roof regions and one or more faç

ade planes associated with the architectural structure.

7. The system of claim 6, further comprising a second rules component configured to derive one or more texture rules from the 3D reconstruction model, wherein the one or more texture rules include repetitive structure elements in the one or more faç

ade planes associated with the architectural structure.

8. The system of claim 7, further comprising a synthesis component configured to determine the textural information from the one or more mesh rules and the different textural information from the one or more texture rules.

9. The system of claim 8, wherein the synthesis component is configured to generate one or more remodeled structure elements for the remodeled image.

10. The system of claim 1, wherein the set of contain rules defines a visibility priority value for the structure elements.

11. The system of claim 1, wherein the set of shape grammar rules includes one or more computer generated architecture (CGA) grammar rules to generate the remodeled image, wherein the one or more CGA grammar rules define at least the set of structure elements contained in the region of the architectural structure.

12. The system of claim 1, wherein overlapping structure elements of the set of structure elements are grouped to form a combined structure element.

13. A method, comprising:

receiving, by a device comprising a processor, one or more ground-level images associated with a building structure;

receiving, by the device, a three-dimensional model generated based on one or more satellite images associated with the building structure and at least one other three-dimensional model generated based on one or more other satellite images associated with one or more other building structures;

generating, by the device, one or more three-dimensional structures based on the one or more ground-level images;

determining, by the device, a set of shape grammar rules based on structural information associated with the three-dimensional model and the at least one other three-dimensional model;

determining, by the device, a set of contain rules that defines a hierarchical relationship for a set of structure elements included in a region of the building structure, a structure element from the set of structure elements being associated with the region of the building structure and a position with respect to one or more other structure elements related to the region, the region and the position being defined by the set of contain rules, the structure element being extracted from an automated bottom-up and top-down analysis of the one or more satellite images and the one or more ground-level images, and descending structure elements in the hierarchical relationship with respect to the structure element being subdivided without a set of split grammar rules;

determining, by the device, textural information based on the three-dimensional model;

determining, by the device, other textural information based on the one or more three-dimensional structures; and

employing, by the device, the set of shape grammar rules, the set of contain rules, the textural information and the other textural information to facilitate remodeled imaging of the building structure.

14. The method of claim 13, wherein the determining the textural information includes determining the textural information based on one or more mesh rules determined from the three-dimensional model, and wherein the determining the other textural information includes determining the other textural information based on one or more texture rules determined from the one or more three-dimensional structures.

15. The method of claim 14, further comprising generating, by the device, at least one remodeled building image based in part on the one or more mesh rules and the one or more texture rules.

16. The method of claim 13, wherein the determining the set of shape grammar rules includes determining one or more computer generated architecture (CGA) grammar rules.

17. The method of claim 13, further comprising defining, by the device, bounding box coordinates for the region of the building structure.

18. The method of claim 13, further comprising defining, by the device, one or more characteristics that are shared by the one or more structure elements of the set of structure elements.

19. A non-transitory computer-readable medium comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, the instructions comprising:

instructions for causing a computer to process at least one ground-level image of at least one building structure, at least one model derived from at least one aerial image of the at least one building structure, and at least one other model derived at least one other aerial image of at least one other building structure;

instructions for causing the computer to generate at least one other model in accordance with at least one motion sequence associated with the at least one ground-level image;

instructions for determining a set of shape grammar rules based on structural information associated with the at least one model and the at least one other model;

instructions for determining a set of contain rules that defines a hierarchical relationship for a set of architectural elements associated with a region of the at least one building structure, wherein an architectural element from the set of architectural elements is associated with the region of the at least one building structure and a position with respect to one or more other architectural elements within the region, wherein the region and the position are defined by the set of contain rules, wherein the structure element is extracted from an automated bottom-up and top-down analysis of the at least one aerial image and the at least one ground-level image, and wherein descending structure elements in the hierarchical relationship with respect to the structure element are subdivided without a set of split grammar rules;

instructions for causing the computer to determine texture data based on the at least one model;

instructions for causing the computer to determine different texture data based on the at least one other model; and

instructions for causing the computer to generate at least one remodeled image of the at least one building structure based at least in part on the set of contain rules, wherein the texture data and the different texture data are synthesized with the set of shape grammar rules to generate the at least one remodeled image.

20. The non-transitory computer-readable medium of claim 19, wherein the texture data is determined based on at least one mesh rule associated with the at least one model and the different texture data is determined based on at least one texture rule associated with the at least one other model.