FOUR-DIMENSIONAL AUGMENTED REALITY MODELS FOR INTERACTIVE VISUALIZATION AND AUTOMATED CONSTRUCTION PROGRESS MONITORING

• US 20130155058A1
• Filed: 12/12/2012
• Published: 06/20/2013
• Est. Priority Date: 12/14/2011
• Status: Active Grant

• Current Assignee: Board of Trustees of The University of Illinois
• Original Assignee: Board of Trustees of The University of Illinois
• Inventors: Golparvar-Fard, Mani, Pea-Mora, Feniosky A., Savarese, Silvio
• Granted Patent: US 9,070,216 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06Q 10/06311 Scheduling, planning or task assignment for a person or group
• CPC Class Codes: G06Q 10/103 Workflow collaboration or project management
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30132 Masonry; Concrete
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06T 7/001 using an image reference approach

Abstract

A method for monitoring construction progress may include storing in memory multiple unordered images obtained from photographs taken at a site; melding the multiple images to reconstruct a dense three-dimensional (3D) as-built point cloud model including merged pixels from the multiple images in 3D space of the site; rectifying and transforming the 3D as-built model to a site coordinate system existing within a 3D as-planned building information model (“as-planned model”); and overlaying the 3D as-built model with the 3D as-planned model for joint visualization thereof to display progress towards completion of a structure shown in the 3D as-planned model. The processor may further link a project schedule to the 3D as-planned model to generate a 4D chronological as-planned model that, when visualized with the 3D as-built point cloud, provides clash detection and schedule quality control during construction.

Claims

1. A computer system comprising:

computer storage for storing an unordered group of images obtained from photographs taken at a site; and

a processor connected with the storage and configured to meld together the unordered group of images such as to reconstruct dense three-dimensional (3D) and four-dimensional (4D) as-built point cloud models (“

as-built models”

) comprising merged pixels from the unordered group of images over time, where time is a fourth dimension added to the three dimensions of the site captured from the group of images.

2. The computer system of claim 1, where the group of images comprise low-resolution images that are widely distributed in the site.

3. The computer system of claim 1, where the processor is further configured to rectify and transform the 3D as-built model to a site coordinate system to be jointly visualized with a 3D as-planned building information model (“

as-planned model”

).

4. The computer system of claim 3, where the processor is further configured to link a project schedule to the 3D as-planned model to generate a 4D chronological as-planned model that, when visualized overlaid with the 4D as-built model, provides clash detection and schedule quality control during building.

5. The computer system of claim 4, where the processor is further configured to register the unordered group of images, which are geo-tagged, with the 4D as-built and as-planned models of the site in an augmented reality environment, where resultant registrations are usable for mutual spatial and temporal comparisons.

6. The computer system of claim 4, where the processor is further configured to track building progress based on an application of the unordered group of images to which is iteratively added on a daily basis, to update the 3D as-built model as compared to the 4D as-planned model.

7. The computer system of claim 6, where the processor is further configured to color code building progress deviations of elements of the 4D as-planned model when compared with elements of the updated 3D as-built model, comprising applying different colors to elements in the 3D as-planned and as-built models according to a status of as-built progress detected within the 4D as-built model.

8. The computer system of claim 7, where the processor is further configured to generate a confidence metric comprising a probability that the color applied to an element to indicate as-built progress is correct, to provide additional data for more precise progress monitoring reports.

9. The computer system of claim 6, where the processor is further configured to apply a probabilistic model that traverses an integrated scene of the site and labels different areas of the integrated scene according to as-built occupancy and as-planned visibility, where the integrated scene comprises an integration of the 4D as-built and as-planned models.

10. A computer-implemented method comprising:

storing in memory multiple unordered images obtained from photographs taken at a site;

melding, with a processor connected with the memory, the multiple images to reconstruct a dense three-dimensional (3D) as-built point cloud model (“

as built-model”

) comprising merged pixels from the multiple images in 3D space of the site;

rectifying and transforming, with the processor, the 3D as-built model to a site coordinate system existing within a 3D as-planned building information model (“

as-planned model”

); and

overlaying, with the processor in a display connected with the processor, the 3D as-built model with the 3D as-planned model for joint visualization thereof to display progress towards completion of a structure shown in the 3D as-planned model.

11. The method of claim 10, further comprising:

transforming the 3D as-built model to a four-dimensional (4D) as-built model by updating the 3D as-built model with images from newly taken photographs as they are stored in the memory, and tracking temporal progress of the 3D as-built model as updated.

12. The method of claim 11, further comprising:

linking a project schedule to the 3D as-planned model to generate a 4D chronological as-planned model that, when visualized overlaid with the 4D as-built model, provides clash detection and schedule quality control during building; and

tracking building progress based on comparing the 4D as-built model to the 4D as-planned model.

13. The method of claim 12, further comprising:

registering the multiple images, which are geo-tagged, with the 4D as-built and as-planned models of the site in an augmented reality environment, where resultant registrations are usable for mutual spatial and temporal comparisons.

14. The method of claim 12, further comprising:

color coding building progress deviations of elements of the 4D as-planned model when compared with elements of the updated 3D as-built model, comprising applying different colors to elements in the 3D as-planned and as-built models according to a status of as-built progress detected within the 4D as-built model.

15. The method of claim 14, further comprising:

generating a confidence metric comprising a probability that the color applied to an element to indicate as-built progress is correct, to provide additional data for more precise progress monitoring reports.

16. The method of claim 12, further comprising:

applying a probabilistic model that traverses an integrated scene of the site and labels different areas of the integrated scene according to as-built occupancy and as-planned visibility, where the integrated scene comprises an integration of the 4D as-built and as-planned models.

17. A computer-readable storage medium comprising a set of instructions for configuring a construction progress monitoring system executable by a computing device having a processor and memory, the computer-readable medium comprising;

instructions to store in the memory multiple unordered images obtained from photographs taken at a site in response to user inputs requesting to store the photographs;

instructions to direct the processor to meld the multiple images together from memory to reconstruct a dense three-dimensional (3D) as-built point cloud model (“

as built-model”

) comprising merged pixels from the multiple images in 3D space of the site;

instructions to direct the processor to rectify and transform the 3D as-built model to a site coordinate system existing within a 3D as-planned building information model (“

as-planned model”

); and

instructions to direct the processor to overlay, in a display connected with the processor, the 3D as-built model with the 3D as-planned model for joint visualization thereof to display progress towards completion of a structure shown in the 3D as-planned model.

18. The computer-readable storage medium of claim 17, further comprising a set of instructions to direct the processor to transform the 3D as-built model to a four-dimensional (4D) as-built model by updating the 3D as-built model with images from newly taken photographs as they are stored in the memory, and tracking temporal progress of the 3D as-built model as updated.

19. The computer-readable storage medium of claim 18, further comprising a set of instructions to direct the processor to:

link a project schedule to the 3D as-planned model to generate a 4D chronological as-planned model that, when visualized overlaid with the 4D as-built model, provides clash detection and schedule quality control during building; and

track building progress based on comparing the 4D as-built model to the 4D as-planned model.

20. The computer-readable storage medium of claim 19, further comprising a set of instructions to direct the processor to register the multiple images, which are geo-tagged, with the 4D as-built and as-planned models of the site in an augmented reality environment, where resultant registrations are usable for mutual spatial and temporal comparisons.

21. The computer-readable storage medium of claim 19, further comprising a set of instructions to direct the processor to:

color code building progress deviations of elements of the 4D as-planned model when compared with elements of the updated 3D as-built model, comprising applying different colors to elements in the 3D as-planned and as-built models according to a status of as-built progress detected within the 4D as-built model; and

generate a confidence metric comprising a probability that the color applied to an element to indicate as-built progress is correct, to provide additional data for more precise progress monitoring reports.

22. The computer-readable storage medium of claim 19, further comprising a set of instructions to direct the processor to apply a probabilistic model that traverses an integrated scene of the site and labels different areas of the integrated scene according to as-built occupancy and as-planned visibility, where the integrated scene comprises an integration of the 4D as-built and as-planned models.