Image capture for a multi-dimensional building model

• US 10,410,413 B2
• Filed: 04/02/2018
• Issued: 09/10/2019
• Est. Priority Date: 05/29/2015
• Status: Active Grant

• Current Assignee: Hover Inc. (Tucows Incorporated)
• Original Assignee: Hover Inc. (Tucows Incorporated)
• Inventors: Upendran, Manish, Castillo, William, Mishra, Ajay, Altman, Adam J.
• Primary Examiner(s): Broome, Said
• Application Number:
  US15/942,786
• Publication Number:
  US 20180225870A1
• Time in Patent Office: 526 Days
• Field of Search: None
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

A process for receiving, from a computing device, a series of captured building images. The process continues by processing, in real-time, each building image in the series of captured building images to determine if each building image meets a minimum criterion, wherein the minimum criteria includes applicability to be used in constructing a specific digital multi-dimensional building model. The process continues by aggregating each image meeting the minimum criteria, determining when a base set of building images has been aggregated, wherein the base set of building images includes a threshold number images to model at least a partial multi-dimensional building model representing the series of captured building images, determining one or more facades present in the partial multi-dimensional building model, determining preliminary dimensions for one or more architectural features of the one or more facades and returning, incrementally (in real-time), the preliminary dimensions to the computing device.

Claims

1. A method of processing building imagery, the method comprises:

receiving, from a computing device, a series of captured building images;

processing, in real-time, each building image in the series of captured building images to determine if each building image meets a minimum criterion, wherein the minimum criterion includes applicability for constructing a multi-dimensional building model;

aggregating each of the building images meeting the minimum criterion;

determining when a base set of building images has been aggregated, wherein the base set of building images includes a threshold number of images to construct at least a partial multi-dimensional building model representing the series of captured building images;

identifying one or more facades present in the base set of building images;

identifying one or more architectural features of the one or more facades;

determining preliminary dimensions for the one or more architectural features of the one or more facades; and

returning the preliminary dimensions to the computing device.

2. The method of claim 1 further comprises incrementally refining the preliminary dimensions based on continual processing of additional ones of the building images in the series of captured building images.

3. The method of claim 1, wherein the returning the preliminary dimensions to the computing device is performed in real-time.

4. The method of claim 1 further comprises incrementally returning the preliminary dimensions to the computing device until at least one complete set of preliminary dimensions for the one or more architectural features is returned.

5. The method of claim 1 further comprises incrementally returning the preliminary dimensions to the computing device until at least one complete set of final dimensions for the one or more architectural features is returned.

6. The method of claim 1 wherein the one or more architectural features includes roofing.

7. The method of claim 1 wherein the one or more architectural features includes siding.

8. The method of claim 1 wherein the one or more architectural features includes paintable surfaces.

9. The method of claim 1 wherein the one or more architectural features includes any of:

windows or doors.

10. The method of claim 1 further comprises incrementally returning the preliminary dimensions to the computing device until a complete set of preliminary dimensions for a plurality of architectural features is returned.

11. The method of claim 10 wherein the plurality of architectural features is selected from a group:

roofing, siding, paint, windows, doors, or vents.

12. The method of claim 1, wherein the processing, in real-time, each building image in the series of captured building images to determine if each building image meets a minimum criterion, occurs on any of:

a trained neural network, a deep learning module, an artificial intelligence module, or machine learning module.

13. The method of claim 1, wherein the receiving, from a computing device, a series of captured building images further includes a request to return one or more of:

the preliminary dimensions, refined preliminary dimensions, final dimensions, a completed multi-dimensional building model, a completed multi-dimensional building model with the preliminary dimensions, or a completed multi-dimensional building model with final dimensions.

14. The method of claim 1, wherein the identifying one or more architectural features of the one or more facades includes identifying one or more openings in the base set of building images.

15. A multi-dimensional modeling server comprises:

an interface;

a local memory; and

a processing circuit operably coupled to the interface and the local memory, wherein the processing circuit is configured to;

receive first communications from a remote computing device, wherein the first communications include a series of captured building images;

process, in real-time, each building image in the series of captured building images to determine if each building image meets a minimum criterion, wherein the minimum criterion includes applicability for constructing a multi-dimensional building model;

aggregate each of the building images meeting the minimum criterion;

determine when a base set of building images has been aggregated, wherein the base set of building images includes a threshold number images to construct at least a partial multi-dimensional building model representing the series of captured building images;

determine one or more facades present in the base set of building images;

determine one or more architectural features of the one or more facades;

determine preliminary dimensions for the one or more architectural features of the one or more facades; and

return second communications to the remote computing device, wherein the second communications include the preliminary dimensions for the one or more architectural features of the one or more facades.

16. The multi-dimensional modeling server of claim 15 further configured to incrementally refine the preliminary dimensions based on continual processing of additional ones of the building images in the series of captured building images.

17. The multi-dimensional modeling server of claim 15 further configured to include any of:

a neural network, a deep learning module, an artificial intelligence, or machine learning module, to compare each building image in the series of captured building images to a database of previously stored images to identify the one or more architectural features.

18. The multi-dimensional modeling server of claim 15 further configured to incrementally return the preliminary dimensions to the computing device until at least one complete set of one or more of:

preliminary dimensions or final dimensions, for the one or more architectural features, is returned.

19. The multi-dimensional modeling server of claim 15 configured to operate directly on an image capture device.

20. An image processing system comprises:

an interface;

a local memory; and

a processing circuit operably coupled to the interface and the local memory, wherein the processing circuit is configured to;

receive first communications, from a remote computing device, wherein the first communications include a series of captured building images;

determine when a base set of building images has been received, wherein the base set of building images includes a threshold number images to construct at least a partial multi-dimensional building model representing the series of captured building images;

determine one or more facades present in the partial multi-dimensional building model;

determine one or more architectural features of the one or more facades;

determine preliminary dimensions for one or more architectural features of the one or more facades; and

return second communications, in real-time, to the remote computing device, wherein the second communications include the preliminary dimensions for one or more architectural features of the one or more facades.