Real-time processing of captured building imagery

• US 10,410,412 B2
• Filed: 04/02/2018
• Issued: 09/10/2019
• Est. Priority Date: 05/29/2015
• Status: Active Grant

• Current Assignee: Hover Inc. (Tucows Incorporated)
• Original Assignee: Hover Inc. (Tucows Incorporated)
• Inventors: Upendran, Manish, Castillo, William, Mishra, Ajay, Altman, Adam J.
• Primary Examiner(s): Broome, Said
• Application Number:
  US15/942,733
• Publication Number:
  US 20180225869A1
• Time in Patent Office: 526 Days
• Field of Search: None
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection

Abstract

A process for receiving, from a computing device, a series of captured building images and processing, in real-time, each building image in the series of captured building images to determine if the building image meets a minimum threshold for quality. The process continues, for each building image meeting the minimum threshold for quality, by returning a first quality indication to the computing device and for each building image not meeting the minimum threshold for quality, by returning a second quality indication to the computing device. When a complete set of quality building images has been received, the process continues by creating at least a partial multi-dimensional building model based at least partially on the complete set of quality building images.

Claims

1. A method of processing building imagery, the method comprises:

receiving, from a computing device, a series of captured building images;

processing, in real-time, each building image in the series of captured building images to determine if the building image meets a minimum threshold for quality;

for each building image meeting the minimum threshold for quality, storing this building image in computer memory and returning a first quality indication to the computing device;

for each building image not meeting the minimum threshold for quality, discarding this building image and returning a second quality indication to the computing device;

determining when a complete set of quality building images has been received, wherein the complete set of quality building images includes a minimum grouping of images to construct at least a partial multi-dimensional building model representing the series of captured building images, where the at least a partial multi-dimensional building model includes surfaces shown in the minimum grouping of images; and

creating at least a partial multi-dimensional building model based at least partially on the complete set of quality building images.

2. The method of claim 1, wherein the processing includes comparing each captured building image in the series of captured building images to a database of previously stored quality images.

3. The method of claim 2, wherein the determining if the captured building image meets a minimum threshold for quality is based on any of:

a trained neural network-based quality image assessment method, a deep learning-based quality image identifying method, an artificial intelligence (AI) based quality image assessment method, or a machine learning-based quality image assessment method.

4. The method of claim 1, wherein a minimum threshold for quality includes any of:

predetermined acceptable camera-based image parameters, applicability in constructing a multi-dimensional model, or capturing instances of specific architectural features to be included in model construction.

5. The method of claim 1, wherein the minimum threshold for quality includes capturing at least a first corner and one or more sides of a corresponding building.

6. The method of claim 1, wherein the minimum threshold for quality includes capturing all instances of at least a first architectural feature for a corresponding building.

7. The method of claim 1, wherein the minimum threshold for quality includes capturing a specific perspective of a corresponding building.

8. The method of claim 1, wherein the first quality indication is one or more of:

good quality or best available quality.

9. The method of claim 1, wherein the second quality indication is one or more of:

bad quality or unacceptable quality.

10. The method of claim 1, wherein the first or second quality indication is provided as a grade between a low and high number ranking from bad to good.

11. A multi-dimensional modeling server comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

receive, from a computing device, a series of captured building images;

process, in real-time, each captured building image in the series of captured building images to determine if the captured building image meets a minimum threshold for quality;

for each of the captured building images meeting the minimum threshold for quality, store these captured building images in computer memory and return a first quality indication to the computing device;

for each of the captured building images not meeting the minimum threshold for quality, discard these captured building images and return a second quality indication to the computing device;

determine when a complete set of quality building images has been received, wherein the complete set of quality building images includes a minimum grouping of images to construct at least a partial multi-dimensional building model representing the series of captured building images, where the at least a partial multi-dimensional building model includes surfaces shown in the minimum grouping of images; and

create at least a partial multi-dimensional building model based at least partially on the complete set of quality building images.

12. The multi-dimensional modeling server of claim 11 further configured to compare each building image in the series of captured building images to a database of previously stored quality images.

13. The multi-dimensional modeling server of claim 12, wherein the determine if the captured building image meets a minimum threshold for quality is performed by any of a non-transitory computer readable medium encoded with any of:

a trained neural network, deep learning, artificial intelligence, or machine learning.

14. The multi-dimensional modeling server of claim 11, wherein the minimum threshold for quality includes at least a first corner and one or more sides of a corresponding building.

15. The multi-dimensional modeling server of claim 11, wherein the minimum threshold for quality includes a specific perspective of a corresponding building.

16. The multi-dimensional modeling server of claim 11, wherein the minimum threshold for quality includes a building image with a boundary inclusion of a corresponding building.

17. The multi-dimensional modeling server of claim 11, wherein the series of captured building images are received from a mobile camera.

18. The multi-dimensional modeling server of claim 11, where the multi-dimensional modeling server is configured to run directly on a mobile camera or image capture device.

19. An image processing system comprises:

an interface;

a local memory;

a machine learning module; and

a processing module operably coupled to the interface, the local memory and the machine learning module, wherein the processing module is configured to;

receive, from a computing device, a series of captured building images;

receive in real-time, from the machine learning module, an indication for each captured building image in the series of captured building images meeting a minimum threshold for quality;

for each of the captured building images meeting the minimum threshold for quality, store these captured building images in computer memory and return a first quality indication to the computing device;

for each of the captured building images not meeting the minimum threshold for quality, discard these captured building images and return a second quality indication to the computing device;

determine when a complete set of quality building images has been received, wherein the complete set of quality building images includes a minimum grouping of images to construct at least a partial multi-dimensional building model representing the series of captured building images, where the at least a partial multi-dimensional building model includes surfaces shown in the minimum grouping of images; and

create at least a partial multi-dimensional building model based at least partially on the complete set of quality building images.

20. The image processing system of claim 19, wherein the image processing system is configured to run directly on a mobile camera or image capture device.