Laser-based methods and systems for capturing the condition of a physical structure

• US 9,336,552 B1
• Filed: 02/25/2015
• Issued: 05/10/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Freeman, James M., Schmidgall, Roger D., Boyer, Patrick Harold, Christopulos, Nicholas U., Maurer, Jonathan D., Tofte, Nathan L., Jordan, Jackie O. II
• Primary Examiner(s): Du, Haixia
• Application Number:
  US14/631,568
• Time in Patent Office: 440 Days
• US Class Current: 1/1
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 2101/32 for cartography or topography
• CPC Class Codes: G01N 21/64 Fluorescence; Phosphorescence
• CPC Class Codes: G01N 21/8851 Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002
• CPC Class Codes: G01N 22/02 Investigating the presence of flaws
• CPC Class Codes: G01N 2201/06113 Coherent sources; lasers
• CPC Class Codes: G01N 2201/10 Scanning
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G06F 30/13 Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
• CPC Class Codes: G06Q 30/0278 Product appraisal
• CPC Class Codes: G06Q 30/0283 Price estimation or determination
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06Q 50/16 Real estate
• CPC Class Codes: G06Q 50/163 Real estate management
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2200/08 : involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 : Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 7/0002 : Inspection of images, e.g. flaw detection
• CPC Class Codes: H04N 13/106 : Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/275 : from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control
• CPC Class Codes: H04R 23/008 : using optical signals for detecting or generating sound

Abstract

In a computer-implemented method and system for capturing the condition of a structure, the structure is scanned with a three-dimensional (3D) scanner. The 3D scanner generates 3D data. A point cloud or 3D model is constructed from the 3D data. The point cloud or 3D model is then analyzed to determine the condition of the structure.

Claims

1. A computer-implemented method of inspecting a structure, the method comprising:

deploying a three-dimensional (3D) data collection system that includes one or more 3D scanners conveyed by a balloon, the 3D data collection system physically connected to a tether;

transmitting, by a laser projector of the 3D scanners, a laser pulse directed toward a point on a surface of a structure;

receiving, at a laser light sensor of the one or more 3D scanners, the laser pulse, wherein the laser pulse has reflected off of the point on the surface of the structure;

calculating, by one or more processors, a time-of-flight measurement by measuring a difference between a first time at which the laser pulse was transmitted and a second time at which the laser pulse was received;

calculating, by the one or more processors, a distance measurement based on the time-of-flight measurement;

based on the calculated distance measurement;

generating a 3D data point corresponding to the point on the surface of the structure, the 3D data point having a coordinate set relating to vertical, horizontal, and depth distance measurements;

storing a point cloud at one or more memories, wherein the point cloud includes the generated 3D data point;

automatically analyzing, by the one or more processors, the point cloud to identify one or more features of the surface of the structure, wherein the one or more features represent potential damage;

generating, by the one or more processors, an estimation of a condition of the surface of the structure based on the identified one or more features; and

storing the generated estimation at the one or more memories.

2. The method of claim 1, wherein the identified one or more features are indents or holes.

3. The method of claim 1, wherein the identified one or more features are cracks or crevices.

4. The method of claim 1, further including causing the processor to calculate a financial cost estimate based on said estimation of the condition of the surface of the structure.

5. The method of claim 1, wherein automatically analyzing, by the one or more processors, the point cloud to identify one or more features of the scanned subareas comprises:

constructing, by the one or more processors, a 3D model from the point cloud;

storing the constructed 3D model at the one or more memories; and

analyzing, by the one or more processors, the 3D model to identify the one or more identified features.

6. The method of claim 5, wherein generating an estimation comprises:

retrieving a second model from the one or more memories; and

comparing the 3D model to the second model.

7. The method of claim 1, further comprising:

repositioning the laser projector of the one or more 3D scanners;

transmitting, by the laser projector of the one or more 3D scanners, a second laser pulse directed toward a second point on a surface of the structure;

receiving, at the laser light sensor of the one or more 3D scanners, the second laser pulse after the second laser pulse has reflected off of the second point on the surface of the structure;

obtaining, by the one or more processors, a second time-of-flight measurement by measuring a difference between a first time at which the second laser pulse was transmitted and a second time at which the second laser pulse was received;

calculating, by the one or more processors, a second distance measurement based on the second time-of-flight measurement; and

based on the calculated second distance measurement and based on the repositioning of the laser projector of the one or more 3D scanners;

generating a second 3D data point corresponding to the second point on the surface of the structure, the second 3D data point having a coordinate set relating to vertical, horizontal, and depth distance measurements;

wherein the point cloud further includes the generated second 3D data point.

8. The method of claim 7, wherein repositioning the laser projector of the one or more 3D scanners comprises:

moving the balloon.

9. The method of claim 8, wherein the balloon includes a control system for autonomous operation, and wherein moving the balloon comprises:

the control system causing the balloon to move.

10. The method of claim 9, wherein the balloon includes an antenna for wirelessly receiving control commands, and wherein moving the balloon comprises:

transmitting for reception at the antenna a wireless signal to cause the balloon to move.

11. The method of claim 10, wherein the tether is physically connected to a base station.

12. The method of claim 11, wherein the tether provides a communication channel between the 3D data collection system and the base station.

13. A property inspection system, the property inspection system comprising:

a 3D data collection system including a balloon and one or more 3D scanners affixed to the balloon for conveyance, the one or more 3D scanners including a laser projector and a laser light sensor, the 3D data collection system physically connected to a tether;

one or more processors communicatively connected to the one or more 3D scanners; and

one or more memory devices communicatively connected to the one or more processors, the one or more memory devices including instructions that, when executed, cause the one or more processors to;

calculate a time-of-flight measurement by measuring a difference between;

(i) a first time at which a laser pulse was transmitted by the laser projector of the one or more 3D scanners toward a point on the surface of the structure, and (ii) a second time at which the laser pulse was received at the laser light sensor;

calculate a distance measurement based on the time-of-flight measurement;

based on the calculated distance measurement;

generate a 3D data point corresponding to the point on the surface of the structure, the 3D data point having a coordinate set relating to vertical, horizontal, and depth distance measurements;

store a point cloud at one or more memories, wherein the point cloud includes the generated 3D data point;

automatically analyze the point cloud to identify one or more features of the surface of the structure, wherein the one or more features represent potential damage;

generate an estimation of a condition of the surface of the structure based on the identified one or more features; and

storing the generated estimation at the one or more memories.

14. The property inspection system of claim 13, wherein the balloon includes a control system for autonomous operation, and wherein the control system is configured to cause the balloon to move.

15. The property inspection system of claim 13, wherein the balloon includes an antenna for wirelessly receiving control commands to cause the balloon to move.

16. The property inspection system of claim 13, wherein the tether is physically connected to a base station.

17. The property inspection system of claim 16, wherein the tether provides a communication channel between the 3D data collection system and the base station.