Methods and systems for capturing the condition of a physical structure via chemical detection

• US 9,131,224 B1
• Filed: 02/25/2015
• Issued: 09/08/2015
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Freeman, James M., Schmidgall, Roger D., Boyer, Patrick Harold, Christopulos, Nicholas U., Maurer, Jonathan D., Tofte, Nathan L., Jordan, Jackie O. II
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Du, Haixia
• Application Number:
  US14/631,558
• Time in Patent Office: 195 Days
• US Class Current: 1/1
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: G01N 21/64 Fluorescence; Phosphorescence
• CPC Class Codes: G01N 21/8851 Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002
• CPC Class Codes: G01N 22/02 Investigating the presence of flaws
• CPC Class Codes: G01N 2201/06113 Coherent sources; lasers
• CPC Class Codes: G01N 2201/10 Scanning
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G06F 30/13 Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
• CPC Class Codes: G06Q 30/0278 Product appraisal
• CPC Class Codes: G06Q 30/0283 Price estimation or determination
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06Q 50/16 Real estate
• CPC Class Codes: G06Q 50/163 Real estate management
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 : Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 7/0002 : Inspection of images, e.g. flaw detection
• CPC Class Codes: H04N 13/106 : Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/275 : from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control
• CPC Class Codes: H04R 23/008 : using optical signals for detecting or generating sound

Abstract

In a computer-implemented method and system for capturing the condition of a structure, the structure is scanned with a three-dimensional (3D) scanner. The 3D scanner generates 3D data. A point cloud or 3D model is constructed from the 3D data. The point cloud or 3D model is then analyzed to determine the condition of the structure.

Claims

1. A computer-implemented method of inspecting a structure, the method comprising:

spraying a chemical on at least part of a structure;

deploying one or more three-dimensional (3D) scanners to scan an area of the structure;

generating, via an image sensor of the one or more 3D scanners, image data based on one or more electromagnetic (EM) waves that are emanating from the scanned area of the structure and that are received at the image sensor;

analyzing the generated image data to detect luminescence for each of a plurality of subareas of the scanned area of the structure, at least some of the detected luminescence emanating from the chemical sprayed on the at least part of the structure;

analyzing the detected luminescence for the plurality of subareas to identify a first subarea from the plurality of subareas having a topographical feature corresponding to potential damage;

causing the one or more 3D scanners to scan the first subarea having the topographical feature corresponding to potential damage;

generating, by one or more processors, a plurality of 3D data points corresponding to the scanned first subarea, wherein each of the plurality of 3D data points has a coordinate set relating to vertical, horizontal, and depth distance measurements;

storing a point cloud at one or more memory devices, wherein the point cloud comprises one or more of the generated plurality of 3D data points;

automatically analyzing, by the one or more processors, the stored point cloud to identify one or more features of the scanned first subarea, wherein the one or more identified features represent potential damage;

generating, by the one or more processors, an estimation of the condition of the scanned first subarea based on the one or more features identified from the analysis of the stored point cloud; and

storing the generated estimation at the one or more memory devices.

2. The method of claim 1, wherein spraying a chemical on at least part of a structure comprises spraying a phosphorescent chemical.

3. The method of claim 1, wherein analyzing the detected luminescence for the plurality of subareas to identify a first subarea from the plurality of subareas having a topographical feature corresponding to potential damage comprises:

analyzing a degree of luminescence for at least the first subarea.

4. The method of claim 1, wherein spraying the chemical on at least part of the structure comprises:

spraying a chemical on a roof of the structure.

5. The method of claim 4, wherein analyzing the generated image data to detect luminescence for each of the plurality of subareas of the scanned area of the structure comprises:

analyzing the generated image data to detect luminescence for each of a plurality of shingles on the roof.

6. The method of claim 1, wherein automatically analyzing, by the one or more processors, the stored point cloud to identify the one or more features of the scanned first subarea comprises:

automatically analyzing, by the one or more processors, the stored point cloud to identify a crack or a crevice.

7. The method of claim 1, further including causing the one or more processors to calculate a financial cost estimate based on said estimation of the condition of the scanned first subarea.

8. The method of claim 1, wherein automatically analyzing, by the one or more processors, the stored point cloud to identify one or more features of the scanned first subarea comprises:

constructing, by the one or more processors, a 3D model from the stored point cloud;

storing the constructed 3D model at the one or more memory devices; and

analyzing, by the one or more processors, the 3D model to identify the one or more identified features.

9. The method of claim 8, wherein generating, by the one or more processors, an estimation of the condition of the scanned first subarea based on the one or more identified features comprises:

retrieving a second model from the one or more memory devices; and

comparing the 3D model to the second model.

10. The method of claim 1, wherein generating, via the image sensor of the one or more 3D scanners, the image data based on the one or more electromagnetic (EM) waves that are emanating from the scanned area of the structure and that are received at the image sensor comprises:

generating the image data based on one or more infrared waves that are received at the image sensor;

generating the image data based on one or more ultraviolet waves that are received at the image sensor;

or

some combination thereof.

11. The method of claim 1, further including projecting, from the one or more 3D scanners, the one or more EM waves;

wherein the one or more EM waves are subsequently reflected from the scanned area of the structure and received by the one or more 3D scanners.

12. The method of claim 1, wherein the one or more 3D scanners are physically connected to a device for conveying the one or more 3D scanners, the device being an aerial device or a non-aerial device;

and

wherein causing the one or more 3D scanners to scan the first subarea having the concentration of the chemical exceeding the threshold comprises;

causing the device to move to a position where the first subarea can be scanned by the one or more 3D scanners.

13. The method of claim 12, wherein the device includes an antenna for wirelessly receiving control commands.

14. A property inspection system, the property inspection system comprising:

one or more three-dimensional (3D) scanners for scanning an area of a structure, the one or more 3D scanners including an image sensor;

one or more processors communicatively connected to the one or more 3D scanners; and

one or more memory devices communicatively connected to the one or more processors, the one or more memory devices including instructions that, when executed, cause the one or more processors to;

analyze image data generated by the image sensor of the one or more 3D scanners to detect luminescence for each of a plurality of subareas of the scanned area of the structure, at least some of the detected luminescence emanating from a chemical sprayed on the at least part of the structure, wherein the image sensor generated the image data based on one or more electromagnetic (EM) waves that emanated from the scanned area of the structure and that were received at the image sensor;

analyze the detected luminescence for the plurality of subareas to identify a first subarea from the plurality of subareas having a topographical feature corresponding to potential damage;

cause the one or more 3D scanners to scan the first subarea having the topographical feature corresponding to potential damage;

generate a plurality of 3D data points corresponding to the scanned first subarea,

wherein each of the plurality of 3D data points has a coordinate set relating to vertical, horizontal, and depth distance measurements;

store a point cloud at the one or more memory devices, wherein the point cloud comprises one or more of the generated plurality of 3D data points;

automatically analyze the stored point cloud to identify one or more features of the scanned first subarea, wherein the one or more identified features represent potential damage;

generate an estimation of the condition of the scanned first subarea based on the one or more features identified from the analysis of the stored point cloud; and

store the generated estimation at the one or more memory devices.

15. The property inspection system of claim 14, further comprising a chemical spraying apparatus for spraying the chemical on the structure.

16. The property inspection system of claim 14, wherein the one or more 3D scanners are physically connected to a device for conveying the one or more 3D scanners, the device being an aerial device or a non-aerial device;

and

wherein the instructions to cause the one or more 3D scanners to scan the first subarea having the topographical feature corresponding to potential damage comprise;

instructions to cause the device to move to a position where the first subarea can be scanned by the one or more 3D scanners.

17. The property inspection system of claim 14, wherein the device includes a control system for autonomous operation.