Audio-based 3D point cloud generation and analysis

• US 10,242,497 B2
• Filed: 03/26/2018
• Issued: 03/26/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Freeman, James M., Schmidgall, Roger D., Boyer, Patrick H., Christopulos, Nicholas U., Maurer, Jonathan D., Tofte, Nathan L., Jordan, II, Jackie O.
• Primary Examiner(s): Du, Haixia
• Application Number:
  US15/935,989
• Publication Number:
  US 20180218537A1
• Time in Patent Office: 365 Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: G01N 21/64 Fluorescence; Phosphorescence
• CPC Class Codes: G01N 21/8851 Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002
• CPC Class Codes: G01N 22/02 Investigating the presence of flaws
• CPC Class Codes: G01N 2201/06113 Coherent sources; lasers
• CPC Class Codes: G01N 2201/10 Scanning
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 15/89 for mapping or imaging
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4817 relating to scanning
• CPC Class Codes: G06F 30/13 Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
• CPC Class Codes: G06Q 30/0278 Product appraisal
• CPC Class Codes: G06Q 30/0283 Price estimation or determination
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06Q 50/16 Real estate
• CPC Class Codes: G06Q 50/163 Real estate management
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 : Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 7/0002 : Inspection of images, e.g. flaw detection
• CPC Class Codes: H04N 13/106 : Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 13/275 : from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 7/185 : from a mobile camera, e.g. for remote control
• CPC Class Codes: H04R 23/008 : using optical signals for detecting or generating sound

Abstract

In a system and method for inspecting a property, a microphone receives one or more audio waves propagating from a structure. One or more processors generate a 3D point cloud based on the received audio waves, analyzed the generated 3D point cloud to identify features of a surface or subsurface of the structure, and generate an estimate of a condition of the surface or subsurface.

Claims

1. A property inspection system comprising:

a microphone configured to receive one or more audio waves propagating from at least part of a structure, wherein the microphone is a laser microphone;

one or more processors communicatively connected to the microphone;

one or more memories communicatively connected to the one or more processors, the one or more memories storing computer readable instructions that, when executed, cause the one or more processors to;

(i) generate a three-dimensional (3D) point cloud based upon the one or more audio waves received at the microphone;

(ii) analyze the generated 3D point cloud to identify one or more features of the structure;

(iii) generate an estimate of a condition of the structure based upon the identified one or more features; and

(iv) store the generated estimate at the one or more memories.

2. The property inspection system of claim 1, further including an aerial device attached to the microphone, the aerial device configured to convey the microphone.

3. The property inspection system of claim 2, wherein the aerial device includes a control system configured to move the aerial device.

4. The property inspection system of claim 3, wherein the control system is configured to autonomously move the aerial device.

5. The property inspection system of claim 3, wherein the control system is configured to move the aerial device in response to wirelessly receiving a command from a controller manipulated by a user.

6. The property inspection system of claim 1, further including a non-aerial device attached to the microphone, the non-aerial device configured to convey the microphone.

7. The property inspection system of claim 1, further comprising:

a mechanical feeler to physically manipulate a surface of the structure, wherein the one or more audio waves represent an audio response to the physical manipulation.

8. The property inspection system of claim 1, further comprising:

an audio projector communicatively connected to the one or more processors and configured to project the one or more audio waves so that the one or more audio waves reflect off a surface or a subsurface of the structure before being received at the microphone.

9. The property inspection system of claim 1, wherein the one or more identified features include one or more of the following:

indents, holes, cracks, and crevices.

10. The property inspection system of claim 1, wherein the processors analyze the generated 3D point cloud to identify the one or more features by:

retrieving a second 3D point cloud from the one or more memories; and

comparing the 3D point cloud to the second 3D point cloud.

11. The property inspection system of claim 1 wherein the instructions, when executed, further cause the one or more processors to:

generate a model surface utilizing the 3D point cloud and a surface reconstruction algorithm;

perform an analysis of the generated model surface; and

identify the one or more features of a surface or subsurface of the structure based, at least in part, on the analysis of the generated model surface.

12. A computer-implemented method of inspecting a structure, the method comprising:

physically manipulating a surface of the structure;

receiving, at a microphone, one or more audio waves propagating from at least part of a structure, wherein the one or more audio waves represent an audio response to the physical manipulation;

generating, by one or more processors communicatively connected to the microphone, a three-dimensional (3D) point cloud of the structure based upon the received one or more audio waves;

analyzing, by the one or more processors, the generated 3D point cloud to identify one or more features of the structure;

generating, by the one or more processors, an estimate of a condition of the structure based upon the identified one or more features; and

storing the generated estimate at one or more memories.

13. The computer-implemented method of claim 12, further comprising moving the microphone to a position where the one or more audio waves are received, wherein moving the microphone includes:

moving an aerial device conveying the microphone, the aerial device including a control system for moving the aerial device.

14. The computer-implemented method of claim 13, wherein moving the aerial device comprises causing a control system to autonomously move the aerial device.

15. The computer-implemented method of claim 13, wherein moving the aerial device comprises causing the control system to move the aerial device in response to wirelessly receiving a command from a controller manipulated by a user.

16. The computer-implemented method of claim 12, further comprising projecting the one or more audio waves so that the one or more audio waves reflect off a surface or a subsurface of the structure before being received at the microphone.

17. The computer-implemented method of claim 12, wherein analyzing the generated 3D point cloud to identify the one or more features comprises:

retrieving a second 3D point cloud from the one or more memories; and

comparing the 3D point cloud to the second 3D point cloud.

18. The computer-implemented method of claim 12, further comprising:

generating a model surface utilizing the 3D point cloud and a surface reconstruction algorithm;

analyzing the generated model surface;

wherein the one or more features of a surface or a subsurface of the structure are identified based, at least in part, on said analyzing.

19. A property inspection system comprising:

a microphone configured to receive one or more audio waves propagating from at least part of a structure;

a mechanical feeler to physically manipulate a surface of the structure, wherein the one or more audio waves represent an audio response to the physical manipulation;

one or more processors communicatively connected to the microphone;

one or more memories communicatively connected to the one or more processors, the one or more memories storing computer readable instructions that, when executed, cause the one or more processors to;

(i) generate a three-dimensional (3D) point cloud based upon the one or more audio waves received at the microphone;

(ii) analyze the generated 3D point cloud to identify one or more features of the structure;

(iii) generate an estimate of a condition of the structure based upon the identified one or more features; and

(iv) store the generated estimate at the one or more memories.