Automatic building assessment

• US 10,275,833 B1
• Filed: 05/20/2015
• Issued: 04/30/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Christopulos, Nicholas U., Maurer, Jonathan D., Tofte, Nathan L., Schmidgall, Roger D., Freeman, James M.
• Primary Examiner(s): Ebersman, Bruce I
• Application Number:
  US14/716,999
• Time in Patent Office: 1,441 Days
• Field of Search: 705 4
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/241 relating to the classification model, e.g. parametric or non-parametric approaches
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/243 relating to the number of classes
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06Q 50/16 Real estate
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G06V 20/39 Urban scenes
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

Disclosed systems and methods automatically assess buildings and structures. A device may receive one or more images of a structure, such as a building or portion of the building, and then label and extract relevant data. The device may then train a system to automatically assess other data describing similar buildings or structures based on the labeled and extracted data. After training, the device may then automatically assess new data, and the assessment results may be sent directly to a client or to an agent for review and/or processing.

Claims

1. A computer-implemented method for automatically assessing a building structure, the method comprising:

receiving, using one or more processors, a first set of captured data points describing a first plurality of building structures;

filtering, using one or more processors, the first set of captured data points by removing one or more data points from the first set of captured data points;

extracting, using one or more processors, feature data describing physical characteristics of one or more features of the first plurality of building structures from the filtered first set of captured data points, wherein the physical characteristics include information regarding three-dimensional shapes of a plurality of physical building material components of the one or more features for each of the first plurality of building structures, and wherein the physical characteristics correspond to one or more conditions of the one or more features;

training, using one or more processors, a building structure classification system based on the extracted feature data by classifying the extracted feature data according to feature vectors describing height, width, and thickness of the three-dimensional shapes of the plurality of physical building material components of the one or more features and determining a plurality of classification rules based upon the extracted feature data;

controlling, using one or more processors, a portable device configured to capture data describing features of building structures;

capturing, using one or more sensors of the controlled portable device, a second set of captured data describing a feature of a particular building structure, wherein the second set of captured data includes a plurality of separately captured sets of data relating to the feature of the particular building structure;

extracting, using one or more processors, a plurality of data points indicating a shape of one or more physical building material components of the feature from the second set of captured data;

determining, using one or more processors, a structural condition of the feature of the particular building structure by classifying the extracted data points by classifying the feature according to one or more of the plurality of classification rules based upon a combination of height, width, and thickness of the one or more physical building material components of the indicated shape of the one or more physical building components using the building structure classification system, wherein the combination of the height, width, and thickness of the one or more physical building material components represents at least one of chipping, cracking, denting, wearing, sagging, or misalignment of the one or more physical building material components of the feature, wherein the one or more of the plurality of classification rules evaluates at least one of the following;

age, type, construction, structural condition, or damage to building structures;

displaying, via a user interface screen, an image of the feature of the particular building structure; and

overlaying, via the user interface screen, a visual augmentation representing the structural condition of the feature over the image of the feature of the particular building structure.

2. The method of claim 1 further comprising:

displaying, via the user interface screen, additional information regarding the determined structural condition of the feature of the particular building structure.

3. The method of claim 1, wherein overlaying the visual augmentation representing the determined structural condition of the feature of the particular building structure is performed in real time as a user scans portions of an image of the building structure.

4. The method of claim 1, wherein the second set of captured data describing the feature of the particular building structure corresponds to a portion of an image of the building structure being reviewed by a user.

5. The method of claim 1, wherein the building structure is assessed as part of at least one of:

an insurance claim assessment, an insurance underwriting process, structural analysis, building code enforcement, estimation, appraisal and assessment, or remodeling.

6. The method of claim 1, wherein training the building structure classification system based on the extracted feature data further includes verifying the training of the building structure classification system using data points describing one or more additional building structures having known characteristics.

7. The method of claim 1, wherein:

the one or more features of the first plurality of building structures comprise roofs of the first plurality of building structures;

the feature of the particular building structure is a roof of the particular building structure;

the physical building material components are roofing tiles; and

determining the structural condition of the feature of the particular building structure includes determining damage to the roof of the particular building structure.

8. A computer system for automatically assessing a building structure, comprising:

one or more processors;

a portable device including one or more sensors configured to capture data describing features of building structures; and

a program memory coupled to the one or more processors and storing executable instructions that when executed by the one or more processors cause the computer system to;

receive a first set of captured data points describing a first plurality of building structures;

filter the first set of captured data points by removing one or more data points from the first set of captured data points;

extract feature data describing physical characteristics of one or more features of the first plurality of building structures from the filtered first set of captured data points, wherein the physical characteristics include information regarding three-dimensional shapes of a plurality of physical building material components of the one or more features for each of the first plurality of building structures, and wherein the physical characteristics correspond to one or more conditions of the one or more features;

train a building structure classification system based on the extracted feature data by classifying the extracted feature data according to feature vectors describing height, width, and thickness of the three-dimensional shapes of the plurality of physical building material components of the one or more features and determining a plurality of classification rules based upon the extracted feature data;

control the portable device;

capture a second set of captured data describing a feature of a particular building structure using the one or more sensors of the controlled portable device, wherein the second set of captured data includes a plurality of separately captured sets of data relating to the feature of the particular building structure;

extract a plurality of data points indicating a shape of one or more physical building material components of the feature from the second set of captured data;

determine a structural condition of the feature of the particular building structure by classifying the extracted data points by classifying the feature according to one or more of the plurality of classification rules based upon a combination of height, width, and thickness of the one or more physical building material components of the indicated shape of the one or more physical building components using the building structure classification system, wherein the combination of the height, width, and thickness of the one or more physical building material components represents at least one of chipping, cracking, denting, wearing, sagging, or misalignment of the one or more physical building material components of the feature, wherein the one or more of the plurality of classification rules evaluates at least one of the following;

age, type, construction, structural condition, or damage to building structures;

display, via a user interface screen, an image of the feature of the particular building structure; and

overlay, via the user interface screen, a visual augmentation representing the structural condition of the feature over the image of the feature of the particular building structure.

9. The computer system of claim 8, wherein the executable instructions further cause the computer system to:

display additional information regarding the determined structural condition of the feature of the particular building structure.

10. The computer system of claim 8, wherein the executable instructions that cause the computer system to overlay the visual augmentation representing the determined structural condition of the feature of the particular building structure in real time as a user scans portions of an image of the building structure.

11. A tangible, non-transitory computer-readable medium storing executable instructions for automatically assessing a building structure that, when executed by at least one processor of a computer system, cause the computer system to:

receive a first set of captured data points describing a first plurality of building structures;

filter the first set of captured data points by removing one or more data points from the first set of captured data points;

extract feature data describing physical characteristics of one or more features of the first plurality of building structures from the filtered first set of captured data points, wherein the physical characteristics include information regarding three-dimensional shapes of a plurality of physical building material components of the one or more features for each of the first plurality of building structures, and wherein the physical characteristics correspond to one or more conditions of the one or more features;

train a building structure classification system based on the extracted feature data by classifying the extracted feature data according to feature vectors describing height, width, and thickness of the three-dimensional shapes of the plurality of physical building material components of the one or more features and determining a plurality of classification rules based upon the extracted feature data;

control a portable device configured to capture data describing features of building structures;

capture a second set of captured data describing a feature of a particular building structure using one or more sensors of the controlled portable device, wherein the second set of captured data includes a plurality of separately captured sets of data relating to the feature of the particular building structure;

extract a plurality of data points indicating a shape of one or more physical building material components of the feature from the second set of captured data;

determine a structural condition of the feature of the particular building structure by classifying the extracted data points by classifying the feature according to one or more of the plurality of classification rules based upon a combination of height, width, and thickness of the one or more physical building material components of the indicated shape of the one or more physical building components using the building structure classification system, wherein the combination of the height, width, and thickness of the one or more physical building material components represents at least one of chipping, cracking, denting, wearing, sagging, or misalignment of the one or more physical building material components of the feature, wherein the one or more of the plurality of classification rules evaluates at least one of the following;

age, type, construction, structural condition, or damage to building structures;

display, via a user interface screen, an image of the feature of the particular building structure; and

overlay, via the user interface screen, a visual augmentation representing the structural condition of the feature over the image of the feature of the particular building structure.

12. The tangible, non-transitory computer-readable medium of claim 11, further comprising executable instructions that cause the computer system to:

display additional information regarding the determined structural condition of the feature of the particular building structure.

13. The tangible, non-transitory computer-readable medium of claim 11, wherein the executable instructions that cause the computer system to overlay the visual augmentation representing the determined structural condition of the feature of the particular building structure in real time as a user scans portions of an image of the building structure.