AUTOMATIC ASSESSMENT OF DAMAGE AND REPAIR COSTS IN VEHICLES

• US 20170293894A1
• Filed: 04/06/2016
• Published: 10/12/2017
• Est. Priority Date: 04/06/2016
• Status: Active Grant

• Current Assignee: American International Group Incorporated
• Original Assignee: American International Group Incorporated
• Inventors: Taliwal, Vikas, Dalal, Siddhartha, Li, Kaigang, Sharma, Gaurav
• Granted Patent: US 10,692,050 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 2203/04802 3D-info-object: information is displayed on the internal or external surface of a three dimensional manipulable object, e.g. on the faces of a cube that can be rotated by the user
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 10/20 Administration of product repair or maintenance
• CPC Class Codes: G06Q 30/0283 Price estimation or determination
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30156 Vehicle coating
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 2201/08 : Detecting or categorising vehicles
• CPC Class Codes: H04W 4/02 : Services making use of location information

Abstract

A system and method are provided for automatically estimating a repair cost for a vehicle. A method includes: receiving, at a server computing device over an electronic network, one or more images of a damaged vehicle from a client computing device; performing image processing operations on each of the one or more images to detect external damage to a first set of parts of the vehicle; inferring internal damage to a second set of parts of the vehicle based on the detected external damage; and, calculating an estimated repair cost for the vehicle based on the detected external damage and inferred internal damage based on accessing a parts database that includes repair and labor costs for each part in the first and second sets of parts.

Claims

1. A method for automatically estimating a repair cost for a vehicle, comprising:

receiving, at a server computing device over an electronic network, one or more images of a damaged vehicle from a client computing device;

performing computerized image processing on each of the one or more images to detect damage to a set of parts of the vehicle; and

calculating an estimated repair cost for the vehicle based on the detected damage based on accessing a parts database that includes repair costs.

2. The method of claim 1, wherein said detecting damage includes estimating external damage to a first set of parts of the vehicle and inferring internal damage to a second set of parts based on detected external damage.

3. The method of claim 1, wherein said database or repair costs includes estimates for parts and labor for individual parts.

4. The method of claim 1, wherein performing image processing on each of the one or more images includes:

removing artifacts from a first image of the one or more images;

aligning the first image to an undamaged version of the vehicle;

segmenting the first image into vehicle parts; and

performing damage assessment by comparing portions of each vehicle part in the first image to corresponding portions of each vehicle part in the undamaged version of the vehicle.

5. The method of claim 4, wherein removing artifacts from the first image includes:

removing background material from the first image; and

removing specular reflection due to incident light on the vehicle shown in the first image.

6. The method of claim 4, wherein performing damage assessment includes:

comparing each of edge distribution, texture comparison, and spatial correlation of portions of each vehicle part in the first image to corresponding portions of each vehicle part in the undamaged version of the vehicle; and

determining whether the edge distribution, the texture comparison, and the spatial correlation exceed respective threshold difference values.

7. The method of claim 4, wherein damage is detected in a portion of a vehicle part in the first image if any one of the edge distribution, the texture comparison, and the spatial correlation exceeds the respective threshold difference value.

8. The method of claim 1, wherein performing image processing on each of the one or more images includes:

training a first Convolutional Neural Network (CNN) to detect a pose of a vehicle;

training a plurality of CNNs to detect damage to a respective plurality of external vehicle parts;

receiving a first image of the one or more images of the damaged vehicle;

executing the first CNN to detect the pose of the vehicle in the first image;

executing the plurality of CNNs to determine which external vehicle parts are damaged in the first image; and

executing a Markov Random Field (MRF) algorithm to infer damage to internal parts of the vehicle from the determined damaged external vehicle parts.

9. A mobile device comprising a camera, a display device, a processor, and a memory, the memory storing instructions that, when executed by the processor, cause the mobile device to display prompts on the display device to capture damage to a vehicle with the camera, by performing the steps of:

receiving, in a first user interface screen displayed on the display device, a selection to initiate a new vehicle claim;

displaying, in a second user interface screen displayed on the display device, graphical elements for selection of a prompting interface for capture of images of damage to the vehicle;

receiving selection of a graphical element corresponding to a prompting interface;

displaying one or more prompts on the display device to capture a portion of the vehicle based on the selection of the graphical element corresponding to the prompting interface;

causing the camera of the client device to capture an image of the vehicle based on displaying an outline of the portion of the vehicle; and

causing the image of the vehicle to be uploaded to a server for estimation of repair costs of the vehicle based on the image.

10. The method of claim 9, wherein the prompting interface comprises a prompting interface based on a three-dimensional (3D) model of the vehicle, and the method further comprises:

displaying a 3D model of the vehicle;

receiving a selection of a damaged part on the 3D model; and

displaying an outline of the selected part to be captured with the camera of the client device.

11. The method of claim 9, wherein the prompting interface comprises a prompting interface based on a parts list of the vehicle, and the method further comprises:

displaying a parts list;

receiving a selection of a part from the parts list; and

displaying an outline of the part to be captured with the camera of the client device.

12. The method of claim 9, wherein the prompting interface comprises a prompting interface based on vehicle views, and the method further comprises:

displaying two or more vehicle views; and

for each of the two or more vehicle views, displaying an outline of the view of the vehicle to be captured with the camera of the client device.

13. The method of claim 9, wherein the mobile device is operated by an owner or driver of the vehicle.

14. The method of claim 9, wherein the mobile device is operated by an insurance adjuster.

15. The method of claim 9, wherein the mobile device is operated by an employee of an auto repair shop.

16. A system for estimating a repair cost for a vehicle, comprising:

a client computing device configured to;

display one or more prompts on a display device of the client computing device to capture a portion of the vehicle that has sustained damage, and

capture an image of the vehicle based on displaying an outline of the portion of the vehicle;

an electronic communications network configured to transfer the image of the vehicle to a server computing device; and

the server computing device configured to;

receive the image over the electronic communications network,

perform image processing operations on the image to identify one or more damaged parts of the vehicle, and

calculate an estimated repair cost for the vehicle based on accessing a parts database that includes repair and labor costs for each part in the one or more damaged parts.

17. The system of claim 16, wherein identifying the one or more damaged parts of the vehicle comprises:

detecting external damage to a first set of parts of the vehicle; and

infer internal damage to a second set of parts of the vehicle based on the detected external damage.

18. The system of claim 16, wherein performing image processing on the image includes:

removing artifacts from the image;

aligning the image to an undamaged version of the vehicle;

segmenting the image into vehicle parts; and

performing damage assessment by comparing portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the vehicle.

19. The system of claim 18, wherein removing artifacts from the image includes:

removing background material from the image; and

removing specular reflection due to incident light on the vehicle shown in the image.

20. The system of claim 18, wherein performing damage assessment includes:

comparing each of edge distribution, texture comparison, and spatial correlation of portions of each vehicle part in the image to corresponding portions of each vehicle part in the undamaged version of the vehicle; and

determining whether the edge distribution, the texture comparison, and the spatial correlation exceed respective threshold difference values.

21. The system of claim 20, wherein damage is detected in a portion of a vehicle part in the image if any one of the edge distribution, the texture comparison, and the spatial correlation exceeds the respective threshold difference value.

22. The system of claim 16, wherein displaying the one or more prompts on the display device of the client computing device comprises:

displaying a 3D model of the vehicle;

receiving a selection of a damaged part on the 3D model; and

displaying an outline of the selected part to be captured with a camera of the client computing device.

23. The system of claim 16, wherein the client computing device comprises a mobile phone configured to execute a mobile application to capture the image of the vehicle and upload the image to the server computing device over the electronic communications network.

24. The system of claim 17, wherein detecting the external damage to the first set of parts of the vehicle is based on a comparing current damage to the vehicle to an undamaged version of the vehicle, or by comparing the current damage to the vehicle with a state of the vehicle prior to the current damage.