Image capturing and automatic labeling system

• US 10,319,035 B2
• Filed: 10/11/2013
• Issued: 06/11/2019
• Est. Priority Date: 10/11/2013
• Status: Active Grant

• Current Assignee: CCC Intelligent Solutions, Inc.
• Original Assignee: CCC Information Services, Inc.
• Inventors: Nelson, Ronald, Fitzgerald, Sean
• Primary Examiner(s): Navas, Jr., Edemio
• Assistant Examiner(s): Adrovel, William
• Application Number:
  US14/052,629
• Publication Number:
  US 20150103170A1
• Time in Patent Office: 2,069 Days
• Field of Search: None
• CPC Class Codes: G06F 40/51 Translation evaluation
• CPC Class Codes: G06Q 10/20 Administration of product repair or maintenance
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: H04N 1/00209 Transmitting or receiving image data, e.g. facsimile data, via a computer, e.g. using e-mail, a computer network, the internet, I-fax
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3205 of identification information, e.g. name or ID code H04N2201/3209 takes precedence
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A vehicle image capture and labeling system operates to enable a user to capture vehicle photos, pictures, and/or images that are automatically labeled, i.e., annotated. In particular, the captured vehicle photos, pictures, or images are automatically labeled with certain vehicle identifier information, like a vehicle identification number (VIN), and pose information, that identifies a portion or view of the vehicle depicted within the image of the vehicle. The vehicle images may also be automatically labeled with one or more other image attributes or indicia, such as geospatial information corresponding to a location at which the photo or image was captured (e.g., global positioning system (GPS) data), time and date of image capture data, etc. The captured vehicle image and its label(s) may then be stored and used by other applications such as vehicle insurance claim applications, automobile repair estimate applications, etc.

Claims

1. A method of recording damage to a vehicle, the method comprising:

presenting, at a user interface of a mobile computing device, a plurality of user controls, wherein each user control of the plurality of user controls indicates a respective portion of the vehicle and includes a pre-assigned identifier that (a) is expressed in a non-natural language, the non-natural language being a computer-readable language having a particular syntax and format, (b) identifies the respective portion of the vehicle, and (c) is not presented on the user interface, and the plurality of user controls is a specific template of a plurality of templates of user controls, each template of the plurality of templates defined to meet a different set of jurisdictional requirements;

receiving, at the user interface of the mobile computing device, a selection of a particular user control included in the plurality of user controls, the selected, particular user control indicative of a particular portion of the vehicle, the particular portion of the vehicle excluding a Vehicle Identification Number (VIN) or other unique identifier of the vehicle;

launching an image capturing system of the mobile computing device;

obtaining a first image of the Vehicle Identification Number (VIN) or other unique identifier of the vehicle, the first image captured by the image capturing system;

image processing the first image to identify the VIN or other unique identifier of the vehicle;

obtaining a second image of the particular portion of the vehicle indicated by the selected, particular user control, the second image captured by the image capturing system; and

causing the captured second image and metadata indicative of attributes corresponding to the captured second image to be transmitted to a data storage system that is configured to store information corresponding to vehicle damage, the attributes corresponding to the captured second image including (i) the pre-assigned identifier corresponding to the particular user control and to the particular portion of the vehicle, and (ii) the identified VIN or other unique identifier of the vehicle, thereby labeling the captured second image of the particular portion of the vehicle with the pre-assigned identifier corresponding to the particular portion of the vehicle and with the identified VIN or other unique identifier of the vehicle.

2. The method of claim 1, wherein:

the method further comprises obtaining the attributes corresponding to the captured second image; and

the attributes further include at least one of;

a time of capture of the second image, a date of capture of the second image, or a geospatial location at which the second image was captured.

3. The method of claim 1, wherein presenting, at the user interface, the plurality of user controls comprises presenting, at the user interface, at least one natural language translation of at least one pre-assigned identifier of at least one user control.

4. The method of claim 3, wherein the at least one respective natural language translation includes a particular respective natural language translation of the pre-assigned identifier corresponding to the particular user control, and wherein the method further comprises displaying, on the user interface, an indication of the captured second image associated with the particular natural language translation.

5. The method of claim 3, further comprising receiving a definition of at least one of the at least one natural language translation via the user interface of the mobile computing device.

6. The method of claim 5, further comprising transmitting the definition of the at least one of the at least one natural language translation to the data storage system for storage.

7. The method of claim 1, wherein presenting the plurality of user controls corresponding to the plurality of respective portions of the vehicle comprises presenting a specific user control corresponding to a portion of the vehicle displaying the vehicle identification number (VIN) or other unique identifier of the vehicle.

8. The method of claim 7, wherein presenting the plurality of user controls corresponding to the plurality of respective portions of the vehicle further comprises presenting at least one other user control respectively corresponding to at least one other portion of the vehicle, the at least one other portion of the vehicle including at least one of:

a left front area of the vehicle;

a center front area of the vehicle;

a right front area of the vehicle;

a right fender of the vehicle;

a right side or T-bone area of the vehicle;

a right rear quarter post of the vehicle;

a right rear area of the vehicle;

a center rear area of the vehicle;

a left rear area of the vehicle;

a left rear quarter post of the vehicle;

a left side or T-bone area of the vehicle;

a left fender of the vehicle;

a hood of the vehicle;

a roof of the vehicle;

a door plate mounted on the vehicle;

an odometer of the vehicle;

a dash plate mounted on the vehicle;

a license plate of the vehicle;

another exterior portion of the vehicle;

or

at least a portion of an interior of the vehicle.

9. The method of claim 1, wherein:

receiving the selection of the particular user control included in the plurality of user controls comprises receiving a selection of two or more particular user controls included in the plurality of user controls;

obtaining the second image comprises obtaining two or more images captured by the image capturing system and respectively corresponding to the two or more particular user controls; and

causing the captured second image and the metadata indicative of the attributes corresponding to the captured second image to be transmitted to the data storage system comprises causing the two or more captured images and respective metadata indicative of respective attributes corresponding to each of the two or more captured images to be transmitted to the data storage system.

10. The method of claim 1, wherein receiving, at the user interface of the mobile computing device, the selection of the particular user control included in the plurality of user controls includes receiving a voice command and, using a processor, decoding the voice command as being associated with a particular user control to determine the selection of the particular user control.

11. The method of claim 1, wherein the method is at least partially performed by a processor of the mobile computing device executing computer-executable instructions stored on a tangible, non-transitory computer-readable storage medium of the mobile computing device, and the method further comprises:

receiving the computer-executable instructions via a communication interface of the mobile computing device; and

storing the computer-executable instructions on the tangible, non-transitory computer-readable storage medium of the mobile computing device.

12. The method of claim 1, wherein causing the captured second image and the metadata indicative of the attributes corresponding to the captured second image to be transmitted to the data storage system comprises causing the captured second image and the metadata indicative of the attributes corresponding to the captured second image to be transmitted to the data storage system using a wireless communication interface of the mobile computing device.

13. The method of claim 1, further comprising obtaining the plurality of pre-assigned identifiers via a communication interface of the mobile computing device, wherein the plurality of pre-assigned identifiers is provided by the data storage system, and wherein the communication interface of the mobile computing device is connected, via a communication channel, to the data storage system or to another computing device.

14. The method of claim 1, further comprising receiving a user indication specifying a least a portion of the plurality of user controls that are to be presented for selection at the user interface.

15. The method of claim 1, further comprising generating an estimate of a repair cost of the vehicle based on the captured second image and the attributes corresponding to the captured second image.

16. The method of claim 1, wherein each template corresponds to a respective make of vehicles, a respective model of vehicles, and/or a respective year.

17. The method of claim 1, wherein each template is further defined to meet a respective set of requirements governed by a respective application that operates on labeled images.

18. The method of claim 1, wherein each template is further defined to meet a respective set of situational requirements.

19. A system for recording damage to a vehicle, the system comprising:

an image attribute capturer, the image attribute capturer comprising one or more tangible, non-transitory memories storing computer-executable instructions that, when executed by one or more processors, cause the system to;

present, at a user interface, a plurality of user controls, wherein each user control of the plurality of user controls indicates a respective portion of the vehicle and includes a pre-assigned identifier that (a) is expressed in a non-natural language, the non-natural language being a computer-readable language having a particular syntax and format, (b) identifies the respective portion of the vehicle, and (c) is not presented on the user interface, and the plurality of user controls is a specific template of a plurality of templates of user controls, each template of the plurality of templates defined to meet a different set of jurisdictional requirements;

receive a first captured image of a Vehicle Identification Number (VIN) or other unique identifier of the vehicle;

image-process the first captured image to determine the VIN or other unique identifier of the vehicle;

receive, at the user interface, a selection of a particular user control included in the plurality of user controls, the selected, particular user control indicative of a particular portion of the vehicle from which the VIN or other unique identifier of the vehicle is excluded;

receive a second captured image of the particular portion of the vehicle corresponding to the selected, particular user control; and

cause the second captured image and metadata indicative of attributes corresponding to the second captured image to be transmitted to a data storage system that is configured to store information corresponding to vehicle damage, the attributes corresponding to the second captured image including (i) the pre-assigned identifier corresponding to the particular user control and the particular portion of the vehicle, and (ii) the identified VIN or other unique identifier of the vehicle, thereby labeling the captured second image of the particular portion of the vehicle with the pre-assigned identifier corresponding to the particular portion of the vehicle and with the identified VIN or other unique identifier of the vehicle.

20. The system of claim 19, further comprising a communicative connection to an imaging apparatus, and wherein the first captured image and the second captured image are captured by the imaging apparatus.

21. The system of claim 19, wherein the image attribute capturer includes further computer-executable instructions stored on the one or more tangible, non-transitory memories that, when executed by the one or more processors, cause the system to:

obtain one or more other attributes corresponding to the second captured image, the one or more other attributes including at least one of;

an indication of the time of capture of the second image, a date of capture of the second captured image, or an indication of a geospatial location at which the second captured image was captured; and

cause metadata indicative of the one or more other attributes corresponding to the second captured image to be transmitted to the data storage system in conjunction with the second captured image.

22. The system of claim 21, further comprising means for obtaining the one or more other attributes, wherein the means for obtaining the one or more other attributes includes at least one of:

a communicative connection to a Global Positioning Satellite System (GPS) device, or a communicative connection to a clock.

23. The system of claim 19, wherein the computer-executable instructions are downloaded from the data storage system via the communications interface for storage in the one or more tangible, non-transitory memories, and the plurality of pre-assigned identifiers is provided by the data storage system.

24. The system of claim 19, wherein each pre-assigned identifier comprises metadata identifying the respective portion of the vehicle.

25. The system of claim 19, wherein the data storage system is in communicative connection with at least one other system, the at least one other system is configured to perform a function, and the function includes at least one of:

generate vehicle damage appraisals, generate insurance claims for vehicles, generate vehicle repair estimates, or store information corresponding to vehicle repairs.

26. The system of claim 25, wherein the function performed by the at least one other system is based on a set of rules, and wherein at least a portion of the set of rules is based on at least one of the second captured image or the attributes corresponding to the second captured image.

27. The system of claim 19, wherein the computer-executable instructions, when executed by the one or more processors, further cause the system to generate at least one of a vehicle damage appraisal, an estimate of a cost to repair the vehicle, or an insurance claim corresponding to the vehicle based on the second captured image and the attributes corresponding to the second captured image.

28. The system of claim 19, wherein the plurality of respective portions of the vehicle includes more than one of:

a particular portion of the vehicle on which the vehicle identification number (VIN) or other unique identifier of the vehicle is displayed;

a left front area of the vehicle;

a center front area of the vehicle;

a right front area of the vehicle;

a right fender of the vehicle;

a right side or T-bone area of the vehicle;

a right rear quarter post of the vehicle;

a right rear area of the vehicle;

a center rear area of the vehicle;

a left rear area of the vehicle;

a left rear quarter post of the vehicle;

a left side or T-bone area of the vehicle;

a left fender of the vehicle;

a hood of the vehicle;

a roof of the vehicle;

a door plate mounted on the vehicle;

an odometer of the vehicle;

a dash plate mounted on the vehicle;

a license plate of the vehicle;

another exterior portion of the vehicle;

or

at least a portion of an interior of the vehicle.

29. The system of claim 19, wherein at least one of the plurality of user controls presented on the user interface is selected, by a user of the system, to be presented on the user interface.

30. The system of claim 19, wherein the second captured image is indicated, at the user interface, as corresponding to a natural language translation of the pre-assigned identifier corresponding to the particular user control.

31. The system of claim 19, wherein the image attribute capturer, the user interface, and the one or more processors are included in a mobile communications device, and wherein the second captured image and the metadata indicative of the attributes are caused to be transmitted to the data storage system via a wireless communications interface of the mobile communications device.

32. The system of claim 19, wherein at least one user control of the plurality of user controls includes a respective natural language translation of the pre-assigned identifier corresponding to the at least one user control.

33. The system of claim 32, wherein the at least one respective natural language translation comprises a plurality of respective natural language translations;

and

wherein at least one of the plurality of respective natural language translations is defined via the user interface and at least another one of the plurality of respective natural language translations is provided by the data storage system.

34. The system of claim 19, wherein the image attribute capturer when executed by one or more processors, causes the system to receive, at the user interface, the selection of the particular user control included in the plurality of user controls by receiving a voice command and, using the one or more processors, decoding the voice command as being associated with the particular user control to determine the selection of the particular user control.

35. The system of claim 19, wherein at least one of:

each template corresponds to a respective make of vehicles, a respective model of vehicles, and/or a respective year of vehicles;

or

each template is further defined to meet a respective set of situational requirements.

36. A method of creating an electronic image of a subject, the method comprising:

presenting, via a user interface of a mobile computing device having a computer processor, a plurality of user controls, wherein each user control of the plurality of user controls corresponds to a respective portion of the subject, the plurality of user controls is a specific template of a plurality of templates of user controls, and each template of the plurality of templates is defined to meet a different set of jurisdictional requirements;

receiving, via the user interface of the mobile computing device interacting with the computer processor, a selection of a particular user control included in the plurality of user controls, the selected, particular user control indicative of a particular portion of the subject from which a unique identifier of the subject is excluded;

launching, using the computer processor, an image capturing system on the mobile computing device;

receiving, using the computer processor, a first electronic image of the unique identifier of the subject, the first electronic image captured by the image capturing system;

image processing, using the computer processor, the first electronic image to determine the unique identifier of the subject;

receiving, using the computer processor, a second electronic image captured by the image capturing system and including an image of the particular portion of the subject corresponding to the selected, particular user control; and

causing, using the computer processor, the captured second electronic image and metadata indicative of attributes corresponding to the captured second electronic image to be stored in a data storage system, the attributes corresponding to the captured second electronic image including (i) a pre-assigned identifier that corresponds to the selected, particular user control and identifies the particular portion of the subject, is expressed in a non-natural language that is a computer-readable language having a particular syntax and format, and is not presented on the user interface of the mobile computing device, and (ii) the determined unique identifier of the subject, thereby automatically labeling the captured second electronic image with the pre-assigned identifier corresponding to the particular portion of the subject and with the unique identifier of the subject.

37. The method of claim 36, further comprising receiving the attributes corresponding to the captured second electronic image at a time of capture of the second electronic image;

and

wherein the attributes further include at least one of;

the time of day of capture of the captured second electronic image, a date of capture of the captured second electronic image, or a geospatial location at which the second electronic image was captured.

38. The method of claim 36, wherein corresponds to a respective make, a respective model, and/or a respective year of subjects.

39. The method of claim 38, wherein each template is further defined to at least one of:

meet a respective set of requirements governed by a respective application that operates on labeled images, or meet a respective set of situational requirements.