Machine learning artificial intelligence system for identifying vehicles

• US 10,235,602 B1
• Filed: 04/09/2018
• Issued: 03/19/2019
• Est. Priority Date: 03/08/2018
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Vasisht, Sunil Subrahmanyam, Dagley, Geoffrey, Tang, Qiaochu, Reddy, Sean, Hoover, Jason Richard, Wylie, Stephen Michael, Price, Micah
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US15/948,286
• Time in Patent Office: 344 Days
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

An artificial intelligence system for identifying attributes in an image. The system may include a processor in communication with a client device; and a storage medium. The storage medium may store instructions that, when executed, configure the processor to perform operations including: extracting first features; categorizing the first images in a first group or a second group; modifying first metadata associated with each image in the first images to include a binary label; calculating a classification function; classifying a second plurality of images using the classification function; extracting second features from the second images classified in the first group; categorizing the second images in the first group by attribute; calculating an attribute identification function that identifies attributes of the second images; and identifying at least one attribute associated with a client image using the attribute identification function, the client image being received from the client device.

Claims

1. A computer-implemented method for identifying attributes in an image of a vehicle, the method comprising:

collecting images of vehicles from the online resource;

classifying the collected images in a first group or a second group using a classifier function, the first group comprising images of vehicle exteriors and the second group comprising images of vehicle interiors;

modifying metadata associated with images in the first group to indicate that the images in the first group display vehicle exteriors;

retrieving an original training data set of images and an original attribute identification function from a database;

generating an updated training data set by adding the images in the first group to the original training data set;

calculating an updated attribution identification function based on the updated training data set and the original attribute identification function;

receiving a client image of a vehicle from a client processor; and

identifying attributes in a client image of a vehicle using the updated attribute identification function.

2. A computer system for identifying attributes in an image of a vehicle, the system comprising:

at least one processor in communication with an online resource and a database; and

at least one storage medium storing instructions that, when executed, configure the processor to execute operations comprising;

collecting images of vehicles from the online resource;

classifying the collected images in a first group or a second group using a classifier function, the first group comprising images of vehicle exteriors and the second group comprising images of vehicle interiors;

modifying metadata associated with images in the first group to indicate that the images in the first group display vehicle exteriors;

retrieving an original training data set of images and an original attribute identification function from the database;

generating an updated training data set by adding the images in the first group to the original training data set;

calculating an updated attribution identification function based on the updated training data set and the original attribute identification function;

receiving a client image of a vehicle from a client processor; and

identifying attributes in the client image using the updated attribute identification function.

3. The computer system of claim 2 wherein the operations further comprise

generating a modified graphical user interface overlaying the identified attributes in the image received from the client processor.

4. The computer system of claim 2, wherein generating an updated training data set comprises generating augmented images based on the images in the first group by applying filters to the images in the first group, the filters being associated with a random function.

5. The computer system of claim 4, wherein:

the filters comprise a zoom filter and a rotation filter; and

the random function determines a zoom percentage of the zoom filter and a rotation angle of the rotation filter.

6. The computer system of claim 4, wherein generating augmented images comprises:

determining a difference measurement between the augmented images and the images in the first group; and

discarding the augmented images when the difference measurement is below a threshold.

7. The computer system of claim 2, wherein calculating an updated attribution identification function comprises extracting features of the images using a pre-trained convolutional neural network.

8. The computer system of claim 2, wherein the classifier function comprises a logistic regression classifier based on clustered images of vehicle interiors and vehicle exteriors.

9. The computer system of claim 2, wherein:

the online resource comprises a website of at least one of a car dealer or a car manufacturer; and

the identified attributes comprise a vehicle make and a vehicle model.

10. The computer system of claim 9, wherein the operations further comprise:

querying the online resource to determine a cost and a condition associated with the identified attributes in the client image; and

generating a graphical user interface overlaying the identified attributes, the cost, and the condition.

11. The computer system of claim 10, wherein the operations further comprise:

querying the online resource for availability of vehicles with the identified attributes;

receiving a location of the client processor; and

generating a graphical user interface when the online resource indicates availability of vehicles with the identified attributes, the graphical user interface displaying at least one of a map to the car dealer or an identification of the car manufacturer associated with the online resource.

12. The computer system of claim 2, wherein collecting images comprises

identifying an occurrence of changes in the online resource;

and

initializing a web scraper configured to collect images.

13. The computer system of claim 2, wherein

the original attribute identification function comprises a convolutional neural network;

and

calculating an updated attribute identification function comprises updating at least one of activation functions or synapsis weights of the convolutional neural network.

14. The computer system of claim 13, wherein the convolutional neural network comprises a multi-output network comprising layers to identify a vehicle year and a vehicle trim level.

15. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to operate a system for identifying attributes in an image of a vehicle, the operations comprising:

collecting images of vehicles from an online resource;

classifying the collected images in a first group or a second group using a classifier function, the first group comprising images of vehicle exteriors and the second group comprising images of vehicle interiors;

modifying metadata associated with images in the first group to indicate that the images in the first group display vehicle exteriors;

retrieving an original training data set of images and an original attribute identification function from a database;

generating an updated training data set by adding the images in the first group to the original training data set;

calculating an updated attribution identification function based on the updated training data set and the original attribute identification function;

receiving a client image of a vehicle from a client processor; and

identifying attributes in a client image using the updated attribute identification function.

16. The non-transitory computer-readable medium of claim 15, wherein the operations further comprise generating a modified graphical user interface overlaying the identified attributes in the image received from the client processor.

17. The non-transitory computer-readable medium of claim 15, wherein generating an updated training data set comprises generating augmented images based on the images in the first group by applying filters to the images in the first group, the filters being associated with a random function.

18. The non-transitory computer-readable medium of claim 17, wherein

the filters comprise a zoom filter and a rotation filter;

and

the random function determines a zoom percentage of the zoom filter and a rotation angle of the rotation filter.

19. The non-transitory computer-readable medium of claim 15, wherein the classifier function comprises a logistic regression classifier based on clustered images of vehicle interiors and vehicle exteriors.