Machine learning artificial intelligence system for identifying vehicles

• US 10,140,553 B1
• Filed: 03/08/2018
• Issued: 11/27/2018
• Est. Priority Date: 03/08/2018
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Vasisht, Sunil Subrahmanyam, Dagley, Geoffrey, Tang, Qiaochu, Reddy, Sean, Hoover, Jason Richard, Wylie, Stephen Michael, Price, Micah
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US15/915,947
• Time in Patent Office: 264 Days
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

An artificial intelligence system for identifying attributes in an image. The system may include a processor in communication with a client device; and a storage medium. The storage medium may store instructions that, when executed, configure the processor to perform operations including: extracting first features; categorizing the first images in a first group or a second group; modifying first metadata associated with each image in the first images to include a binary label; calculating a classification function; classifying a second plurality of images using the classification function; extracting second features from the second images classified in the first group; categorizing the second images in the first group by attribute; calculating an attribute identification function that identifies attributes of the second images; and identifying at least one attribute associated with a client image using the attribute identification function, the client image being received from the client device.

Claims

1. A system for identifying attributes in an image, comprising:

at least one processor in communication with a client processor; and

at least one storage medium storing instructions that, when executed, configure the processor to perform operations comprising;

obtaining a plurality of first images, the first images being associated with first metadata, the first images comprising car images;

extracting a plurality of first features from the first images;

categorizing the first images into one of a first group or a second group based on the first features, the first group comprising car exterior images, the second group comprising car interior images;

modifying the first metadata associated with images in the first images to include a binary label, the binary label representing into which of first and second groups the image was categorized;

calculating a classification function that classifies the first images based on image features and the binary label;

obtaining a plurality of second images, the second images being associated with second metadata;

classifying the second images into one of the first group or the second group using the classification function, the classification function comprising a regression function;

extracting a plurality of second features from the second images classified into the first group;

categorizing the second images in the first group by attribute, based on the second features;

calculating an attribute identification function that identifies attributes of the second images based on image features;

receiving a client image from the client processor; and

identifying at least one attribute associated with the client image using the attribute identification function.

2. The artificial intelligence system of claim 1, wherein:

the operations further comprise generating augmented images by applying filters to the second images classified in the first group; and

extracting second features comprises extracting second features from the augmented images.

3. The artificial intelligence system of claim 2, wherein the filters include at least one of a zoom filter, a shear filter, a rotation filter, or a channel filter and are associated with a random function.

4. The artificial intelligence system of claim 1, wherein categorizing the first images comprises:

associating coordinates with each first images based on its corresponding first features;

determining a threshold distance based on distances between the associated coordinates of pairs of the first images; and

iteratively cluster coordinates that are within the threshold distance from each other.

5. The artificial intelligence system of claim 1, wherein extracting first features comprises:

transmitting the first images to a cluster comprising graphic processing units configured to extract image features; and

receiving, from the cluster comprising graphic processing units, a file with the first features.

6. The artificial intelligence system of claim 1, wherein extracting first features comprises:

identifying a file type for each image in the first images; and

normalizing each image in the first images based on the file type and a model image.

7. The artificial intelligence system of claim 6, wherein extracting first features further comprises using a pre-trained convolutional neural network.

8. The artificial intelligence system of claim 7, wherein using a pre-trained convolutional neural network comprises:

importing layers of the pre-trained convolutional neural network;

determining features described in a target layer of the pre-trained convolutional neural network; and

initializing a multiclass fitting model using the features in the target layer and the first images.

9. The artificial intelligence system of claim 1, wherein

the second images comprise car images, at least one of the second images being different from at least one of the first images;

and

the attribute identification function identifies at least one of a car make, a car model, a car year, or a car trim.

10. The artificial intelligence system of claim 1, wherein

the first metadata includes notes indicating the first images correspond to car interior or exterior;

and

the second metadata includes notes indicating car make and car model.

11. The artificial intelligence system of claim 10, wherein the second images comprise at least one million car images.

12. The artificial intelligence system of claim 1, wherein calculating an attribute identification function comprises:

partitioning the second images in the first group as at least training images and validation images;

generating an input file comprising features from images in the training images and a result file comprising the second metadata associated with the training images;

determining hyper parameters and activation functions;

calculating a plurality of weights based on the input file and the result file, the weights associating a plurality of nodes determined by the hyper parameters; and

recursively calculating the attribute identification function until a cost function is below a threshold error.

13. The artificial intelligence system of claim 1, wherein the operations further comprise transmitting display information to the client processor, the display information instructing the client processor to modify the client image by including the identified attribute.

14. The artificial intelligence system of claim 13, wherein the display information instruct the client processor to modify the client image by:

generating a new layer including the identified attribute; and

overlaying the new layer on top of original layers of the client image.

15. The artificial intelligence system of claim 1 wherein, the operations further comprise:

determining at least one of a cost or a condition based on the identified attribute by querying a database; and

transmitting information to the client processor including the at least one identified attribute, and the cost or the condition.

16. The artificial intelligence system of claim 1 wherein, the operations further comprise:

determine that the client image is not acceptable for identification;

transmitting display information to the client processor, the display information instructing the client processor to modify the client image by including at least one of a guiding line or an error message.

17. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to operate a computer system for identifying attributes in an image by performing operations, the operations comprising:

obtaining a plurality of first images, the first images being associated with first metadata, the first images comprising car images;

extracting a plurality of first features from the first images;

categorizing the first images into one of a first group or a second group based on the first features, the first group comprising car exterior images, the second group comprising car interior images;

modifying the first metadata associated with images in the first images to include a binary label, the binary label representing into which of first and second groups the image was categorized;

calculating a classification function that classifies the first images based on image features and the binary label;

obtaining a plurality of second images, the second images being associated with second metadata;

classifying the second images into one of the first group or the second group using the classification function, the classification function comprising a regression function;

extracting a plurality of second features from the second images classified into the first group;

categorizing the second images in the first group by attribute, based on the second features;

calculating an attribute identification function that identifies attributes of the second images based on image features;

receiving a client image from a client processor; and

identifying at least one attribute associated with the client image using the attribute identification function.

18. A computer-implemented method for identifying attributes in an image, the method comprising:

obtaining a plurality of first images, the first images being associated with first metadata, the first images comprising car images;

extracting a plurality of first features from the first images;

categorizing the first images into one of a first group or a second group based on the first features, the first group comprising car exterior images, the second group comprising car interior images;

modifying the first metadata associated with images in the first images to include a binary label, the binary label representing into which of first and second groups the image was categorized;

calculating a classification function that classifies the first images based on image features and the binary label;

obtaining a plurality of second images, the second images being associated with second metadata;

classifying the second images into one of the first group or the second group using the classification function, the classification function comprising a regression function;

extracting a plurality of second features from the second images classified into the first group;

categorizing the second images in the first group by attribute, based on the second features;

calculating an attribute identification function that identifies attributes of the second images based on image features;

receiving a client image from a client processor; and

identifying at least one attribute associated with the client image using the attribute identification function.