TRAINING A CLASSIFIER ALGORITHM USED FOR AUTOMATICALLY GENERATING TAGS TO BE APPLIED TO IMAGES

• US 20160379091A1
• Filed: 06/23/2015
• Published: 12/29/2016
• Est. Priority Date: 06/23/2015
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Lin, Zhe, Guggisberg, Stefan, Brandt, Jonathan, Marth, Michael
• Granted Patent: US 9,767,386 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/776 Validation; Performance evaluation
• CPC Class Codes: G06V 20/42 of sport video content
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations

Abstract

This disclosure relates to training a classifier algorithm that can be used for automatically selecting tags to be applied to a received image. For example, a computing device can group training images together based on the training images having similar tags. The computing device trains a classifier algorithm to identify the training images as semantically similar to one another based on the training images being grouped together. The trained classifier algorithm is used to determine that an input image is semantically similar to an example tagged image. A tag is generated for the input image using tag content from the example tagged image based on determining that the input image is semantically similar to the tagged image.

Claims

1. A method for training a classifier algorithm used for automatically generating tags to be applied to a received image, the method comprising:

grouping, by a processor, training images together based on the training images having similar tags;

training, by the processor, a classifier algorithm to identify the training images as semantically similar to one another based on the training images being grouped together;

executing the trained classifier algorithm to determine that an input image is semantically similar to an example tagged image; and

generating a tag for the input image using tag content from the example tagged image based on determining that the input image is semantically similar to the example tagged image.

2. The method of claim 1, wherein the training images are grouped into a cluster and wherein training the classifier algorithm comprises executing a neural network classification algorithm with inputs including the training images in the cluster and excluding other training images from a set of training images that are not grouped into the cluster.

3. The method of claim 1, further comprising determining that the training images have tags with similar content by performing operations comprising:

identifying, for each training image, at least one respective tag; and

determining that at least a portion of each respective tag in the training image is identical to portions of tags associated with the other training images.

4. The method of claim 1, wherein the training images are also grouped based on the training images having tags with similar rates of occurrence within a set of tagged training images that includes the training images and additional training images.

5. The method of claim 1,

wherein the training images are grouped into a cluster by performing operations comprising:

identifying a set of available tags, wherein each tag in the set of available tags is associated with at least one image in a set of tagged training images that includes the training images and additional training images;

determining, for each tag in the set of available tags, a respective weight indicative of a respective number of occurrences of the tag in the set of tagged training images;

generating, for each training image in the set of tagged training images, a respective tag vector having a plurality of elements, wherein each element has a respective value representing at least one of (i) the weight of a corresponding tag from the set of available tags or (ii) the absence of the corresponding tag from the training image;

grouping the training images into the cluster based on the grouped training images having respective tag vectors with respective magnitudes within a threshold distance of a centroid for the cluster,

wherein the method further comprises, prior to determining that the input image is semantically similar to the example tagged image, generating a feature vector representing semantic content of the example tagged image;

wherein determining that the input image is semantically similar to the example tagged image comprises;

generating an input feature vector representing semantic content of the input image;

determining that a distance between the feature vector for the example tagged image and the input feature vector for the input image is less than a threshold distance for selecting similar semantic content.

6. The method of claim 5, wherein generating the respective tag vector for each training image comprises:

identifying a number of tags in the training image;

computing, for each tag from the set of available tags that is associated with the training image, the respective weight based on (i) the number of tags in the training image and (ii) the respective number of occurrences of the tag in the set of tagged training images;

assigning the computed weights to a first subset of elements in the tag vector that represent tags from the set of available tags that are associated with the training image; and

assigning zero values to a second subset of elements in the tag vector that represent tags from the set of available tags that are not associated with the training image.

7. The method of claim 5, wherein determining at least one weight for at least one tag comprises:

determining that the at least one tag has been previously selected for application to a previous input image; and

increasing the at least one weight based on determining that the at least one tag has been previously selected for the previous input image.

8. The method of claim 1, wherein each tag for a training image comprises metadata identifying at least one of:

a description of subject matter depicted in the training image;

an author or editor of the training image;

a time period associated with the training image;

an image style; and

a location depicted in or associated with the training image.

9. A method for optimizing a classifier algorithm for matching images, the method comprising:

generating, by a processor, tag vectors for respective training images, wherein each tag vector for a respective training image has elements representing a respective set of tags associated with the respective training image;

identifying, by the processor, groups of related tag vectors, wherein each group of related tag vectors corresponds to a respective subset of the training images and is collocated in a respective region of a space defined by the tag vectors;

for each subset of training images, training a classifier algorithm to identify the subset of training images as semantically similar to one another based on the subset of training images being associated with a respective group of tag vector; and

transmitting the trained classifier algorithm to an application that uses the trained classifier algorithm to identify semantic similarities between images.

10. A system comprising:

a processor; and

a non-transitory computer-readable medium communicatively coupled to the processor,

wherein the processor is configured to execute program code stored in the non-transitory computer-readable medium and thereby perform operations comprising;

grouping training images together based on the training images having similar tags,

training a classifier algorithm to identify the training images as semantically similar to one another based on the training images being grouped together,

executing the trained classifier algorithm to determine that an input image is semantically similar to an example tagged image, and

generating a tag for the input image using tag content from the example tagged image based on determining that the input image is semantically similar to the example tagged image.

11. The system of claim 10, wherein the training images are grouped into a cluster and wherein training the classifier algorithm comprises executing a neural network classification algorithm with inputs including the training images in the cluster and excluding other training images from a set of training images that are not grouped into the cluster.

12. The system of claim 10, wherein the training images are also grouped based on the training images having tags with similar rates of occurrence within a set of tagged training images that includes the training images and additional training images.

13. The system of claim 10,

wherein the training images are grouped into a cluster by performing operations comprising:

identifying a set of available tags, wherein each tag in the set of available tags is associated with at least one image in a set of tagged training images that includes the training images and additional training images;

determining, for each tag in the set of available tags, a respective weight indicative of a respective number of occurrences of the tag in the set of tagged training images;

generating, for each training image in the set of tagged training images, a respective tag vector having a plurality of elements, wherein each element has a respective value representing at least one of (i) the weight of a corresponding tag from the set of available tags or (ii) the absence of the corresponding tag from the training image;

grouping the training images into the cluster based on the grouped training images having respective tag vectors with respective magnitudes within a threshold distance of a centroid for the cluster,

wherein the method further comprises, prior to determining that the input image is semantically similar to the example tagged image, generating a feature vector representing semantic content of the example tagged image;

wherein determining that the input image is semantically similar to the example tagged image comprises;

generating an input feature vector representing semantic content of the input image;

determining that a distance between the feature vector for the example tagged image and the input feature vector for the input image is less than a threshold distance for selecting similar semantic content.

14. The system of claim 13, wherein generating the respective tag vector for each training image comprises:

identifying a number of tags in the training image;

computing, for each tag from the set of available tags that is associated with the training image, the respective weight based on (i) the number of tags in the training image and (ii) the respective number of occurrences of the tag in the set of tagged training images;

assigning the computed weights to a first subset of elements in the tag vector that represent tags from the set of available tags that are associated with the training image; and

assigning zero values to a second subset of elements in the tag vector that represent tags from the set of available tags that are not associated with the training image.

15. The system of claim 13, wherein determining at least one weight for at least one tag comprises:

determining that the at least one tag has been previously selected for application to a previous input image; and

increasing the at least one weight based on determining that the at least one tag has been previously selected for the previous input image.

16-20. -20. (canceled)