Visual pattern recognition in an image

• US 9,141,885 B2
• Filed: 07/29/2013
• Issued: 09/22/2015
• Est. Priority Date: 07/29/2013
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Yang, Jianchao, Chen, Guang, Brandt, Jonathan, Jin, Hailin, Shechtman, Elya, Agarwala, Aseem Omprakash
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/953,394
• Publication Number:
  US 20150030238A1
• Time in Patent Office: 785 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis

Abstract

A system may be configured as an image recognition machine that utilizes an image feature representation called local feature embedding (LFE). LFE enables generation of a feature vector that captures salient visual properties of an image to address both the fine-grained aspects and the coarse-grained aspects of recognizing a visual pattern depicted in the image. Configured to utilize image feature vectors with LFE, the system may implement a nearest class mean (NCM) classifier, as well as a scalable recognition algorithm with metric learning and max margin template selection. Accordingly, the system may be updated to accommodate new classes with very little added computational cost. This may have the effect of enabling the system to readily handle open-ended image classification problems.

Claims

1. A method comprising:

generating a first set of vectors by dividing an image into pixel blocks and transforming pixel values from the pixel blocks, a first vector of the first set being generated from a pixel block among the pixel blocks;

encoding the first set of vectors into a second set of vectors, a second vector of the second set being an encoded representation of the first vector;

generating a first array of ordered pairs, a first ordered pair of the first array pairing a value from the second vector with an index of the second vector, the index indicating the pixel block;

using a processor, generating a second array of ordered pairs, a second ordered pair of the second array pairing the value with the first vector; and

storing the second array as a feature vector of the image.

2. The method of claim 1, wherein:

the generating of the second array of ordered pairs is based on the first array of ordered pairs and includes identifying the first vector based on the index of the second vector.

3. The method of claim 1, wherein:

the value from the second vector has a maximum absolute value among the second set of vectors for a dimension in common among the second set of vectors; and

the generating of the first array includes selecting the value based on the value having the maximum absolute value for the dimension.

4. The method of claim 1 further comprising:

training an image classifier to classify the image based on the second array of ordered pairs stored as the feature vector of the image.

5. The method of claim 4, wherein:

the image depicts text rendered in a font; and

the training of the image classifier trains the image classifier to classify the image by classifying the font in which the text depicted in the image is rendered.

6. The method of claim 5, wherein:

the classifying of the image includes identifying the font based on the second array or ordered pairs stored as the feature vector of the image.

7. The method of claim 4, wherein:

the image depicts a face of a person; and

the training of the image classifier trains the image classifier to classify the image by classifying the face depicted in the image.

8. The method of claim 4, wherein:

the training of the image classifier includes calculating a probability vector for the second ordered pair, the probability vector defining a distribution of probabilities that include a probability of the first vector representing a feature that characterizes a class of images.

9. The method of claim 8, wherein:

the training of the image classifier includes determining a weight vector of scalar values that, when the weight vector is multiplied to the probability vector, modify the distribution of probabilities to indicate that the first vector does represent the feature that characterizes the class of images.

10. The method of claim 9, wherein:

the image belongs to the class of images; and

the training of the image classifier determines the weight vector based on a constraint that the feature represented by the first vector characterizes the class of images to which the image belongs.

11. The method of claim 9, wherein:

the training of the image classifier includes storing the weight vector as a template applicable to multiple classes of images.

12. The method of claim 1, wherein:

the image is a reference image; and

the method further comprises classifying a candidate image based on the second array of ordered pairs stored as the feature vector of the reference image.

13. The method of claim 12, wherein:

the candidate image depicts text rendered in a font; and

the classifying of the candidate image includes classifying the font in which the text depicted in the candidate image is rendered.

14. The method of claim 13, wherein:

the classifying of the candidate image includes identifying the font using an image classifier trained with the second array of ordered pairs stored as the feature vector of the reference image, the identifying of the font being based on a further feature vector of the candidate image.

15. The method of claim 12, wherein:

the reference image belongs to a class of images; and

the classifying of the candidate image is based on a weight vector determined based on a constraint that a feature represented by the first vector characterizes the class of images to which the reference image belongs.

16. The method of claim 1, wherein:

the first vector has a first number of dimensions;

the second vector has a second number of dimensions that is less than the first number of dimensions;

the ordered pairs in the first array are equal in number to the second number of dimensions; and

the ordered pairs in the second array are equal in number to the second number of dimensions.

17. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

generating a first set of vectors by dividing an image into pixel blocks and mathematically transforming pixel values from the pixel blocks, a first vector of the first set being generated from a pixel block among the pixel blocks;

encoding the first set of vectors into a second set of vectors, a second vector of the second set being an encoded representation of the first vector;

generating a first array of ordered pairs, a first ordered pair of the first array pairing a value from the second vector with an index of the second vector, the index indicating the pixel block from which the first vector is generated;

generating a second array of ordered pairs, a second ordered pair of the second array pairing the value from the second vector with the first vector generated from the pixel block; and

storing the second array of ordered pairs as a feature vector of the image.

18. The non-transitory machine-readable storage medium of claim 17, wherein:

the image is a reference image; and

the operations further comprise classifying a candidate image based on the second array of ordered pairs stored as the feature vector of the reference image.

19. A system comprising:

a processor configured by a feature vector module to;

generate a first set of vectors by dividing an image into pixel blocks and transforming pixel values from the pixel blocks, a first vector of the first set being generated from a pixel block among the pixel blocks;

encode the first set of vectors into a second set of vectors, a second vector of the second set being an encoded representation of the first vector;

generate a first array of ordered pairs, a first ordered pair of the first array pairing a value from the second vector with an index of the second vector, the index indicating the pixel block; and

generate a second array of ordered pairs, a second ordered pair of the second array pairing the value with the first vector; and

a vector storage module configured to store the second array as a feature vector of the image.

20. The system of claim 19 further comprising:

a classifier trainer module configured to train an image classifier to classify the image based on the second array of ordered pairs stored as the feature vector of the image.