LOCAL FEATURE REPRESENTATION FOR IMAGE RECOGNITION

• US 20160132750A1
• Filed: 11/07/2014
• Published: 05/12/2016
• Est. Priority Date: 11/07/2014
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Yang, Jianchao, Brandt, Jonathan
• Granted Patent: US 10,043,101 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 30/1906 based also on statistics of image patches, e.g. active appearance models of the pattern to be recognised

Abstract

Techniques are disclosed for image feature representation. The techniques exhibit discriminative power that can be used in any number of classification tasks, and are particularly effective with respect to fine-grained image classification tasks. In an embodiment, a given image to be classified is divided into image patches. A vector is generated for each image patch. Each image patch vector is compared to the Gaussian mixture components (each mixture component is also a vector) of a Gaussian Mixture Model (GMM). Each such comparison generates a similarity score for each image patch vector. For each Gaussian mixture component, the image patch vectors associated with a similarity score that is too low are eliminated. The selectively pooled vectors from all the Gaussian mixture components are then concatenated to form the final image feature vector, which can be provided to a classifier so the given input image can be properly categorized.

Claims

1. A computer-implemented method, comprising:

receiving a digital image;

dividing the image into image patches;

generating a vector for each image patch;

comparing each image patch vector to Gaussian mixture components of a Gaussian Mixture Model (GMM), each mixture component being a vector, thereby generating a similarity score for each image patch vector;

for each Gaussian mixture component, eliminating one or more image patch vectors associated with a similarity score that is below a given threshold; and

generating a final image feature vector from the remaining image patch vectors of all the Gaussian mixture components.

2. The method of claim 1 further comprising providing the final image feature vector to a classifier so the given input image can be categorized.

3. The method of claim 1 wherein, for each Gaussian mixture component, a plurality of image patch vectors remain, so as to provide k-nearest neighbor pooling or thresholding.

4. The method of claim 1 wherein, for each Gaussian mixture component, one image patch vector remains, so as to provide 1-nearest neighbor pooling.

5. The method of claim 1 wherein, for one or more Gaussian mixture components, no image patch vectors remain.

6. The method of claim 1 wherein the remaining vectors are represented as matrices, and generating the final image feature vector includes combining those matrices via concatenation.

7. The method of claim 1 wherein an encoding matrix is used to combine select image patch vectors to produce the final image feature vector, and eliminating the one or more image patch vectors includes keeping only the largest value in each row of the encoding matrix and setting all others to be zero.

8. A non-transient computer program product having instructions encoded thereon that when executed by one or more processors causes a process to be carried out, the process comprising:

dividing a digital image into image patches;

generating a mathematical representation for each image patch;

comparing each image patch mathematical representation to Gaussian mixture components of a Gaussian Mixture Model (GMM), thereby generating a similarity score for each image patch mathematical representation;

for each Gaussian mixture component, eliminating one or more image patch mathematical representations associated with a similarity score that is below a given threshold; and

generating a final image feature mathematical representation from the remaining image patch mathematical representations of all the Gaussian mixture components.

9. The computer program product of claim 8, the process further comprising categorizing the input image using the final image feature mathematical representation.

10. The computer program product of claim 8 wherein, for each Gaussian mixture component, a plurality of image patch mathematical representations remain, so as to provide k-nearest neighbor pooling or thresholding.

11. The computer program product of claim 8 wherein, for each Gaussian mixture component, one image patch mathematical representation remains, so as to provide 1-nearest neighbor pooling.

12. The computer program product of claim 8 wherein, for one or more Gaussian mixture components, no image patch mathematical representations remain.

13. The computer program product of claim 8 wherein the remaining mathematical representations are represented as matrices, and generating the final image feature mathematical representation includes combining those matrices via concatenation.

14. The computer program product of claim 8 wherein an encoding matrix is used to combine select image patch vectors to produce the final image feature vector, and eliminating the one or more image patch mathematical representations includes keeping only the largest value in each row of the encoding matrix and setting all others to be zero.

15. A computing system, comprising:

an electronic memory for storing executable instructions;

a processor configured to execute the instructions to;

divide a digital image into image patches;

generate a vector for each image patch;

compare each image patch vector to Gaussian mixture components of a Gaussian Mixture Model (GMM), each mixture component being a vector, thereby generating a similarity score for each image patch vector;

for each Gaussian mixture component, eliminate one or more image patch vectors associated with a similarity score that is below a given threshold;

generate a final image feature vector from the remaining image patch vectors of all the Gaussian mixture components; and

categorize the image using the final image feature vector.

16. The system of claim 15 wherein, for each Gaussian mixture component, a plurality of image patch vectors remain, so as to provide k-nearest neighbor pooling or thresholding.

17. The system of claim 15 wherein, for each Gaussian mixture component, one image patch vector remains, so as to provide 1-nearest neighbor pooling.

18. The system of claim 15 wherein, for one or more Gaussian mixture components, no image patch vectors remain.

19. The system of claim 15 wherein the remaining vectors are represented as matrices, and processor generates the final image feature vector by combining those matrices via concatenation.

20. The system of claim 15 wherein an encoding matrix is used to combine select image patch vectors to produce the final image feature vector, and eliminating the one or more image patch vectors includes keeping only the largest value in each row of the encoding matrix and setting all others to be zero.