Image Classification

• US 20130148881A1
• Filed: 12/11/2012
• Published: 06/13/2013
• Est. Priority Date: 12/12/2011
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Xue, Hui
• Granted Patent: US 9,342,758 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 2218/00 Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions

Abstract

The present disclosure introduces a method and an apparatus for classifying images. Classification image features of an image for classification are extracted. Based on a similarity relationship between each classification image feature and one or more visual words in a pre-generated visual dictionary, each classification image feature is quantified by multiple visual words in the visual dictionary and a similarity coefficient between each classification image feature and each of the visual words is determined. Based on the similarity coefficient of each visual word that corresponds to different classification image features, a weight of each visual word is determined to establish a classification visual word histogram of the image for classification. The classification visual word histogram is input into an image classifier that is trained by sample visual word histograms arising from multiple sample images. An output result is used to determine a classification of the image for classification.

Claims

1. A method performed by one or more processors configured with computer-executable instructions, the method comprising:

extracting one or more classification image features from an image for classification;

based on a similarity relationship between each classification image feature and each visual word in a pre-generated visual dictionary, quantifying each classification image feature by multiple visual words in the visual dictionary and determining a similarity coefficient between each classification image feature and each visual word after the quantifying;

based on one or more similarity coefficients of each visual word corresponding to different classification image features, determining a weight of each visual word to establish a classification visual word histogram;

inputting the classification visual word histogram into an image classifier; and

using an output of the inputting to determine a classification of the image for classification.

2. The method as recited in claim 1, wherein the image classifier is generated through training by sample visual word histograms from multiple sample images.

3. The method as recited in claim 2, further comprising after inputting the classification visual word histogram into an image classifier, comparing the inputted classification visual word histogram to a pre-generated classification visual word histogram in the image classifier to determine a classification of the image for classification.

4. The method as recited in claim 1, wherein the quantifying each classification image feature by multiple visual words in the visual dictionary and determining the similarity coefficient between each classification image feature and each visual word after the quantifying comprises:

based on the similarity relationship between each classification image feature and each visual words in the visual dictionary, through a sparse coding method, establishing a sparse coding model between a respective classification image feature and one or more visual words in the visual dictionary;

resolving the sparse coding model to quantify the respective classification image feature by the multiple visual words in the visual dictionary; and

obtaining the one or more similarity coefficients between the respective classification image feature and the multiple visual words in the visual dictionary.

5. The method as recited in claim 4, wherein the sparse coding model is represented by

arg_C^minΣ

_i=1^N∥

X_i−

BC_i∥

+λ

∥

C_i∥

_l1

wherein;

B represents the visual dictionary;

X_i represents the respective classification image feature;

C_i represents a dataset formed by coefficients between the respective classification image feature and each visual word;

N represents a number of classification image features;

λ

represents a constraining coefficient;

∥

C_i∥

_l1 represents a sum of all elements in C_i; and

the sparse coding model is used to calculate a value of C_i when the sparse coding model has a minimum value.

6. The method as recited in claim 1, wherein the quantifying each classification image feature by multiple visual words in the visual dictionary and determining the similarity coefficient between each classification image feature and each visual word after the quantifying comprises:

based on the similarity relationship between each classification image feature and the visual words in the pre-generated visual dictionary, calculating a Euclidean distance between each classification image feature and each visual word;

determining a smallest Euclidean distance among calculated Euclidean distances;

with respect to each classification image feature, determining one or more visual words of which Euclidean distances are within a preset times range of the smallest Euclidean distance as the visual words for quantification of the respective classification image feature; and

based on the Euclidean distance between the respective classification image feature and each of the visual words for quantification, calculating the one or more coefficients between the respective classification image feature and the visual words.

7. The method as recited in claim 1, wherein the determining the weight of each visual word to establish the classification visual word histogram comprises:

adding up the one or more coefficients of a respective visual word that correspond to different classification image features to calculate the weight of the respective visual word; and

establishing the classification visual word histogram.

8. The method as recited in claim 1, wherein the determining the weight of each visual word to establish the classification visual word histogram comprises:

dividing the image for classification into multiple layer images based on a pyramid image algorithm; and

dividing each layer image to form multiple child images.

9. The method as recited in claim 1, wherein the visual dictionary is generated through clustering of multiple sample image features extracted from multiple sample images.

10. An apparatus comprising:

a feature extraction module that extracts one or more classification image features from an image for classification;

a quantification determination module, that based on a similarity relationship between each classification image feature and each visual word in a pre-generated visual dictionary, quantifies each classification image feature by multiple visual words in the visual dictionary and determining a similarity coefficient between each classification image feature and each visual word after the quantifying;

an establishment module that, based on one or more similarity coefficients of each visual word corresponding to different classification image features, determines a weight of each visual word to establish a classification visual word histogram; and

a classification module that inputs the classification visual word histogram into an image classifier and using an output to determine a classification of the image for classification.

11. The apparatus as recited in claim 10, wherein the image classifier is generated through training by sample visual word histograms from multiple sample images.

12. The apparatus as recited in claim 11, wherein the image classifier compares the inputted classification visual word histogram with a pre-generated classification visual word histogram to determine a classification of the image for classification.

13. The apparatus as recited in claim 10, wherein the quantification determination module comprises:

a model construction module that based on the similarity relationship between each classification image feature and each visual words in the visual dictionary, through a sparse coding method, establishing a sparse coding model between a respective classification image feature and one or more visual words in the visual dictionary; and

a quantification determination module that resolves the sparse coding model to quantify the respective classification image feature by the multiple visual words in the visual dictionary and obtains the one or more similarity coefficients between the respective classification image feature and the multiple visual words in the visual dictionary.

14. The apparatus as recited in claim 13, wherein the sparse coding model is represented by

arg_C^minΣ

_i=1^N∥

X_i−

BC_i∥

+λ

∥

C_i∥

_l1

wherein;

B represents the visual dictionary;

X_i represents the respective classification image feature;

C_i represents a dataset formed by coefficients between the respective classification image feature and each visual word;

N represents a number of classification image features;

λ

represents a constraining coefficient;

∥

C_i∥

_l1 represents a sum of all elements in C_i; and

the sparse coding model is used to calculate a value of C_i when the sparse coding model has a minimum value.

15. The apparatus as recited in claim 10, wherein the quantification determination module comprises:

a first calculation module that, based on the similarity relationship between each classification image feature and the visual words in the pre-generated visual dictionary, calculates a Euclidean distance between each classification image feature and each visual word; and

a quantification module that determines a smallest Euclidean distance among calculated Euclidean distances and, with respect to each classification image feature, determining one or more visual words of which Euclidean distances are within a preset times range of the smallest Euclidean distance as the visual words for quantification of the respective classification image feature; and

a second calculation module that, based on the Euclidean distance between the respective classification image feature and each of the visual words for quantification, calculates the one or more coefficients between the respective classification image feature and the visual words.

16. The apparatus as recited in claim 10, wherein the establishment module comprises:

a division module that divides the image for classification into multiple child images based on a pyramid image algorithm;

a first establishment module that determines classification image features of each child image, adds the coefficients of a respective visual word that correspond to each classification image features in a respective child image, and calculates the weight of the respective visual word corresponding to the respective child image to establish a child classification visual word histogram of the respective child image; and

a second establishment module that combines each child classification visual word histogram of each child image to establish the classification visual word histogram.

17. One or more computer storage media including processor-executable instructions that, when executed by one or more processors, direct the one or more processors to perform a method comprising:

extracting one or more classification image features from an image for classification;

based on a similarity relationship between each classification image feature and each visual word in a pre-generated visual dictionary, quantifying each classification image feature by multiple visual words in the visual dictionary and determining a similarity coefficient between each classification image feature and each visual word after the quantifying;

based on one or more similarity coefficients of each visual word corresponding to different classification image features, determining a weight of each visual word to establish a classification visual word histogram;

inputting the classification visual word histogram into an image classifier that is generated through training by sample visual word histograms from multiple sample images; and

using an output of the inputting to determine a classification of the image for classification.

18. The one or more computer storage media as recited in claim 17, wherein the method further comprises after inputting the classification visual word histogram into an image classifier comprises, comparing the inputted classification visual word histogram to a pre-generated classification visual word histogram in the image classifier to determine a classification of the image for classification.

19. The one or more computer storage media as recited in claim 17, wherein the determining the similarity coefficient between each classification image feature and each visual word after the quantifying comprises:

based on the similarity relationship between each classification image feature and each visual words in the visual dictionary, through a sparse coding method, establishing a sparse coding model between a respective classification image feature and one or more visual words in the visual dictionary;

resolving the sparse coding model to quantify the respective classification image feature by the multiple visual words in the visual dictionary; and

obtaining the one or more similarity coefficients between the respective classification image feature and the multiple visual words in the visual dictionary,

wherein;

the sparse coding model is represented by

arg_C^minΣ

_i=1^N∥

X_i−

BC_i∥

+λ

∥

C_i∥

_l1

wherein B represents the visual dictionary;

X_i represents the respective classification image feature;

C_i represents a dataset formed by coefficients between the respective classification image feature and each visual word;

N represents a number of classification image features;

λ

represents a constraining coefficient;

∥

C_i∥

_l1 represents a sum of all elements in C_i; and

the sparse coding model is used to calculate a value of C_i when the sparse coding model has a minimum value.

20. The one or more computer storage media as recited in claim 17, wherein the quantifying each classification image feature by multiple visual words in the visual dictionary and determining the similarity coefficient between each classification image feature and each visual word after the quantifying comprises:

based on the similarity relationship between each classification image feature and the visual words in the pre-generated visual dictionary, calculating a Euclidean distance between each classification image feature and each visual word;

determining a smallest Euclidean distance among calculated Euclidean distances;

with respect to each classification image feature, determining one or more visual words of which Euclidean distances are within a preset times range of the smallest Euclidean distance as the visual words for quantification of the respective classification image feature; and

based on the Euclidean distance between the respective classification image feature and each of the visual words for quantification, calculating the one or more coefficients between the respective classification image feature and the visual words.