Generic visual categorization method and system

• US 20070005356A1
• Filed: 06/30/2005
• Published: 01/04/2007
• Est. Priority Date: 06/30/2005
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Perronnin, Florent
• Granted Patent: US 7,756,341 B2
• Time in Patent Office: Days
• US Class Current: 704/245
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations

Abstract

Generic visual categorization methods complement a general vocabulary with adapted vocabularies that are class specific. Images to be categorized are characterized within different categories through a histogram indicating whether the image is better described by the general vocabulary or the class-specific adapted vocabulary.

Claims

1. A method for assigning one of a plurality of classes to an input image, comprising:

identifying a plurality of key-patches in the input image;

computing a feature vector for each of (at least ones of) the plurality of key-patches;

computing a histogram for each of (at least ones of) the plurality of classes using the plurality of feature vectors computed;

assigning at least one of the plurality of classes to the input image using the plurality of histograms computed as input to a classifier.

2. The method according to claim 1, wherein each histogram is computed by estimating occupancy probabilities of the feature vectors of the input image for each class vocabulary.

3. The method according to claim 2, further comprising defining each class vocabulary by merging a general vocabulary and an adapted vocabulary for each class.

4. The method according to claim 3, wherein the histogram computed for each of the plurality of classes indicates whether the input image is better described by the general vocabulary or the adapted vocabulary of its corresponding class.

5. The method according to claim 4, further comprising:

computing key-patches in images of class training sets;

computing feature vectors for key-patches of the images of the class training sets;

computing the general vocabulary by clustering feature vectors of the images of class training sets;

computing each class'"'"' adapted vocabulary by estimating occupancy probabilities, of the feature vectors of the images of its class training set, for each class vocabulary.

6. The method according to claim 5, further comprising training the classifier with histograms computed for each class by estimating occupancy probabilities, of the feature vectors of images of the class trainings set, for each class vocabulary.

7. The method according to claim 2, wherein said assigning further comprises:

computing a score for the input image for each of the plurality of classes using its computed histogram;

assigning the input image a label corresponding to the class with the highest computed score.

8. The method according to claim 7, wherein said computing computes the score using a Support Vector Machine classifier for each class.

9. An apparatus for assigning one of a plurality of classes to an input image, comprising:

a key-patch detector identifying a plurality of key-patches in the input image;

a feature description module for computing a feature vector for each of (at least ones of) the plurality of key-patches;

a multi-histogram computation module computing a histogram for each of (at least ones of) the plurality of classes using the plurality of feature vectors computed;

a classifier for assigning at least one of the plurality of classes to the input image using the plurality of histograms computed as input to the classifier.

10. The apparatus according to claim 9, wherein the multi-histogram computation module computes each histogram by estimating occupancy probabilities of the feature vectors of the input image for each class vocabulary.

11. The apparatus according to claim 10, wherein the multi-histogram computation module defines each class vocabulary by merging a general vocabulary and an adapted vocabulary for each class.

12. The apparatus according to claim 11, wherein the histogram computed by the multi-histogram computation module for each of the plurality of classes indicates whether the input image is better described by the general vocabulary or the adapted vocabulary of its corresponding class.

13. The apparatus according to claim 10, wherein

the key-patch detector computes key-patches in images of class training sets;

the feature description module computes feature vectors for key-patches of the images of the class training sets;

the multi-histogram computes;

the general vocabulary by clustering feature vectors of the images of class training sets, and each class'"'"' adapted vocabulary by estimating occupancy probabilities, of the feature vectors of the images of its class training set, for each class vocabulary.

14. The apparatus according to claim 13, further comprising a classifier training module for training the classifier with histograms computed for each class by estimating occupancy probabilities, of the feature vectors of images of the class trainings set, for each class vocabulary.

15. The apparatus according to claim 10, wherein the classifier:

computes a score for the input image for each of the plurality of classes using its computed histogram;

assigns the input image a label corresponding to the class with the highest computed score.

16. The apparatus according to claim 15, wherein the classifier comprises a Support Vector Machine classifier for computing the score for each class.

17. A method for training a classifier, comprising:

identifying key-patches in images of a plurality of class training sets;

computing feature vectors for the identified key-patches;

computing a general vocabulary by clustering the computed feature vectors;

for each of a plurality of classes, computing an adapted vocabulary using the general vocabulary;

computing a histogram for each of the plurality of classes;

training the classifier using the histograms for each of the plurality of classes.

18. The method according to claim 17, further comprising categorizing an input image with the image classifier.

19. The method according to claim 18, wherein said categorizing further comprises:

identifying a plurality of key-patches in the input image;

computing a feature vector for each of (at least ones of) the plurality of key-patches;

computing a histogram for each of (at least ones of) the plurality of classes using the plurality of feature vectors computed;

assigning at least one of the plurality of classes to the input image using the plurality of histograms computed as input to the classifier.

20. The method according to claim 19, wherein each histogram is computed by estimating occupancy probabilities of the feature vectors of the input image for each class vocabulary.