DYNAMIC FEATURE SELECTION FOR JOINT PROBABILISTIC RECOGNITION

• US 20160063358A1
• Filed: 08/24/2015
• Published: 03/03/2016
• Est. Priority Date: 08/29/2014
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: MEHRSERESHT, NAGITA
• Granted Patent: US 10,380,173 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 18/2115 by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 10/85 Markov-related models; Markov random fields
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene

Abstract

A method of jointly classifying a plurality of objects in an image using a feature type selected from a plurality of feature types determines classification information for each of the plurality of objects in the image by applying a predetermined joint classifier to at least one feature of a first type. The feature is generated from the image using a first feature extractor, the classification information being based on a probability of each of a plurality of possible classifications. The method estimates, for each of the feature types, an improvement in an accuracy of classification for each of the plurality of objects. The method selects features of a further type, from the plurality of feature types, according to the estimated improvement in the accuracy of the classification of each of the objects, and classifies the plurality of objects in the image using the selected features of the further type.

Claims

1. A computer-implemented method of jointly classifying a plurality of objects in an image using a feature type selected from a plurality of feature types, the method comprising:

determining classification information for each of the plurality of objects in the image by applying a predetermined joint classifier to at least one feature of a first type, the at least one feature being generated from the image using a first feature extractor, the classification information being based on a probability of each of a plurality of possible classifications;

estimating, for each of the feature types, an improvement in an accuracy of classification for each of the plurality of objects, the estimated improvement being formed using the determined classification information for each of the objects and a type of each of the objects in the image;

selecting features of a further type, from the plurality of feature types, according to the estimated improvement in the accuracy of the classification of each of the objects; and

classifying the plurality of objects in the image using the selected features of the further type.

2. The method according to claim 1, further comprising determining a benefit of evaluating a particular feature type of the plurality of feature types based on the estimated improvement in accuracy of classification.

3. The method according to claim 2, wherein the benefit is determined by normalising the estimated improvement in accuracy with a computation cost, such that the benefit is a measure of the estimated improvement in accuracy improvement per computation cost.

4. The method according to claim 2, wherein the benefit is determined using a trade-off across expected total accuracy improvement and a computation cost.

5. The method according to claim 1, wherein the estimating comprises determining a direct accuracy improvement and an indirect accuracy improvement associated with a particular feature, and summing the direct accuracy improvement and the indirect accuracy improvement to provide the estimated improvement in accuracy.

6. The method according to claim 5, wherein the determining of at least one of the direct accuracy improvement- and the indirect accuracy improvement comprises linear regression of unprocessed features.

7. The method according to claim 6, wherein evaluation of the regression utilizes the determined classification information.

8. The method according to claim 5, wherein the determining of the direct accuracy improvement comprises non-linear regression of unprocessed features.

9. The method according to claim 1, wherein the estimated improvement is formed using a pair-wise interaction of the determined classification information for each of the objects and a type of each of the objects in the image.

10. A method of jointly classifying a plurality of objects in data using a feature type selected from a plurality of feature types, the method comprising:

determining classification information for each of the plurality of objects in the data by applying a predetermined joint classifier to at least one feature of a first type, the at least one feature being generated from the data using a first feature extractor, the classification information being based on a probability of each of a plurality of possible classifications;

estimating, for each of the feature types, an improvement in an accuracy of classification for each of the plurality of objects, the estimated improvement being formed using the determined classification information for each of the objects and a type of each of the objects in the data;

selecting features of a further type, from the plurality of feature types, according to the estimated improvement in the accuracy of the classification of each of the objects; and

classifying the plurality of objects in the data using the selected features of the further type.

11. The method according to claim 10, wherein the data is at least one of image data, audio data, and text data.

12. The method according to claim 11, wherein the data is image data from which text feature data is extracted by at least one of the feature extractors.

13. A non-transitory computer readable storage medium having a program recorded thereon, the program being executable by a processor to perform a method to jointly classify a plurality of objects in an image using a feature type selected from a plurality of feature types, the method comprising:

determining classification information for each of the plurality of objects in the image by applying a predetermined joint classifier to at least one feature of a first type, the at least one feature being generated from the image using a first feature extractor, the classification information being based on a probability of each of a plurality of possible classifications;

estimating, for each of the feature types, an improvement in an accuracy of classification for each of the plurality of objects, the estimated improvement being formed using the determined classification information for each of the objects and a type of each of the objects in the image;

selecting features of a further type, from the plurality of feature types, according to the estimated improvement in the accuracy of the classification of each of the objects; and

classifying the plurality of objects in the image using the selected features of the further type.

14. The non-transitory computer readable storage medium according to claim 13, wherein the method further comprises determining a benefit of evaluating a particular feature type of the plurality of feature types based on the estimated improvement in accuracy of classification, wherein the benefit is determined according to at least one of:

normalising the estimated improvement in accuracy with a computation cost, such that the benefit is a measure of the estimated improvement in accuracy improvement per computation cost; and

using a trade-off across expected total accuracy improvement and a computation cost.

15. The non-transitory computer readable storage medium according to claim 13, wherein the estimating comprises determining a direct accuracy improvement and an indirect accuracy improvement associated with a particular feature, and summing the direct accuracy improvement and the indirect accuracy improvement to provide the estimated improvement in accuracy.

16. The non-transitory computer readable storage medium according to claim 15, wherein the determining of at least one of the direct accuracy improvement- and the indirect accuracy improvement-comprises linear regression of unprocessed features and the evaluation of the regression utilizes the determined classification information.

17. The non-transitory computer readable storage medium according to claim 16, wherein the determining of the direct accuracy improvement comprises non-linear regression of unprocessed features.

18. The non-transitory computer readable storage medium according to claim 13, wherein the estimated improvement is formed using a pair-wise interaction of the determined classification information for each of the objects and a type of each of the objects in the image.

19. A computer apparatus comprising a processor, a memory and a program recorded to the memory, the program being executable by the processor to perform a method to jointly classify a plurality of objects in data using a feature type selected from a plurality of feature types, the method comprising:

determining classification information for each of the plurality of objects in the data by applying a predetermined joint classifier to at least one feature of a first type, the at least one feature being generated from the data using a first feature extractor, the classification information being based on a probability of each of a plurality of possible classifications;

estimating, for each of the feature types, an improvement in an accuracy of classification for each of the plurality of objects, the estimated improvement being formed using the determined classification information for each of the objects and a type of each of the objects in the data;

selecting features of a further type, from the plurality of feature types, according to the estimated improvement in the accuracy of the classification of each of the objects; and

classifying the plurality of objects in the data using the selected features of the further type.

20. The computer apparatus according to claim 19, wherein the data is at least one of image data, audio data, text data, and image data from which text feature data is extracted by at least one of the feature extractors.