Method and system for optimizing accuracy-specificity trade-offs in large scale visual recognition

• US 9,779,291 B2
• Filed: 10/12/2015
• Issued: 10/03/2017
• Est. Priority Date: 06/14/2012
• Status: Active Grant

• Current Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Inventors: Li, Fei-Fei, Deng, Jia, Krause, Jonathan, Berg, Alexander C.
• Primary Examiner(s): Dunphy, David F
• Application Number:
  US14/881,092
• Publication Number:
  US 20160162731A1
• Time in Patent Office: 722 Days
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

As visual recognition scales up to ever larger numbers of categories, maintaining high accuracy is increasingly difficult. Embodiment of the present invention include methods for optimizing accuracy-specificity trade-offs in large scale recognition where object categories form a semantic hierarchy consisting of many levels of abstraction.

Claims

1. A computerized method for classifying images, comprising:

receiving an image hierarchy;

receiving a first image of interest wherein the image includes at least one feature;

classifying the at least one feature of the image according to the image hierarchy;

generating a dual variable for a Lagrange function associated with the classification of the at least one feature, where the dual variable controls a trade-off between an information gain and an accuracy;

optimizing the classification of the at least one feature of the image using the dual variable controlling the trade-off between the information gain and the accuracy by generating a classifier that maximizes a relationship between the information gain and the accuracy with respect to the dual variable; and

generating at least one optimized classification of the at least one feature of the image for the first image of interest.

2. The computerized method of claim 1, wherein the image hierarchy is a semantic hierarchy.

3. The computerized method of claim 2, wherein the semantic hierarchy is a directed acyclic graph.

4. The computerized method of claim 2, wherein the semantic hierarchy further comprises a plurality of levels.

5. The computerized method of claim 2, wherein the semantic hierarchy comprises a plurality of nodes, wherein the plurality of nodes comprises a unique root node, a plurality of leaf nodes, and a plurality of internal nodes.

6. The computerized method of claim 5, wherein the plurality of leaf nodes corresponds to a set of mutually exclusive classifications of the at least one feature and each internal node in the plurality of internal nodes corresponds to a particular classification that is a union of classifications corresponding to at least one leaf node.

7. The computerized method of claim 6, wherein the unique root node has a maximum accuracy classification and the plurality of leaf nodes have the maximum information gain classification.

8. The computerized method of claim 6, wherein the plurality of internal nodes have a trade-off classification between the information gain and the accuracy.

9. The computerized method of claim 1, wherein the Lagrange function can be evaluated using the following expression:

L(f,λ

)=R(f)+λ

(Φ

(f)−

1−

∈

)

where L(f,λ

) is the Lagrange function, f is the classifier, λ

is the dual variable, R(f) is the information gain, Φ

(f) is the accuracy, and ∈

is an arbitrary accuracy value.

10. The computerized method of claim 1, wherein generating the at least one optimized classification further comprises classifying the at least one feature of the image for the first image of interest using the classifier that maximizes the relationship between the information gain and the accuracy with respect to the measure of uncertainty.

11. An image classification system, comprising:

a processor;

a memory;

where the processor is configured by software stored in memory to;

receive an image hierarchy;

receive a first image of interest wherein the image includes at least one feature;

classify the at least one feature of the image according to the image hierarchy;

generate a dual variable for a Lagrange function associated with the classification of the at least one feature, where the dual variable controls a trade-off between an information gain and an accuracy;

optimize the classification of the at least one feature of the image using the dual variable controlling the trade-off between the information gain and the accuracy by generating a classifier that maximizes a relationship between the information gain and the accuracy with respect to the dual variable; and

generate at least one optimized classification of the at least one feature of the image for the first image of interest.

12. The image classification system of claim 11, wherein the image hierarchy is a semantic hierarchy.

13. The image classification system of claim 12, wherein the semantic hierarchy is a directed acyclic graph.

14. The image classification system of claim 12, wherein the semantic hierarchy further comprises a plurality of levels.

15. The image classification system of claim 12, wherein the semantic hierarchy comprises a plurality of nodes, wherein the plurality of nodes comprises a unique root node, a plurality of leaf nodes, and a plurality of internal nodes.

16. The image classification system of claim 15, wherein the plurality of leaf nodes corresponds to a set of mutually exclusive classifications of the at least one feature and each internal node in the plurality of internal nodes corresponds to a particular classification that is a union of classifications corresponding to at least one leaf node.

17. The image classification system of claim 16, wherein the unique root node has a maximum accuracy classification and the plurality of leaf nodes have the maximum information qain classification.

18. The image classification system of claim 16, wherein the plurality of internal nodes have a trade-off classification between the information gain and the accuracy.

19. The image classification system of claim 11, wherein the Lagrange function can be evaluated using the following expression:

L(f,λ

)=R(f)+λ

(Φ

(f)−

1−

∈

)

where L(f,λ

) is the Lagrange function, f is the classifier, λ

is the dual variable, R(f) is the information gain, Φ

(f) is the accuracy, and E is an arbitrary accuracy value.

20. The image classification system of claim 19, wherein the processor is configured to generate at least one optimized classification of the at least one feature of the image for the first image of interest using the classifier that maximizes the relationship between the information gain and the accuracy with respect to the dual variable.