Devices, systems, and methods for learning a discriminant image representation

• US 9,275,306 B2
• Filed: 11/13/2013
• Issued: 03/01/2016
• Est. Priority Date: 11/13/2013
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Lu, Juwei, Dusberger, Daruisz T., Denney, Bradley Scott
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): WOLDEMARIAM, AKLILU K
• Application Number:
  US14/079,560
• Publication Number:
  US 20150131899A1
• Time in Patent Office: 839 Days
• Field of Search: 382/117, 382/118, 382/124, 382/159, 382/105, 382/170, 382/173, 382/195, 382/218, 382/224, 382/225
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/2132 based on discrimination criteria, e.g. discriminant analysis
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/24765 Rule-based classification
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations

Abstract

Systems, devices, and methods for generating an image representation obtain a set of low-level features from an image; generate a high-dimensional generative representation of the low-level features; generate a lower-dimensional representation of the low-level features based on the high-dimensional generative representation of the low-level features; generate classifier scores based on classifiers and on one or more of the high-dimensional generative representation and the lower-dimensional representation, wherein each classifier uses the one or more of the high-dimensional generative representation and the lower-dimensional representation as an input, and wherein each classifier is associated with a respective label; and generate a combined representation for the image based on the classifier scores and the lower-dimensional representation.

Claims

1. A method comprising:

obtaining a set of low-level features from each image in a set of images, wherein each image is associated with one or more labels in a set of labels;

generating a respective high-dimensional generative representation of each image based on the low-level features;

generating a respective lower-dimensional representation of each image based on the respective high-dimensional generative representation of the image;

generating respective classifiers for each label in the set of labels based on one or more of the high-dimensional generative representations of the images that are associate with the label and the lower-dimensional representations of the images that are associated with the label; and

generating a respective combined representation for each image, wherein the combined representation includes the respective classifiers of the labels that are associated with the respective image and includes the lower-dimensional representation of the respective image, and wherein the respective combined representation for an image is generated according to

{tilde over (x)}_ij=λ

·

w_i+(1−

λ

)·

x_ij,

where x_ij denotes the lower-dimensional representation of the image, where {tilde over (x)}_ij denotes the respective combined representation of the image, where w_i denotes the respective classifiers of the labels that are associated with the image, and where λ

is a regularization parameter.

2. The method of claim 1, wherein the higher-dimensional generative representation is generated with a Fisher-vector technique or a bag-of-visual-features technique.

3. The method of claim 1, wherein the lower-level representation is a discriminant representation that is generated using regularized linear-discriminant analysis.

4. The method of claim 1, wherein the combined representations are generated using vector addition.

5. A method comprising:

obtaining a query image;

generating a high-dimensional generative representation of low-level features the query image;

generating a lower-dimensional representation of the low-level features of the query image based on the high-dimensional generative representation; and

generating a comparison score for the query image and a reference image based on the lower-dimensional representation of the query image and on a combined representation of the reference image, wherein the combined representation of the reference image includes one or more classifiers and a lower-dimensional representation of the reference image, and wherein the combined representation of the reference image was generated according to

{tilde over (x)}_ij=λ

·

w_i+(1−

λ

)·

x_ij,

where x_ij is the lower-dimensional representation of the reference image, where w_i is a respective classifier of a label that is associated with the reference image, and where λ

is a weighting parameter.

6. The method of claim 5, wherein the one or more classifiers include a category vector.

7. The method of claim 5, wherein the comparison score can be described according to

κ

^on(x,x_ij)=λ

·

l_i(x)+(1−

λ

)·

s(x,x_ij),

where x is the lower-dimensional representation of the query image, where κ

^on(x,x_ij) is the comparison score, where s(x,x_ij) is a matching score between the lower-dimensional representation of the query image x and the lower-dimensional representation of the reference image x_ij, where l_i(x) is a classifier output, and where λ

is a weighting parameter.

8. The method of claim 7, wherein the matching score is a cosine distance, where s(x,x_ij)=x_ij^Tx, and where the lower-dimensional representation of the the query image x and the lower-demensional representation of the reference image x_ij have been pre-normalized by L2-normalization.

9. The method of claim 5, wherein the comparison score is generated by a dot-product operation.

10. The method of claim 5, wherein the comparison score is further based on a significance of the reference image, and wherein the significance of the reference image is a classification confidence g(x_ij) that can be computed according to

11. The method of claim 10, wherein the comparison score κ

(x,x_ij) can be described according to

κ

(x,x_ij)=λ

·

κ

^on(x,x_ij)+(1−

λ

)·

g(x_ij),

where κ

^on(x,x_ij) is a function that compares the lower-dimensional reprentation of the query image to the combined representation of the reference image, where g(x_ij) is the classification confidence, and where γ

is a weighting parameter.

15. The method of claim 6, wherein the category vector is a category mean vector of an LDA classifier or is a normal vector of an SVM classifier.

16. The method of claim 9, wherein the comparison score is generated according to

κ

^on(x,x_ij)={tilde over (x)}_ij^Tx,

where {tilde over (x)}_ij is the combined representation of the reference image and where x is the query image.

12. One or more computer-readable storage media storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:

obtaining low-level features from each image in a set of images, wherein each image is associated with one or more labels in a set of labels;

generating a high-dimensional representation for each image based on the low-level features of the respective image;

generating a lower-dimensional representation for each image based on the high-dimensional representation of the respective image;

generating a classifier for each label in the set of labels based on the high-dimensional representations of the respective images that are associated with the label or the lower-dimensional representations of the respective images that are associated with the label; and

generating a respective combined representation for each image, wherein the respective combined representation of an image includes the respective classifiers of the one or more labels that are associated with the respective image and includes the lower-dimensional representation of the respective image, and wherein the respective combined representation for an image is generated according to

{tilde over (x)}_ij=λ

·

w_i+(1−

λ

)·

x_ij,

where x_ij denotes the lower-dimensional representation of the image, where {tilde over (x)}_ij denotes the respective combined representation of the image, where w_i denotes the respective classifiers of the labels that are associated with the image and where λ

is a regularization parameter.

13. The method of claim 12, wherein the high-dimensional representations include a Fisher vector or a bag of visual features.

14. The method of claim 12, wherein the lower-dimensional representations are generated using regularized linear-discriminant analysis.