Devices, systems, and methods for visual-attribute refinement

• US 9,158,970 B2
• Filed: 11/11/2013
• Issued: 10/13/2015
• Est. Priority Date: 11/16/2012
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Zhang, Liyan, Lu, Juwei, Denney, Bradley Scott
• Primary Examiner(s): COUSO, JOSE L
• Application Number:
  US14/077,079
• Publication Number:
  US 20140140625A1
• Time in Patent Office: 701 Days
• US Class Current: 1/1
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Systems, devices, and methods for generating attribute scores obtain a plurality of object images; generate a respective first attribute score of a first attribute for each object image in the plurality of object images based on the object images; calculate a respective pairwise object-similarity measure for pairs of object images in the plurality of object images; and refine the first attribute score of an object image in the plurality of object images based at least in part on the attribute scores of other object images in the plurality of object images and on the object-similarity measures of the pairs of object images in the plurality of object images.

Claims

1. A method for refining attribute scores, the method comprising:

obtaining a plurality of object images;

generating a respective first attribute score of a first attribute for each object image in the plurality of object images based on the respective object image and an attribute classifier;

calculating a respective pairwise object-similarity measure for pairs of object images in the plurality of object images by comparing image features; and

refining the first attribute score of a processing target object image based on an attribute score of an object image that is highly similar, based on the respective pairwise object-similarity measure, to the processing target object image.

2. The method of claim 1, further comprising:

generating an object-similarity relation based at least in part on the pairwise object-similarity measures,

wherein refining the first attribute score of the processing target object image includes performing a random walk on the object-similarity relation.

3. The method of claim 1, further comprising

generating a second attribute score of a second attribute for each object image in the plurality of object images based on the object images,

wherein refining the first attribute score of the processing target object image is further based on a correlation of the first attribute and the second attribute.

4. The method of claim 3, further comprising

refining the second attribute score of the processing target object image based on the correlation of the first attribute and the second attribute, on the attribute scores of the other object images in the plurality of object images, and on the pairwise object-similarity measures of the pairs of object images in the plurality of object images.

5. The method of claim 1, further comprising

generating refined pairwise object-similarity measures for the pairs of object images;

and

further refining the first attribute score and a second attribute score of an object image based at least in part on refined attribute scores of other object images in the plurality of object images and on the refined pairwise object-similarity measures of the pairs of object images.

6. The method of claim 5, wherein generating the refined pairwise object-similarity measures for the pairs of object images includes projecting high-dimensional features of the object images to a lower-dimensional space.

7. A system for refining attribute scores, the system comprising:

a computer-readable storage medium; and

one or more processors that are coupled to the computer-readable storage medium and that are configured to cause the system to

generate respective attribute scores of a plurality of attributes for each object image in a plurality of object images based on the object images and on attribute classifiers,

generate respective pairwise object-similarity measures for pairs of object images in the plurality of object images by comparing image features, and

generate refined attribute scores of target object images based on the attribute scores of respective object images in the plurality of object images that, as indicated by the pairwise object-similarity measures, are highly similar to the target object images.

8. The system of claim 7, wherein the one or more processors are further configured to cause the system to generate the refined attribute scores further based on attribute correlations of the plurality of attributes.

9. The system of claim 8, wherein the one or more processors are further configured to cause the system to generate refined pairwise object-similarity measures for the pairs of object images in the plurality of object images based on the refined attribute scores.

10. The system of claim 9, wherein the one or more processors are further configured to cause the system to generate further-refined attribute scores of the object images based on the refined attribute scores of the object images and on the refined pairwise object-similarity measures of the pairs of object images.

11. The system of claim 10, wherein the one or more processors are further configured to cause the system to generate further-refined attribute scores and further-refined pairwise object-similarity measures until a convergence condition is satisfied.

12. The system of claim 7, wherein the one or more processors are further configured to cause the system to generate an object-similarity graph based on the pairwise object-similarity measures.

13. The system of claim 7, wherein the one or more processors are further configured to cause the system to retrieve a target object image based on the refined attribute scores of the target object image.

14. The system of claim 13, wherein, to retrieve the target object image based on the refined attribute scores of the target object image, the one or more processors are further configured to cause the system to weight the refined attribute scores of the target object image.

15. One or more computer-readable storage media storing instructions that, when executed by one or more computing device, cause the one or more computing devices to perform operations comprising:

generating respective attribute scores of a plurality of attributes for each object image in a plurality of object images based on the object images and on attribute classifiers;

generating respective pairwise object-similarity measures for pairs of object images in the plurality of object images by comparing image features; and

generating refined attribute scores of target object images based on one or more of the attribute scores of respective object images that, as indicated by the pairwise object-similarity measures, are highly similar to the target object images.

16. The one or more computer-readable storage media of claim 15, wherein the refined attribute scores are generated further based on attribute correlations of the plurality of attributes, and

wherein the attribute correlations are correlations of stable attributes or persistent attributes.

17. The one or more computer-readable storage media of claim 16, wherein the attribute correlations are defined by an attribute-affinity matrix.

18. The one or more computer-readable storage media of claim 15, wherein the operations further comprise generating respective attribute-score probabilities for the plurality of attributes for each object image.

19. The one or more computer-readable storage media of claim 15, wherein the operations further comprise:

iteratively generating further-refined attribute scores of the object images based at least on previous attribute scores of the object images in the plurality of object images,

wherein the previous attribute scores are the refined attribute scores in a first iteration of generating the further-refined attribute scores.

20. The one or more computer-readable storage media of claim 19, wherein iteratively generating the further-refined attribute scores includes projecting the previous attribute scores to an attribute-score subspace.