Image ordering system optimized via user feedback

• US 8,285,052 B1
• Filed: 12/15/2009
• Issued: 10/09/2012
• Est. Priority Date: 12/15/2009
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Khosla, Deepak, Bhattacharyya, Rajan, Huber, David J., Tasinga, Penn
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): SHAH, UTPAL D
• Application Number:
  US12/653,561
• Time in Patent Office: 1,029 Days
• Field of Search: 382/190, 382218-220
• US Class Current: 382/190
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content

Abstract

Described is a system for ordering images. The system receives a plurality of images. Image features are extracted from each image. A set of all possible image pairs are generated for all images. A similarity metric with weights is generated between the images in each image pair in the set, with a net similarity metric thereafter generated by combining the similarity metrics. The images are then ordered according to the net similarity metrics to generate a computer-ordered set of images. The computer-ordered set of images is then displayed to the user, which allows the user to re-order the images to generate a user-ordered set of images. The weights are then optimized to minimize the distance between the computer-ordered set of images and the user-ordered set of images. The similarity metrics are then re-weighted, with the images thereafter being re-ordered according to the new metrics.

Claims

1. A computer implemented method for ordering images, comprising an act of causing a computer having a processor to perform operations of

i. receiving, in the processor, a plurality of images;

ii. extracting, from each image, image features;

iii. generating a set of all possible image pairs for all images;

iv. generating at least one similarity metric between the images in each image pair in the set;

1. if more than one similarity metric exists between the images in each image pair in the set, then generating a net similarity metric by combining the similarity metrics;

2. if only one similarity metric exists between the images in each image pair in the set, then a net similarity metric is the similarity metric; and

v. ordering the plurality of images according to the net similarity metrics to generate a computer-ordered set of images.

2. The method as set forth in claim 1, wherein in generating at least one similarity metric, the similarity metric is a perceptual similarity metric, and further comprising an act of selecting the perceptual similarity metric from a group consisting of a gradient of an HSV feature space and a diffusion distance between histograms of HSV feature space.

3. The method as set forth in claim 2, wherein the act of generating a net similarity metric further comprises acts of:

i. weighting the perceptual similarity metrics according to predetermined weights to generate weighted perceptual similarity metrics; and

ii. summing the weighted perceptual similarity metrics into the net similarity metric.

4. The method as set forth in claim 3, further comprising an act of receiving user input to the computer-ordered set of images to generate a user-ordered set of images.

5. The method as set forth in claim 4, further comprising an act of optimizing the weights to generate optimized weights such that the distance between the computer-ordered set of images and the user-ordered set of images is minimized.

6. The method as set forth in claim 5, further comprising acts of:

i. weighting the perceptual similarity metrics according to optimized weights to generate re-weighted perceptual similarity metrics; and

ii. summing the re-weighted perceptual similarity metrics into a new net similarity metric;

iii. ordering the plurality of images according to the new net similarity metrics to generate a computer-ordered set of images; and

iv. displaying the computer-ordered set of images to a user;

v. receiving user input to the computer-ordered set of images to generate a user-ordered set of images;

vi. optimizing the weights to generate optimized weights such that the distance between the computer-ordered set of images and the user-ordered set of images is minimized; and

vii. repeating the acts of (i) through (vi) until the distance is less than a predetermined threshold or receipt of an optimized ordering as indicated by a user.

7. The method as set forth in claim 6, further comprising acts of

i. receiving, in the processor, a second plurality of images;

ii. extracting, from each image, image features;

iii. generating a set of all possible image pairs for all images;

iv. generating at least one similarity metric between the images in each image pair in the set;

1. if more than one similarity metric exists between the images in each image pair in the set, then generating a net similarity metric by combining the similarity metrics;

2. if only one similarity metric exists between the images in each image pair in the set, then a net similarity metric is the similarity metric; and

v. ordering the second plurality of images according to the net similarity metrics to generate a computer-ordered set of images.

8. The method as set forth in claim 7, wherein in ordering the plurality of images according to the net similarity metrics, the images are ordered such that images that are most similar are ordered next to one another.

9. The method as set forth in claim 7, wherein in ordering the plurality of images according to the net similarity metrics, the images are ordered such that images that are most dissimilar are ordered next to one another.

10. A system for ordering images, comprising a processor configured to perform operations of:

i. receiving, in the processor, a plurality of images;

ii. extracting, from each image, image features;

iii. generating a set of all possible image pairs for all images;

iv. generating at least one similarity metric between the images in each image pair in the set;

1. if more than one similarity metric exists between the images in each image pair in the set, then generating a net similarity metric by combining the similarity metrics;

2. if only one similarity metric exists between the images in each image pair in the set, then a net similarity metric is the similarity metric; and

v. ordering the plurality of images according to the net similarity metrics to generate a computer-ordered set of images.

11. The system as set forth in claim 10, wherein in generating at least one similarity metric, the similarity metric is a perceptual similarity metric, and the perceptual similarity metric is selected from a group consisting of a gradient from an HSV feature space and a diffusion distance between histograms of HSV feature space.

12. The system as set forth in claim 11, wherein in performing the operation of generating a net similarity metric, the processor is further configured to perform operations of

i. weighting the perceptual similarity metrics according to predetermined weights to generate weighted perceptual similarity metrics;

and

ii. summing the weighted perceptual similarity metrics into the net similarity metric.

13. The system as set forth in claim 12, wherein the processor is further configured to perform an operation of receiving user input to the computer-ordered set of images to generate a user-ordered set of images.

14. The system as set forth in claim 13, wherein the processor is further configured to perform an operation of optimizing the weights to generate optimized weights such that the distance between the computer-ordered set of images and the user-ordered set of images is minimized.

15. The system as set forth in claim 14, wherein the processor is further configured to perform operations of:

i. weighting the perceptual similarity metrics according to optimized weights to generate re-weighted perceptual similarity metrics; and

ii. summing the re-weighted perceptual similarity metrics into a new net similarity metric;

iii. ordering the plurality of images according to the new net similarity metrics to generate a computer-ordered set of images; and

iv. displaying the computer-ordered set of images to a user;

v. receiving user input to the computer-ordered set of images to generate a user-ordered set of images;

vi. optimizing the weights to generate optimized weights such that the distance between the computer-ordered set of images and the user-ordered set of images is minimized; and

vii. repeating the operations of (i) through (vi) until the distance is less than a predetermined threshold or receipt of an optimized ordering as indicated by a user.

16. The system as set forth in claim 15, wherein the processor is further configured to perform operations of:

i. receiving, in the processor, a second plurality of images;

ii. extracting, from each image, image features;

iii. generating a set of all possible image pairs for all images;

iv. generating at least one similarity metric between the images in each image pair in the set;

1. if more than one similarity metric exists between the images in each image pair in the set, then generating a net similarity metric by combining the similarity metrics;

2. if only one similarity metric exists between the images in each image pair in the set, then a net similarity metric is the similarity metric; and

v. ordering the second plurality of images according to the net similarity metrics to generate a computer-ordered set of images.

17. A computer program product for ordering images, the computer program product comprising computer-readable instruction means stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of:

i. receiving, in the processor, a plurality of images;

ii. extracting, from each image, image features;

iii. generating a set of all possible image pairs for all images;

iv. generating at least one similarity metric between the images in each image pair in the set;

1. if more than one similarity metric exists between the images in each image pair in the set, then generating a net similarity metric by combining the similarity metrics;

2. if only one similarity metric exists between the images in each image pair in the set, then a net similarity metric is the similarity metric; and

v. ordering the plurality of images according to the net similarity metrics to generate a computer-ordered set of images.

18. The computer program product as set forth in claim 17, wherein in generating at least one similarity metric, the similarity metric is a perceptual similarity metric, and the perceptual similarity metric is selected from a group consisting of a gradient from an HSV feature space and a diffusion distance between histograms of HSV feature space.

19. The computer program product as set forth in claim 18, further comprising instruction means for causing the processor to perform operations of

i. weighting the perceptual similarity metrics according to predetermined weights to generate weighted perceptual similarity metrics;

and

ii. summing the weighted perceptual similarity metrics into the net similarity metric.

20. The computer program product as set forth in claim 19, further comprising instruction means for causing the processor to perform an operation of receiving user input to the computer-ordered set of images to generate a user-ordered set of images.

21. The computer program product as set forth in claim 20, further comprising instruction means for causing the processor to perform an operation of optimizing the weights to generate optimized weights such that the distance between the computer-ordered set of images and the user-ordered set of images is minimized.

22. The computer program product as set forth in claim 21, further comprising instruction means for causing the processor to perform operations of

i. weighting the perceptual similarity metrics according to optimized weights to generate re-weighted perceptual similarity metrics;

and

ii. summing the re-weighted perceptual similarity metrics into a new net similarity metric;

iii. ordering the plurality of images according to the new net similarity metrics to generate a computer-ordered set of images; and

iv. displaying the computer-ordered set of images to a user;

v. receiving user input to the computer-ordered set of images to generate a user-ordered set of images;

vi. optimizing the weights to generate optimized weights such that the distance between the computer-ordered set of images and the user-ordered set of images is minimized; and

vii. repeating the operations of (i) through (vi) until the distance is less than a predetermined threshold or receipt of an optimized ordering as indicated by a user.

23. The computer program product as set forth in claim 22, further comprising instruction means for causing the processor to perform operations of

i. receiving, in the processor, a second plurality of images;

ii. extracting, from each image, image features;

iii. generating a set of all possible image pairs for all images;

iv. generating at least one similarity metric between the images in each image pair in the set;

1. if more than one similarity metric exists between the images in each image pair in the set, then generating a net similarity metric by combining the similarity metrics;

2. if only one similarity metric exists between the images in each image pair in the set, then a net similarity metric is the similarity metric; and

v. ordering the second plurality of images according to the net similarity metrics to generate a computer-ordered set of images.