Image ranking based on abstract concepts

• US 8,532,377 B2
• Filed: 12/22/2010
• Issued: 09/10/2013
• Est. Priority Date: 12/22/2010
• Status: Expired due to Fees

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Skaff, Sandra, Marchesotti, Luca, Csurka, Gabriela
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): Niu, Feng
• Application Number:
  US12/975,712
• Publication Number:
  US 20120163710A1
• Time in Patent Office: 993 Days
• Field of Search: None
• US Class Current: 382/165
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene

Abstract

A system and method for ranking images are provided. The method includes receiving a query comprising a semantic part and an abstract part, retrieving a set of images responsive to the semantic part of the query, and computing first scores for the retrieved images in the set of retrieved images. The first score of an image can be based on a relevance of that image to the semantic part of the query (and not to the abstract part of the query). The method further includes identifying a chromatic concept model from a set of chromatic concept models. This identification can be based on the abstract part of the query (and not on the semantic part of the query). The chromatic concept model includes an optionally-weighted set of colors expressed in a perceptually uniform color space. For retrieved images in the set of retrieved images, the method includes computing a chromatic image model based on colors of the image, the chromatic image model comprising a weighted set of colors expressed in the perceptually uniform color space and computing a comparison measure between the chromatic image model and the chromatic concept model. The retrieved images are scored with respective second scores that are based on the computed comparison measures. The retrieved images are ranked based on a combined score for a respective retrieved image which is a function of the first and second scores.

Claims

1. A method for ranking images comprising:

receiving a query comprising a semantic part and an abstract part;

retrieving a set of images responsive to the semantic part of the query;

computing first scores for the retrieved images in the set of retrieved images, each first score being based on a relevance of a retrieved image to the semantic part of the query;

computing a chromatic concept model for each of a set of abstract concepts, the chromatic concept model comprising a Gaussian Mixture Model wherein observations for the Gaussian mixture model are colors of at least one color palette which is associated with the abstract concept expressed in a perceptually uniform color space;

identifying one of the set of chromatic concept models, based on the abstract part of the query;

for retrieved images in the set of retrieved images;

computing a chromatic image model based on colors of the image, the chromatic image model comprising a weighted set of colors expressed in the perceptually uniform color space, and

computing a comparison measure between the chromatic image model and the identified chromatic concept model;

computing second scores for the retrieved images that are based on the computed comparison measures; and

ranking the retrieved images based on a combined score for a respective retrieved image which is a function of the first and second scores.

2. The method of claim 1, wherein at least one of the computing of the first scores, computing of the second scores, and ranking of the retrieved images is performed with a computer processor.

3. The method of claim 1, further comprising, providing for a user to review at least some of the retrieved images.

4. The method of claim 3, wherein the method includes displaying at least some of the retrieved images in an arrangement which reflects the combined scores.

5. The method of claim 3, further comprising visualizing at least one of:

the chromatic concept model; and

at least one of the chromatic image models.

6. The method of claim 1, wherein the perceptually uniform color space is one which has a lightness component and two chromatic components.

10. The method of claim 1, wherein the computing of the chromatic image model comprises computing a Gaussian Mixture Model wherein observations for the Gaussian mixture model are colors of pixels of the image.

11. The method of claim 1, wherein the computing of the comparison measure between the chromatic image model and the chromatic concept model comprises computing a distance measure which is based on Euclidian distances in the perceptually uniform color space between the colors of the chromatic image model and colors of the chromatic concept model and the associated weights.

14. The method of claim 1, wherein the computing a combined score for retrieved images in the set as a function of the first and second scores comprises computing a weighed combination of the first and second scores.

15. The method of claim 1, further comprising displaying the images in a first arrangement based on the first scores and displaying the images in a second arrangement based on the combined scores.

16. The method of claim 1, wherein the method further comprises converting colors of at least one of the retrieved images from an original color space to the perceptually uniform color space prior to computing the image'"'"'s chromatic image model.

17. A computer program product comprising a non-transitory recording medium encoding instructions which, when executed by a computer, perform the method of claim 1.

18. A system for performing the method of claim 1, comprising memory which stores instructions for performing the method and a processor in communication with the memory for executing the instructions.

7. A method for ranking images comprising:

receiving a query comprising a semantic part and an abstract part;

retrieving a set of images responsive to the semantic part of the query;

computing first scores for the retrieved images in the set of retrieved images, each first score being based on a relevance of a retrieved image to the semantic part of the query;

identifying one of a set of chromatic concept models, based on the abstract part of the query, the identified chromatic concept model comprising an optionally-weighted set of colors expressed in a perceptually uniform color space which has a lightness component and two chromatic components;

for retrieved images in the set of retrieved images;

computing a chromatic image model based on colors of the image, the chromatic image model comprising a weighted set of colors expressed in the perceptually uniform color space, and wherein the colors of the chromatic image model are expressed only in the chromatic components of the perceptually uniform color space, and

computing a comparison measure between the chromatic image model and the identified chromatic concept model;

computing second scores for the retrieved images that are based on the computed comparison measures; and

ranking the retrieved images based on a combined score for a respective retrieved image which is a function of the first and second scores.

8. The method of claim 7, further comprising computing a chromatic concept model for each of a set of abstract concepts.

9. The method of claim 8, wherein the computing of the chromatic concept model for each of the set of abstract concepts comprises computing a Gaussian Mixture Model wherein observations for the Gaussian mixture model are colors of at least one color palette which is associated with the abstract concept.

12. A method for ranking images comprising:

receiving a query comprising a semantic part and an abstract part;

retrieving a set of images responsive to the semantic part of the query;

computing first scores for the retrieved images in the set of retrieved images, each first score being based on a relevance of a retrieved image to the semantic part of the query;

identifying one of a set of chromatic concept models, based on the abstract part of the query, the identified chromatic concept model comprising an optionally-weighted set of colors expressed in a perceptually uniform color space;

for retrieved images in the set of retrieved images;

computing a chromatic image model based on colors of the image, the chromatic image model comprising a weighted set of colors expressed in the perceptually uniform color space, and

computing a comparison measure between the chromatic image model and the identified chromatic concept model, wherein the computing of the comparison measure between the chromatic image model and the chromatic concept model comprises computing a Color-based Earth Mover'"'"'s Distance (CEMD) according to the expression;

13. The method of claim 12, wherein the computing of the comparison measure comprises minimizing: