Ranking of images and image labels

• US 8,326,091 B1
• Filed: 05/09/2011
• Issued: 12/04/2012
• Est. Priority Date: 06/30/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jing, Yushi, Baluja, Shumeet
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US13/103,258
• Time in Patent Office: 575 Days
• Field of Search: 382/145, 382/305, 382/306, 382/311, 704/10, 707/3, 707/5, 707/7, 707/104.1, 707/752
• US Class Current: 382/305
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 18/2323 based on graph theory, e.g. minimum spanning trees [MST] or graph cuts
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

The subject matter of this specification can be embodied in, among other things, a method that includes determining a score for an image of a plurality of images with respect to each of one or more terms, identifying one or more of the terms for each of which the score for the image with respect to the respective identified term satisfies a criterion, and associating the identified terms with the image. Determining the score for the image with respect to a respective term includes determining probabilities of navigating between images in the plurality of images and determining the score for the image with respect to the respective term based on the probabilities.

Claims

1. A method comprising:

generating, by the computer system, initial scores for one or more terms and a plurality of images based on content that is associated with the plurality of images, wherein the initial scores indicate how closely the one or more terms are associated with the plurality of images in the content;

determining probabilities of navigating between images in the plurality of images, wherein the probabilities of navigating between images indicate how likely a user, who has viewed an image, is to perform an action that will cause another image to be displayed to the user;

determining, by the computer system, scores for the plurality of images and the one or more terms based on i) the probabilities of navigating between images and ii) the initial scores;

identifying a particular image from the plurality of images to associate with a particular term from the one or more terms based on the scores, wherein the content did not indicate an initial association between the particular image and the particular term based on a portion of the content that is associated with the particular image not including the particular term; and

providing, by the computer system, information that identifies the particular image and the particular term.

2. The method of claim 1, wherein the determining the probabilities of navigating between images in the plurality of images comprises:

determining degrees of similarity between one or more pairs of images in the plurality of images; and

wherein the probabilities of navigating between images in the plurality of images are determined based on the degrees of similarity.

3. The method of claim 2, wherein determining degrees of similarity between the one or more pairs of images in the plurality of images comprises, for a particular pair of images, identifying and comparing one or more features of each image in the particular pair of images.

4. The method of claim 2, wherein the probabilities of navigating between images in the plurality of images are further determined based on quality metrics of the images in the plurality of images.

5. The method of claim 4, wherein the quality metrics comprise one or more of a degree of focus of a respective image, a number of colors in a respective image, or a degree of saturation of a respective image.

6. The method of claim 2, wherein the probabilities of navigating between images in the plurality of images are further determined based on one or more relationship metrics between electronic documents that include or link to the plurality of images.

7. The method of claim 2, wherein the probabilities of navigating between images in the plurality of images are further determined based on relationship metrics between electronic documents that include or link to the plurality of images and quality metrics of the images in the plurality of images.

8. The method of claim 2, wherein a first probability of navigating between a first pair of images is determined to be greater than a second probability of navigating between a second pair of images based on determining that i) a first degree of similarity for the first pair is greater than a first threshold degree of similarity and is less than a second threshold degree of similarity and ii) a second degree of similarity for the second pair is greater than or equal to the second threshold degree of similarity.

9. The method of claim 1, wherein the content that is associated with the plurality of images includes one or more of:

metadata, filenames, textual content that is presented with one or more of the plurality of images in one or more electronic documents, anchor text for one or more links that link to one or more of the plurality of images, and linked-to content that is linked to by one or more of the plurality of images serving as anchors for links.

10. The method of claim 1, wherein a particular initial score from the initial scores is determined for the particular image based on a comparison of i) a number of times that the one or more terms occur in particular content that is associated with the particular image and ii) a number of times that the one or more terms occur in the content that is associated with the plurality of images.

11. The method of claim 1, wherein a particular initial score from the initial scores is determined for the particular image based on whether the one or more terms occurs in particular content that is associated with the particular image, the particular initial score being set to a first predefined value when the one or more terms occur in the content or to a second predefined value when the one or more terms do not occur in the content.

12. The method of claim 1, wherein

a particular score from the scores is determined for the particular image based on a combination of probability adjusted scores for other images from the plurality of images;

and

the probability adjusted scores for the other images comprise initial scores for the other images weighted based on corresponding probabilities of navigating between the particular image and the other images.

13. A system comprising:

one or more computing devices that each include one or more processors;

an image terms module of the one or more computing devices to generate initial scores for one or more terms and a plurality of images based on content that is associated with the plurality of images, wherein the initial scores indicate how closely the one or more terms are associated with the plurality of images in the content;

a probabilities module of the one or more computing devices to determine probabilities of navigating between images in the plurality of images wherein the probabilities of navigating between images indicate how likely a user, who has viewed an image, is to perform an action that will cause another image to be displayed to the user;

a scoring and ranking module of the one or more computing devices to;

determine scores for the plurality of images and the one or more terms based on i) the probabilities of navigating between images and ii) the initial scores, and

identify a particular image from the plurality of images to associate with a particular term from the one or more terms based on the scores, wherein the content did not indicate an initial association between the particular image and the particular term based on a portion of the content that is associated with the particular image not including the particular term; and

an image indexing module of the one or more computing devices to provide information that identifies the particular image and the particular term.

14. The system of claim 13, further comprising:

an image similarity module of the one or more computing devices to determine degrees of similarity between one or more pairs of images in the plurality of images; and

wherein the probabilities of navigating between images in the plurality of images are determined by the probabilities module using the degrees of similarity.

15. The system of claim 14, wherein the probabilities module is configured to determine the probabilities of navigating between the plurality of images further based on quality metrics of the images in the plurality of images.

16. The system of claim 14, wherein the probabilities module is configured to determine the probabilities of navigating between the plurality of images further based on one or more relationship metrics between electronic documents that include or link to the plurality of images.

17. The system of claim 14, wherein a first probability of navigating between a first pair of images is determined to be greater than a second probability of navigating between a second pair of images based on determining that i) a first degree of similarity for the first pair is greater than a first threshold degree of similarity and is less than a second threshold degree of similarity and ii) a second degree of similarity for the second pair is greater than or equal to the second threshold degree of similarity.

18. The system of claim 13, wherein a particular initial score from the initial scores is determined for the particular image based on a comparison of i) a number of times that the one or more terms occur in particular content that is associated with the particular image and ii) a number of times that the one or more terms occur in the content that is associated with the plurality of images.

19. The system of claim 13, wherein

a particular score from the scores is determined for the particular image based on a combination of probability adjusted scores for other images from the plurality of images;

and

the probability adjusted scores for the other images comprise initial scores for the other images weighted based on corresponding probabilities of navigating between the particular image and the other images.

20. A computer program product, tangibly encoded on a non-transitory computer-readable medium, operable to cause a data processing apparatus to perform operations comprising:

generating initial scores for one or more terms and a plurality of images based on content that is associated with the plurality of images, wherein the initial scores indicate how closely the one or more terms are associated with the plurality of images in the content;

determining probabilities of navigating between images in the plurality of images, wherein the probabilities of navigating between images indicate how likely a user, who has viewed an image, is to perform an action that will cause another image to be displayed to the user;

determining scores for the plurality of images and the one or more terms based on i) the probabilities of navigating between images and ii) the initial scores;

identifying a particular image from the plurality of images to associate with a particular term from the one or more terms based on the scores, wherein the content did not indicate an initial association between the particular image and the particular term based on a portion of the content that is associated with the particular image not including the particular term; and

providing information that identifies the particular image and the particular term.