Image re-rank based on image annotations

• US 8,606,780 B2
• Filed: 07/08/2011
• Issued: 12/10/2013
• Est. Priority Date: 07/08/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hu, Rui, Wang, Xin-Jing, Kong, Xiao, Xu, Juan
• Primary Examiner(s): Richardson, James E
• Application Number:
  US13/179,258
• Publication Number:
  US 20130013591A1
• Time in Patent Office: 886 Days
• Field of Search: 707/730, 707/731, 707/750, 707/776, 707/915, 707/749
• US Class Current: 707/730
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

Search queries for images are received from users. An original order of responsive images to the query is determined. Duplicate images and words associated with the duplicate images are identified for each of the responsive images. Common words associated with the duplicate images are identified. The responsive images are annotated with the common words and an annotated order is determined. A re-ranked order is determined based on the original order and the annotated order. Responsive images are presented to the user in the re-ranked order.

Claims

1. Computer-storage media storing computer-useable instructions, that, when executed by a computing device, perform a method for re-ranking images based on image annotations, the method comprising:

receiving a search query for an image from a user;

determining an original order of responsive images to the search query;

identifying duplicate images and words associated with the duplicate images;

identifying common words of the words associated with the duplicate images;

annotating the responsive images with the common words;

determining an annotated order of the responsive images with annotations, the annotated order being determined based on a matching score that is estimated between the search query and the annotations;

determining a re-ranked order of the responsive images based on the original order and the annotated order; and

presenting the responsive images to the user in the re-ranked order.

2. The media of claim 1 wherein determining a re-ranked order of the responsive images comprises:

training a ranking mathematical model to determine a weight factor for the original order and the annotated order;

applying the respective weight factor to the original order and the annotated order to determine the re-ranked order.

3. The media of claim 1, wherein duplicate images includes near-duplicate images.

4. The media of claim 1 further comprising training a common word mathematical model to identify the common words.

5. The media of claim 1 further comprising counting co-occurrences of words associated with the duplicate images to identify the common words.

6. The media of claim 1 further comprising utilizing Bayesian analysis to identify the common words.

7. A computer system for re-ranking images based on image annotations, the computer system comprising a processor coupled to a computer-storage medium, the computer-storage medium having stored thereon a plurality of computer software components executable by the processor, the computer software components comprising:

a search query component for receiving a search query for an image from a user;

a ranking component for determining an original order for responsive images to the search query;

an annotation component for annotating the responsive images with annotations;

a re-ranking component for determining a re-ranked order for responsive images to the search query based on the original order and an annotated order, the annotated order being determined based on a matching score that is estimated between the search query and the annotations;

a results component for presenting the responsive images in the re-ranked order to the user;

a near-duplicates component for identifying near-duplicate images;

an extraction component for extracting text associated with the near-duplicate images; and

a common word component for identifying common words extracted by the extraction component, wherein the annotation component receives the common words from the common word component and annotates the responsive images with annotations comprised of the common words.

8. The computer system of claim 7, wherein a weight factor is applied to the original order and an annotated order to determine the re-ranked order.

9. The computer system of claim 8, wherein the weight factor is configurable.

10. The computer system of claim 8, further comprising a weight factor training component for training a ranking mathematical model to determine the weight factor.

11. The computer system of claim 7, wherein duplicate images includes near-duplicate images.

12. The computer system of claim 7, wherein the common word component trains a common word mathematical model to identify the common words.

13. A computerized method for re-ranking images based on image annotations, the method comprising:

receiving a search query for an image from a user;

determining an original order of responsive images to the search query;

identifying duplicate images and words associated with the duplicate images;

training a common word mathematical model to identify common words of the words associated with the duplicate images;

identifying the common words of the words associated with the duplicate images;

annotating the responsive images with the common words;

determining an annotated order of the responsive images with annotations, the annotated order being determined based on a matching score that is estimated between the search query and the annotations;

determining a re-ranked order of the responsive images based on the original order and the annotated order; and

presenting the responsive images to the user in the re-ranked order.

14. The computerized method of claim 13, further comprising training a ranking mathematical model to determine a weight factor for each of the original order and the annotated order.

15. The computerized method of claim 14, further comprising applying the respective weight factors to the original order and the annotated order.