Clustering queries for image search

• US 8,745,059 B1
• Filed: 05/29/2012
• Issued: 06/03/2014
• Est. Priority Date: 05/11/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jing, Yushi, Covell, Michele, Holiday, Stephen Conor
• Primary Examiner(s): GOFMAN, ALEX N
• Application Number:
  US13/482,343
• Time in Patent Office: 735 Days
• Field of Search: 707/737
• US Class Current: 707/737
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/5838 using colour

Abstract

Aspects of the subject matter described herein relate to functions used for retrieving image results based on search queries. More specifically, image search queries can be pre-grouped or classified based on visual and semantic similarity. For example, a pairwise image similarity value for a pair of queries can be computed based on one or more of the sum of all of the overlapping the image results, the sum of the image distances between all of the pairs of images in the image results, and the rank of each of the images in the image results. The pairwise image similarity values can then be used to generate image query clusters. Each image query clusters can include a set of queries with high pairwise image similarity values. In some examples, a distance function can be determined for each image query cluster. This data can be used to provide image results.

Claims

1. A computer-implemented method comprising:

obtaining a pair of search queries including (i) a first search query and (ii) a second search query;

obtaining (i) a first set of ranked images that are identified as responsive to the first search query, and (ii) a second set of ranked images that are identified as responsive to the second search query;

for each of the images of the first set of images, determining a respective image distance between the image of the first set of images and each image of the second set of images;

generating a score for each pair of images that includes an image from the first set of images and an image from the second set of images, wherein the score for each pair of images is based at least on (i) the rank of the image of the first set of images, (ii) the rank of the image of the second set of images, and (iii) the determined image distance between the image of the first set of images and the image of the second set of images;

determining a measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query based on the scores; and

assigning the pair of search queries to a cluster of search queries based at least on the measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query.

2. The method of claim 1, wherein determining a respective image distance between the image of the first set of images and each image of the second set of images, comprises:

determining the respective image distance based on comparing the image of the first set of images and each image of the second set of images at a pixel level.

3. The method of claim 1, wherein assigning the pair of search queries to a cluster of search queries based at least on the measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query comprises:

determining that the measure of similarity satisfies a similarity threshold; and

in response to determining that the measure of similarity satisfies the similarity threshold, assigning the pair of search queries to the cluster of search queries.

4. The method of claim 1, comprising:

generating a distance function associated with the cluster of search queries based on images that are identified as responsive to the search queries in the cluster of search queries.

5. The method of claim 4, wherein generating a distance function associated with the cluster of search queries based on images that are identified as responsive to the search queries in the cluster of search queries comprises:

determining a reference image for the cluster of search queries from the images that are identified as responsive to the search queries in the cluster of search queries;

determining a set of secondary images from the cluster of search queries from the images that are identified as responsive to the search queries in the cluster of search queries;

determining a respective image distance between the reference image and each image of the set of secondary images;

determining the distance function associated with the cluster of search queries based on the determined image distances between the reference image and each image of the set of secondary images.

6. The method of claim 4, comprising:

receiving a third search query;

identifying the cluster of search queries as responsive to the third search query based on one or more search terms of the third search query; and

identifying a set of search results based on the generated distance function associated with the cluster of search queries.

7. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

obtaining a pair of search queries including (i) a first search query and (ii) a second search query;

obtaining (i) a first set of ranked images that are identified as responsive to the first search query, and (ii) a second set of ranked images that are identified as responsive to the second search query;

for each of the images of the first set of images, determining a respective image distance between the image of the first set of images and each image of the second set of images;

generating a score for each pair of images that includes an image from the first set of images and an image from the second set of images, wherein the score for each pair of images is based at least on (i) the rank of the image of the first set of images, (ii) the rank of the image of the second set of images, and (iii) the determined image distance between the image of the first set of images and the image of the second set of images;

determining a measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query based on the scores; and

assigning the pair of search queries to a cluster of search queries based at least on the measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query.

8. The system of claim 7, wherein determining a respective image distance between the image of the first set of images and each image of the second set of images, comprises:

determining the respective image distance based on comparing the image of the first set of images and each image of the second set of images at a pixel level.

9. The system of claim 7, wherein assigning the pair of search queries to a cluster of search queries based at least on the measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query comprises:

determining that the measure of similarity satisfies a similarity threshold; and

in response to determining that the measure of similarity satisfies the similarity threshold, assigning the pair of search queries to the cluster of search queries.

10. The system of claim 7, the operations comprising:

generating a distance function associated with the cluster of search queries based on images that are identified as responsive to the search queries in the cluster of search queries.

11. The system of claim 10, wherein generating a distance function associated with the cluster of search queries based on images that are identified as responsive to the search queries in the cluster of search queries comprises:

determining a reference image for the cluster of search queries from the images that are identified as responsive to the search queries in the cluster of search queries;

determining a set of secondary images from the cluster of search queries from the images that are identified as responsive to the search queries in the cluster of search queries;

determining a respective image distance between the reference image and each image of the set of secondary images;

determining the distance function associated with the cluster of search queries based on the determined image distances between the reference image and each image of the set of secondary images.

12. The system of claim 10, the operations comprising:

receiving a third search query;

identifying the cluster of search queries as responsive to the third search query based on one or more search terms of the third search query; and

identifying a set of search results based on the generated distance function associated with the cluster of search queries.

13. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

obtaining a pair of search queries including (i) a first search query and (ii) a second search query;

obtaining (i) a first set of ranked images that are identified as responsive to the first search query, and (ii) a second set of ranked images that are identified as responsive to the second search query;

for each of the images of the first set of images, determining a respective image distance between the image of the first set of images and each image of the second set of images;

generating a score for each pair of images that includes an image from the first set of images and an image from the second set of images, wherein the score for each pair of images is based at least on (i) the rank of the image of the first set of images, (ii) the rank of the image of the second set of images, and (iii) the determined image distance between the image of the first set of images and the image of the second set of images;

determining a measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query based on the scores; and

assigning the pair of search queries to a cluster of search queries based at least on the measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query.

14. The medium of claim 13, wherein determining a respective image distance between the image of the first set of images and each image of the second set of images, comprises:

determining the respective image distance based on comparing the image of the first set of images and each image of the second set of images at a pixel level.

determining a measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query based on the scores; and

assigning the pair of search queries to a cluster of search queries based at least on the measure of similarity between (i) the first set of images that are identified as responsive to the first search query, and (ii) the second set of images that are identified as responsive to the second search query.