Visual and Textual Query Suggestion

• US 20100205202A1
• Filed: 02/11/2009
• Published: 08/12/2010
• Est. Priority Date: 02/11/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zha, Zhengjun, Mei, Tao, Hua, Xian-Sheng, Wang, Meng, Yang, Linjun
• Granted Patent: US 8,452,794 B2
• Time in Patent Office: Days
• US Class Current: 707/767
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Techniques described herein enable better understanding of the intent of a user that submits a particular search query. These techniques receive a search request for images associated with a particular query. In response, the techniques determine images that are associated with the query, as well as other keywords that are associated with these images. The techniques then cluster, for each set of images associated with one of these keywords, the set of images into multiple groups. The techniques then rank the images and determine a representative image of each cluster. Finally, the tools suggest, to the user that submitted the query, to refine the search based on user selection of a keyword and a representative image. Thus, the techniques better understand the user'"'"'s intent by allowing the user to refine the search based on another keyword and based on an image on which the user wishes to focus the search.

Claims

1. One or more computer-readable media storing computer-executable instructions that, when executed on one or more processors, perform acts comprising:

receiving, from a user, a search request for images associated with a textual query;

determining multiple images that are associated with the received textual query;

determining a first set of the multiple images that are associated with a first keyword and determining a second set of the multiple images that are associated with a second keyword;

clustering the first set of the multiple images into two or more clusters and clustering the second set of the multiple images into two or more clusters;

determining;

(i) an image that is representative of the first cluster of the first set of the multiple images;

(ii) an image that is representative of the second cluster of the first set of the multiple images;

(iii) an image that is representative of the first cluster of the second set of the multiple images, and (iv) an image that is representative of the second cluster of the second set of the multiple images;

suggesting, to the user, to refine the search request based on the first or the second keyword and based on one of the representative images; and

responsive to receiving a selection of the first or the second keyword and a representative image, refining the search request based on the selected keyword and based on the selected representative image.

2. One or more computer-readable media as recited in claim 1, wherein the refining of the search request comprises:

ranking images that are associated with the received textual query based on a similarity of each of the images to;

(i) the selected keyword, and (ii) the selected representative image; and

outputting one or more of the images based at least in part on the ranking.

3. One or more computer-readable media as recited in claim 1, wherein the first set of the multiple images defines a subset of images that reflect a first aspect associated with the textual query and the second set of the multiple images defines a subset of images that reflect a second aspect associated with the textual query that is different than the first aspect.

4. One or more computer-readable media as recited in claim 1, wherein the suggesting comprises causing display of the first and the second keywords and the images that are each representative of a respective cluster, and wherein the receiving of the selection comprises receiving a selection of a displayed representative image.

5. One or more computer-readable media storing computer-executable instructions that, when executed on one or more processors, perform acts comprising:

receiving a request for images that are associated with a query;

determining images that are associated with the received query;

determining multiple keywords that are each associated with a portion of the determined images;

for each of the multiple keywords, determining at least one image that is representative of a respective portion of the determined images; and

suggesting to refine the request for images based on selection of one of the multiple keywords and selection of a representative image.

6. One or more computer-readable media as recited in claim 5, wherein each of the determined multiple keywords defines a different aspect of the received query.

7. One or more computer-readable media as recited in claim 5, wherein the determining of at least one image that is representative of a respective portion of the determined images comprises:

for each respective portion of the multiple images that is associated with a respective keyword, clustering the portion of the multiple images into at least a first group of images and a second group of images; and

determining an image that is representative of the first group and an image that is associated with the second group.

8. One or more computer-readable media as recited in claim 7, wherein the suggesting to refine the request comprises providing, for each respective portion of the multiple images, the image that is representative of the first group and the image that is representative of the second group.

9. One or more computer-readable media as recited in claim 5, further storing computer-executable instructions that, when executed on the one or more processors, perform acts comprising:

receiving a selection of one of the multiple keywords and a selection of a representative image; and

responsive to the receiving of the selection, refining the request based on the selected keyword and the selected representative image.

10. One or more computer-readable media as recited in claim 9, wherein the refining of the request comprises:

determining a similarity between;

(i) each of multiple images that are associated with the received query, and (ii) the selected keyword;

determining a similarity between;

(i) each of the multiple images that are associated with the received query, and (ii) the selected representative image; and

ranking the multiple images that are associated with the received query based at least in part on the determined similarities.

11. One or more computer-readable media as recited in claim 10, wherein the similarity between each image and the selected representative image is determined based on color similarity, texture similarity, or shape similarity.

12. One or more computing devices, comprising:

one or more processors; and

the one or more computer-readable media storing the computer-executable instructions as recited in claim 5.

13. One or more computer-readable media storing computer-executable instructions that, when executed on one or more processors, perform acts comprising:

receiving a request for images associated with a query;

determining images that are associated with the received query;

determining keywords that are associated with the determined images; and

suggesting to refine the request based on selection of a keyword and selection of an image that is associated with the selected keyword.

14. One or more computer-readable media as recited in claim 13, wherein the determining of the keywords that are associated with the determined images comprises analyzing tags that a community of users has associated with the images that are associated with the received query.

15. One or more computer-readable media as recited in claim 13, wherein the determining of the keywords that are associated with the determined images comprises:

determining a first keyword that is associated with multiple images of the determined images, the first keyword reflecting a first aspect of the received query; and

determining a second keyword that is associated with multiple images of the determined images, the second keyword reflecting a second aspect of the received query that is different than the first aspect.

16. One or more computer-readable media as recited in claim 13, wherein each of the determined keywords is associated with a group of images of the images that are associated with the received query, and further storing computer-executable instructions that, when executed on the one or more processors, perform acts comprising:

clustering, for each group of images associated with a keyword, the respective group into multiple clusters;

for each cluster of the multiple clusters, determining an image of the cluster that is representative of the cluster.

17. One or more computer-readable media as recited in claim 16, wherein the suggesting to refine the request comprises providing, for user selection, the multiple keywords and each image that is representative of a cluster of a group of images associated with a keyword.

18. One or more computer-readable media as recited in claim 13, further storing computer-executable instructions that, when executed on the one or more processors, perform acts comprising:

responsive to the suggesting to refine the request, receiving a selection of an image that is associated with a keyword;

responsive to the receiving of the selection;

determining a similarity between;

(i) each of multiple images that are associated with the received query, and (ii) the selected keyword; and

determining a similarity between;

(i) each of multiple images that are associated with the received query, and (ii) the selected image that is associated with the keyword.

19. One or more computer-readable media as recited in claim 18, further storing computer-executable instructions that, when executed on the one or more processors, perform acts comprising:

ranking at least a portion of the multiple images that are associated with the received query based on the determined similarities; and

outputting images for user selection at least in part based upon the ranking.

20. One or more computer-readable media as recited in claim 13, wherein the determining of keywords that are associated with the determined images comprises analyzing a query log that stores queries previously received from other users, and wherein the suggesting to refine the request comprises suggesting a keyword found within the analyzed query log.