Grouping of image search results

• US 9,116,921 B2
• Filed: 09/22/2014
• Issued: 08/25/2015
• Est. Priority Date: 09/03/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jing, Yushi, Rowley, Henry Allan, Chennapragada, Aparna
• Primary Examiner(s): NGUYEN, PHONG H
• Application Number:
  US14/492,515
• Publication Number:
  US 20150169635A1
• Time in Patent Office: 337 Days
• Field of Search: 707/723, 707/737, 707/748, 707/778, 707/829, 707/915, 707/956, 707/749, 707/E17.046
• US Class Current: 1/1
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06V 10/7625 Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms
• CPC Class Codes: G06V 10/7788 the supervisor being a human, e.g. interactive learning with a human teacher
• CPC Class Codes: G06V 10/945 User interactive design; Environments; Toolboxes

Abstract

This specification relates to presenting image search results. In general, one aspect of the subject matter described in this specification can be embodied in methods that include the actions of receiving an image query, the image query being a query for image search results; receiving ranked image search results responsive to the image query, the image search results each including an identification of a corresponding image resource; generating a similarity matrix for images identified by the image search results; generating a hierarchical grouping of the images using the similarity matrix; identifying a canonical image for each group in the hierarchical grouping using a ranking measure; and presenting a visual representation of the image search results based on the hierarchical grouping and the identified canonical images.

Claims

1. A method implemented by one or more computers comprising one or more processors, the method comprising:

receiving an image query;

receiving ranked image search results responsive the image query;

generating, by the one or more processors, a hierarchical grouping of the image search results based on measures of similarity between images identified by the image search results;

identifying, by the one or more processors, a respective canonical image for each of a plurality of groups in the hierarchical grouping based on a likelihood that users will interact with the canonical image when presented with multiple images from the group; and

providing, in response to the image query and by the one or more processors, data that present a visual representation of the identified canonical images wherein canonical images for lower levels of the hierarchy are presented smaller than canonical images for higher levels of the hierarchy.

2. The method of claim 1, further comprising:

determining that user interaction with one of the identified canonical images; and

in response to the determination, increasing a presentation size of images that are included in a same hierarchical grouping as the one of the canonical images.

3. The method of claim 2, further comprising:

in response to the determination, decreasing a presentation size of images that are included in the same hierarchical grouping as the one of the canonical images.

4. The method of claim 2, wherein increasing in presentation size of images that are included in a same hierarchical grouping as the one of the canonical images comprises increasing a presentation size of child or grandchild images of the one of the canonical images more than an amount by which a presentation size of the one of the canonical images is increased.

5. The method of claim 2, further comprising presenting, in response to the determination, another hierarchical grouping of images that was not presented prior to the user interaction with the one of the identified canonical images.

6. The method of claim 1, wherein the measures of similarity between the images is based on one or more of content-based signals, user behavior-based signals, or text-based signals.

7. The method of claim 1, wherein the likelihood that users will interact with the canonical image is based on a number of local descriptors that the canonical image has in common with the multiple images from the group.

8. A system, comprising:

data processing apparatus comprising a data storage device and one or more processors programmed to perform operations comprising;

receiving an image query;

receiving ranked image search results responsive the image query;

generating a hierarchical grouping of the image search results based on measures of similarity between images identified by the image search results;

identifying a respective canonical image for each of a plurality of groups in the hierarchical grouping based on a likelihood that users will interact with the canonical image when presented with multiple images from the group; and

providing, in response to the image query, data that present a visual representation of the identified canonical images wherein canonical images for lower levels of the hierarchy are presented smaller than canonical images for higher levels of the hierarchy.

9. The system of claim 8, wherein the one or more processors are programmed to perform operations further comprising:

determining that user interaction with one of the identified canonical images; and

in response to the determination, increasing a presentation size of images that are included in a same hierarchical grouping as the one of the canonical images.

10. The system of claim 9, wherein the one or more processors are programmed to perform operations further comprising:

in response to the determination, decreasing a presentation size of images that are included in the same hierarchical grouping as the one of the canonical images.

11. The system of claim 9, wherein increasing in presentation size of images that are included in a same hierarchical grouping as the one of the canonical images comprises increasing a presentation size of child or grandchild images of the one of the canonical images more than an amount by which a presentation size of the one of the canonical images is increased.

12. The system of claim 9, wherein the one or more processors are programmed to perform operations further comprising presenting, in response to the determination, another hierarchical grouping of images that was not presented prior to the user interaction with the one of the identified canonical images.

13. The system of claim 8, wherein the measures of similarity between the images is based on one or more of content-based signals, user behavior-based signals, or text-based signals.

14. The system of claim 8, wherein the likelihood that users will interact with the canonical image is based on a number of local descriptors that the canonical image has in common with the multiple images from the group.

15. A non-transitory computer readable storage medium storing instructions that, when executed by data processing apparatus, cause the data processing apparatus to perform operations comprising:

receiving an image query;

receiving ranked image search results responsive the image query;

generating a hierarchical grouping of the image search results based on measures of similarity between images identified by the image search results;

identifying a respective canonical image for each of a plurality of groups in the hierarchical grouping based on a likelihood that users will interact with the canonical image when presented with multiple images from the group; and

providing, in response to the image query, data that present a visual representation of the identified canonical images wherein canonical images for lower levels of the hierarchy are presented smaller than canonical images for higher levels of the hierarchy.

16. The computer readable medium of claim 15, wherein the instructions cause the data processing apparatus to perform operations further comprising:

determining that user interaction with one of the identified canonical images; and

in response to the determination, increasing a presentation size of images that are included in a same hierarchical grouping as the one of the canonical images.

17. The computer readable medium of claim 16, wherein the instructions cause the data processing apparatus to perform operations further comprising:

in response to the determination, decreasing a presentation size of images that are included in the same hierarchical grouping as the one of the canonical images.

18. The computer readable medium of claim 16, wherein increasing in presentation size of images that are included in a same hierarchical grouping as the one of the canonical images comprises increasing a presentation size of child or grandchild images of the one of the canonical images more than an amount by which a presentation size of the one of the canonical images is increased.

19. The computer readable medium of claim 16, wherein the instructions cause the data processing apparatus to perform operations further comprising presenting, in response to the determination, another hierarchical grouping of images that was not presented prior to the user interaction with the one of the identified canonical images.

20. The computer readable medium of claim 15, wherein the measures of similarity between the images is based on one or more of content-based signals, user behavior-based signals, or text-based signals.