Intelligent Image Search Results Summarization and Browsing

• US 20140321761A1
• Filed: 07/07/2014
• Published: 10/30/2014
• Est. Priority Date: 02/08/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Jingdong, Hua, Xian-Sheng
• Granted Patent: US 10,521,692 B2
• Time in Patent Office: Days
• US Class Current: 382/218
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: G06V 40/197 Matching; Classification

Abstract

Techniques for intelligent image search results summarization and browsing scheme are described. Images having visual attributes are evaluated for similarities based in part on their visual attributes. At least one preference score indicating a probability of an image to be selected into a summary is calculated for each image. Images are selected based on the similarity of the selected images to the other images and the preference scores of the selected images. A summary of the plurality of images is generated including the selected one individual image.

Claims

1. (canceled)

2. A method comprising:

receiving a plurality of images from one or more image providers in response to receiving a search term from a client, each of the images having at least one visual attribute;

selecting a particular image of the plurality of images based at least in part on a similarity of the particular image to other images of the plurality of images;

generating a summary of the plurality of images, the summary including the particular image; and

providing the summary of the plurality of images and the plurality of images to the client.

3. The method of claim 2, wherein the client displays the summary of the plurality of images in a first portion of a display and the plurality of images in a second portion of the display.

4. The method of claim 2, further comprising determining a local scaling parameter for the plurality of images by evaluating similarities between individual images of the plurality of images and other images of the plurality of images based in part on the at least one visual attribute and wherein the selecting the particular image of the plurality of images is based at least in part on the local scaling parameter.

5. The method of claim 2, further comprising calculating at least one preference score for the plurality of images, the at least one preference score indicating a probability for an associated image of the plurality of images to be selected into a summary of the plurality of images and wherein the selecting the particular image of the plurality of images is based at least in part on the at least one preference score.

6. The method of claim 2, further comprising calculating at least one a relevancy score based at least in part on sequence order of the images in the plurality of images and wherein the selecting the particular image of the plurality of images is based at least in part on the at least one relevancy score.

7. The method of claim 2, further comprising calculating at least image quality score based at least in part on a visual quality of individual ones of the plurality of images and wherein the selecting the particular image of the plurality of images is based at least in part on the at least one image quality score.

8. The method of claim 2, wherein the visual attribute includes at least one of a color entropy, a brightness, a blur, a block, a dynamic range, an intensity contrast, an image width, or an image height.

9. The method of claim 2, wherein the visual attributes include at least one of a color, a shape, or a texture.

10. The method of claim 2, wherein:

the client concurrently displays the summary of the plurality of images and the plurality of images; and

selected ones of the plurality of images displayed based at least in part on a strength of similarity between the images of the summary and individual ones of the plurality of images.

11. An device comprising:

one or more processors;

one or more computer-readable media storing instructions, which when executed by the one or more processors, cause the one or more processors to;

receive a plurality of images from one or more image provider;

evaluate similarities between individual ones of the plurality of images based in part on the at least one visual attribute of the plurality of images;

select a particular image form the plurality of images based on the similarity of the particular image to others of the plurality of images; and

generate a summary of the plurality of images, the summary including the particular image.

12. The device of claim 11, wherein receiving the plurality of images is based at least in part on receiving a search term from a client device.

13. The device of claim 11, wherein the one or more computer-readable media store instructions, which when executed by the one or more processors, cause the one or more processors to calculate preference scores for individual images of the plurality of images, the preference scores indicating a probability for the individual image to be selected into the summary of the plurality of images and wherein the selecting of the particular image is based at least in part on the preference scores.

14. The device of claim 13, wherein the one or more computer-readable media store instructions, which when executed by the one or more processors, cause the one or more processors to adjust the preference scores for the others of the plurality of the images based on the selection of the particular image for inclusion in the summary of the plurality of images.

15. The device of claim 11, wherein the one or more computer-readable media store instructions, which when executed by the one or more processors, cause the one or more processors to calculate a relevancy score based at least in part on a sequence associated with the plurality of images and wherein the selecting of the particular image is based at least in part on the relevancy score.

16. The device of claim 11, wherein the one or more computer-readable media store instructions, which when executed by the one or more processors, cause the one or more processors to calculate an image quality score based at least in part on a visual quality associated with individual ones of the plurality of images and wherein the selecting of the particular image is based at least in part on the image quality score.

17. The device of claim 11, wherein the one or more computer-readable media store instructions, which when executed by the one or more processors, cause the one or more processors to display the plurality of images concurrently with the particular image.

18. The device of claim 11, wherein the one or more computer-readable media store instructions, which when executed by the one or more processors, cause the one or more processors to provide the summary of the plurality of images and the plurality of images to a client device and wherein the client device displays at least one of the plurality of images concurrently with the summary of the plurality of images.

19. A computer readable storage device comprising instructions that when executed by one or more processors, cause the one or more processors to:

provide a user input to a server;

receive a plurality of images and one or more summary images, the one or more summary images representative of the plurality of images in response to providing the user input, individual ones of the one or more summary images selected based at least in part on a similarity between individual images and the images of the plurality of images; and

concurrently display the one or more summary images and at least one image of the plurality of images on a display.

20. The computer readable storage device of claim 19, wherein the at least one image of the plurality of images displayed is selected based at least in part on a strength of similarity between the one or more summary images and the at least one image of the plurality of images.

21. The computer readable storage device of claim 19, wherein the one or more summary images displayed a browsing path associated with similar images of the plurality of images in response to a user selection.