Intelligent image search results summarization and browsing

• US 10,521,692 B2
• Filed: 07/07/2014
• Issued: 12/31/2019
• Est. Priority Date: 02/08/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Wang, Jingdong, Hua, Xian-Sheng
• Primary Examiner(s): Dulaney, Kathleen Y
• Application Number:
  US14/324,375
• Publication Number:
  US 20140321761A1
• Time in Patent Office: 2,003 Days
• Field of Search: 382224, 382225, 382305, 382219, 707737, 707766
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: G06V 40/197 Matching; Classification

Abstract

Techniques for intelligent image search results summarization and browsing scheme are described. Images having visual attributes are evaluated for similarities based in part on their visual attributes. At least one preference score indicating a probability of an image to be selected into a summary is calculated for each image. Images are selected based on the similarity of the selected images to the other images and the preference scores of the selected images. A summary of the plurality of images is generated including the selected one individual image.

Claims

1. A method, comprising:

receiving a plurality of images from one or more image providers in response to receiving a search term from a client, each of the plurality of images having at least one visual attribute;

determining a local scaling parameter for each image in the plurality of images by evaluating similarities between the images in the plurality of images based, at least in part, on the at least one visual attribute of each image in the plurality of images;

selecting a particular image from the plurality of images based, at least in part, on the local scaling parameter associated with the particular image;

comparing the local scaling parameter of the particular image to the local scaling parameter of each image in the plurality of images to determine a probability, for each image in the plurality of images, that each image will be grouped with the particular image, wherein the probability is based, at least in part, on a strength of similarity between the particular image and each individual image of the plurality of images and a determined preference score of each individual image of the plurality of images, the preference score including an image quality of each individual image in the plurality of images and a relevance score of each individual image in the plurality of images;

generating a summary of the plurality of images, the summary including the particular image;

for each image in the plurality of images, adjusting the probability that at least a portion of the image corresponds to the search term and will be grouped with the particular image based, at least in part, on the selecting of the particular image; and

providing the summary of the plurality of images and the plurality of images to the client.

2. The method of claim 1, wherein the client displays the summary of the plurality of images in a first portion of a display and the plurality of images in a second portion of the display.

3. The method of claim 1, wherein the relevance score is calculated based, at least in part, on sequence order of the images in the plurality of images.

4. The method of claim 1, wherein the image quality is calculated based, at least in part, on a visual quality of individual ones of the plurality of images.

5. The method of claim 1, wherein the at least one visual attribute includes at least one of a color entropy, a brightness, a blur, a block, a dynamic range, an intensity contrast, an image width, or an image height.

6. The method of claim 1, wherein the at least one visual attribute includes at least one of a color, a shape, or a texture.

7. The method of claim 1, wherein the client concurrently displays the summary of the plurality of images and the plurality of images.

8. The method of claim 1, wherein the summary of the plurality of images displays a browsing path associated with similar images of the plurality of images in response to a user selection.

9. The method of claim 1, wherein grouping of each image and the particular image is associated with a category.

10. A device, comprising:

one or more processors; and

one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to;

receive, in response to execution of a search term, a plurality of images from one or more image providers;

determine a local scaling parameter for each individual image of the plurality of images by evaluating similarities between each individual image of the plurality of images based, at least in part, on at least one visual attribute of each of the individual images of the plurality of images;

select a particular image from the plurality of images based, at least in part, on the local scaling parameter associated with the particular image and a determined quality score that indicates a visual quality of the particular image;

compare the local scaling parameter of the particular image to the local scaling parameter of each image in the plurality of images to determine a probability that each individual image in the plurality of images will be grouped with the particular image, wherein the probability is based, at least in part, on a strength of similarity between the particular image and each individual image of the plurality of images and a determined preference score of each individual image of the plurality of images, the preference score including an image quality of each individual image of the plurality of images and a relevance score of each individual image of the plurality of images;

generate a summary of the plurality of images, including the particular image; and

adjust the probability of each individual image in the plurality of images that each individual image is associated with the search term and will be grouped with the particular image for inclusion in the summary of the plurality of images.

11. The device of claim 10, wherein the relevance score is based, at least in part, on a sequence associated with the plurality of images and wherein the selecting of the particular image is based, at least in part, on the relevance score.

12. The device of claim 10, further comprising instructions for displaying the plurality of images concurrently with the particular image.

13. The device of claim 10, further comprising instructions for providing the summary of the plurality of images and the plurality of images to a client device, wherein the client device displays at least one of the plurality of images concurrently with the summary of the plurality of images.

14. The device of claim 10, wherein the summary of the plurality of images displays a browsing path associated with similar images of the plurality of images in response to a user selection.

15. A hardware computer readable storage device comprising instructions that, when executed by one or more processors, cause the one or more processors to:

provide a user input to a server;

receive a plurality of images from the server in response to the user input;

determine a plurality of local scaling parameters for individual images of the plurality of images by evaluating similarities between the individual images of the plurality of images using at least one visual attribute of the plurality of images;

select one or more summary images representative of the plurality of images based, at least in part, on the plurality of local scaling parameters of the individual images of the plurality of images;

compare individual ones of the plurality of local scaling parameters to determine a preference score for each image of the plurality of images, the preference score including an image quality score and a relevance score, the preference score indicating a probability, for each image of the plurality of images, that each image will be grouped with the one or more summary images;

adjust the probability of each individual image of the plurality of images that each individual image corresponds to the user input and will be grouped with the one or more summary images; and

concurrently display the one or more summary images and at least one individual image of the plurality of images on a display.

16. The hardware computer readable storage device of claim 15, wherein the plurality of local scaling parameters of the individual images of the plurality of images includes an image quality score based, at least in part, on a visual quality associated with the individual images of the plurality of images and wherein the selecting of the one or more summary images is based, at least in part, on the image quality score.

17. The hardware computer readable storage device of claim 15, wherein the one or more summary images display a browsing path associated with similar images of the plurality of images in response to a user selection.

18. The hardware computer readable storage device of claim 15, wherein the plurality of scaling parameters are used to adjust a similarity degree between the individual images of the plurality of images.

19. The hardware computer readable storage device of claim 15, wherein a strength of similarities between the individual images of the plurality of images includes a probability that the individual images of the plurality of images will be selected.

20. The hardware computer readable storage device of claim 15, wherein the one or more summary images and the at least one individual image are arranged according to the probability that each image will be grouped with the one or more summary images.