INTERACTIVELY RANKING IMAGE SEARCH RESULTS USING COLOR LAYOUT RELEVANCE

• US 20100158412A1
• Filed: 12/22/2008
• Published: 06/24/2010
• Est. Priority Date: 12/22/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hua, Xian Sheng, Zhao, Yinghai, Wang, Jingdong, Li, Shipeng
• Granted Patent: US 8,406,573 B2
• Time in Patent Office: Days
• US Class Current: 382/305
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60

Abstract

This disclosure describes various exemplary user interfaces, methods, and computer program products for the interactively ranking image search results refinement method using a color layout. The method includes receiving a text query for an image search, presenting image search results in a structured presentation based on the text query and information from an interest color layout. The process creates image search results that may be selected by the user based on color selection palettes or color layout specification schemes. Then the process ranks the image search results by sorting the results according to similarity scores between color layouts from the image search results and the interest color layout from a user based on the color selection palettes and the color layout specification schemes.

Claims

1. A user interface having computer-readable instructions that, when executed by a processor, cause the processor to perform acts comprising:

receiving a text query for an image search;

presenting image search results in a structured presentation based on the text query and information from an interest color layout;

enabling an interactive design of creating ranking image search results by selecting from color selection palettes and color layout specification schemes;

ranking image search results by sorting the image search results according to similarity scores between color layouts from the image search results and the interest color layout from a user based on the color selection palettes and the color layout specification schemes.

2. The user interface of claim 1, further comprising a checkbox feature for turning on a color layout ranking.

3. The user interface of claim 1, wherein the color selection palettes to indicate interest colors comprises:

a standard color palette, wherein colors are organized without using information from the image search results;

a query-dependent color palette, wherein colors that appear most-frequently in the image search results; and

a color palette generated from selecting images from the image search results.

4. The user interface of claim 1, wherein the color layout specification schemes comprises:

a free-scribbling to allow a user to freely draw strokes to indicate an interest layout;

a template layout to allow the user to select an interest layout from a set of templates, which are common for text queries;

query-dependent layouts to allow the user to select an interest layout from a set of query-dependent templates, which are mined from image search results corresponding to the text query; and

a layout from a sample image to allow the user to select a sample image from the image search results and to draw on the sample image to indicate the interest color layout.

5. The user interface of claim 1, further comprising a browser toolbar integrated with the ranking image search results, the toolbar enabling the user to view an item in the ranking image search results in a separate browser without interrupting the image search results.

6. The user interface of claim 1, further comprising saving an item from the ranking image search results into a file, enabling the user to open the item in the file for later use, move the file to another computing device, or to share the file with another user.

7. The user interface of claim 1, wherein the ranking image search results comprises selecting an item in the ranking image search results, which becomes highlighted with a box around the item.

8. A method for interactively ranking image search results by using a color layout relevance, implemented at least in part by a computing device, the method comprising:

extracting color layouts for images after the images are crawled from an internet;

receiving a text query for an image search;

evaluating a similarity between an interest color layout and a color layout of any image in an image search results, wherein the similarity is based on a color appearance compatibility, a spatial distribution consistency, and a contextual layout similarity, wherein the color appearance compatibility, the spatial distribution consistency, and the contextual layout similarity are based on a color similarity and a layout component-wise similarity; and

reordering the image search results based on evaluated similarities between color layouts of images and the interest color layout from the user according to a selection of a color selection palette and a color layout specification scheme.

9. The method of claim 8, wherein the color selection palettes comprises a standard color palette, a query dependent color palette, and a color palette using colors from image search results.

10. The method of claim 8, wherein the color layout specification scheme comprises at least one of:

a free-scribbling to allow a user to freely draw strokes to indicate an interest layout;

a template layout to allow the user to select an interest layout from a set of templates, which are common for text queries; and

query-dependent layouts to allow the user to select an interest layout from a set of query-dependent templates, which are mined from image search results corresponding to the text query; and

a layout from a sample image to allow the user to select a sample image from the search results and to draw on the sample image to indicate the interest color layout.

11. The method of claim 8, wherein the color appearance compatibility comprises a degree that the colors in the image corresponding to the interest layout region are similar to the colors in the interest color layout.

12. The method of claim 8, wherein the spatial distribution consistency comprises that the color appearing in the region corresponding to the interest color region should be less similar than the other interest colors.

13. The method of claim 8, wherein the contextual layout similarity comprises assigning different weights to a propagated color, the weights are set according to a distance of the grid containing the propagated color from seeded grids.

14. A computer-readable storage media comprising computer-readable instructions executed on a computing device, the computer-readable instructions comprising instructions for:

receiving a text query for an image search;

extracting a color layout;

measuring a similarity based on a color appearance compatibility, a spatial distribution consistency, a contextual layout similarity, a color similarity, and a layout component-wise similarity;

providing interactive options for the image search results with color selection palettes and color layout specification schemes;

reordering the image search results using a color layout ranking; and

presenting ranked image search results in a structured presentation based on the text query and information from the interest color layout.

15. The computer-readable storage media of claim 14, wherein the color appearance compatibility comprises a degree that the colors in the image corresponding to the interest layout region are similar to the colors in the interest color layout.

16. The computer-readable storage media of claim 14, wherein the spatial distribution consistency comprises that the color appearing in the region corresponding to the interest color region should be less similar than the other interest colors.

17. The computer-readable storage media of claim 14, wherein the contextual layout similarity comprises assigning different weights to a propagated color, the weights are set according to a distance of the grid containing the propagated color from seeded grids.

18. The computer-readable storage media of claim 14, wherein the color selection palettes to indicate interest colors comprises:

a standard color palette, wherein colors are organized without using information from the image search results;

a query-dependent color palette, wherein colors that appear most-frequently in the image search results; and

a color palette generated from selecting images from the image search results.

19. The computer-readable storage media of claim 14, wherein the color specification scheme comprises:

a free-scribbling to allow a user to freely draw strokes to indicate an interest layout;

a template layout to allow the user to select an interest layout from a set of templates, which are common for text queries;

query-dependent layouts to allow the user to select an interest layout from a set of query-dependent templates, which are mined from image search results corresponding to the text query; and

a layout from an example image to allow the user to select a sample image from the search results and to draw on the example image to indicate the interest color layout.

20. The computer-readable storage media of claim 14, further comprising a checkbox feature for turning on a color layout ranking.