COLORBLIND ACCESSIBLE IMAGE SEARCH

• US 20100185624A1
• Filed: 01/18/2009
• Published: 07/22/2010
• Est. Priority Date: 01/18/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Meng, Hua, Xian-Sheng, Yang, Linjun
• Granted Patent: US 8,412,694 B2
• Time in Patent Office: Days
• US Class Current: 707/748
• CPC Class Codes: G06F 16/5838 using colour

Abstract

Colorblind accessible image search technique embodiments are presented that re-rank the results of a relevance-ranked image search to account for the accessibility of the images to a colorblind person. This is accomplished by first computing a colorblind accessibility quantity for each image of interest in the search results. A colorblind accessibility quantity quantizes the degree to which color information is preserved when an image is perceived by a colorblind person viewing the image. It is computed by generating a colorblind version of an image that simulates how the image would appear to the colorblind person. An amount quantifying the loss of color information between the image and the colorblind version of the image is then estimated. This estimate is used to compute the colorblind accessibility quantity for the image. Once the colorblind accessibility quantities have been computed, the image search results are re-ranked based on these quantities.

Claims

1. A computer-implemented process for re-ranking the results of a relevance-ranked image search to account for the accessibility of the images to a colorblind person, comprising using a computer to perform the following process actions:

computing a colorblind accessibility quantity for each image of interest in the search results; and

re-ranking the image search results based on the computed colorblind accessibility quantities.

2. The process of claim 1, wherein the process action of computing a colorblind accessibility quantity for each image of interest in the search results, comprises the actions of:

estimating the fraction of the loss of color information between the image and a colorblind version of the image; and

defining the colorblind accessibility quantity for an image as one minus the estimated fraction of the loss of color information.

3. The process of claim 1, wherein the process action of re-ranking the image search results based on the computed colorblind accessibility quantities, comprises an action of ordering the images of the search results by their associated colorblind accessibility quantity values in descending order.

4. The process of claim 1, wherein the process action of re-ranking the image search results based on the computed colorblind accessibility quantities, comprises the actions of:

identifying a number of the top ranked images in the image search results, wherein the number is less than all the images; and

ordering the identified images of the search results by their associated colorblind accessibility quantity values in descending order.

5. The process of claim 4, wherein the number of top ranked images is user-specified.

6. The process of claim 1, wherein the process action of re-ranking the image search results based on the computed colorblind accessibility quantities, comprises the actions of:

ranking the images of the search results by their associated colorblind accessibility quantity values; and

ordering the images of the search results in descending order based on a combined ranking score of each image, wherein the combined ranking score of an image is a combination of a relevance score of the image and the colorblind accessibility quantity of that image.

7. The process of claim 6, wherein said combination of the relevance score of the image and the colorblind accessibility quantity of that image is as a weighted linear combination computed using the equation NewRankScore=β

RelevanceScore+(1−

β

)Accessibility, wherein NewRankScore is the combined ranking score, RelevanceScore is a relevance score based on which an original search ranking list was generated, Accessibility is the colorblind accessibility quantity, and β

is a weighting factor.

8. The process of claim 6, wherein the process of ordering the images of the search results in descending order based on the combined ranking score of each image is performed whenever it is specified by a user.

9. The process of claim 1, wherein the process action of re-ranking the image search results based on the computed colorblind accessibility quantities, comprises the actions of:

identifying a number of the top ranked images in the image search results, wherein the number is less than all the images;

ranking the identified images of the search results by their associated colorblind accessibility quantity values; and

ordering the identified images of the search results in descending order based on a combined ranking score of each image, wherein the combined ranking score of an image is a combination of a relevance score of the image and the colorblind accessibility quantity of that image.

10. The process of claim 9, wherein the number of top ranked images is user-specified.

11. A computer-implemented process for quantizing a degree to which color information is lost when an image is perceived by a colorblind person viewing the image, comprising using a computer to perform the following process actions:

generating a colorblind version of the image which simulates how the image would appear to the colorblind person; and

estimating an amount quantifying a loss of color information between the image and the colorblind version of the image.

12. The process of claim 11, wherein the process action of generating a colorblind version of the image, comprises the actions of:

inputting a designation of what type of colorblindness the colorblind person suffers from; and

generating the colorblind version of the image to simulate how the image would appear to the colorblind person with the designated type of colorblindness.

13. The process of claim 11, wherein the process action of estimating the amount quantifying the loss of color information between the image and the colorblind version of the image, comprises an action of computing a color distinguish-ability loss which measures the degree to which a color difference between color pairs in the image are preserved in the corresponding color pairs in the colorblind version of the image.

14. The process of claim 13, wherein the process action of computing the color distinguish-ability loss, comprises an action of employing an equation

15. The process of claim 13, wherein a RGB color space of the image and the colorblind version thereof are equally quantized into Q bins, and wherein the process action of computing the color distinguish-ability loss, comprises an action of employing an equation

16. The process of claim 11, wherein the process action of estimating the amount quantifying the loss of color information between the image and the colorblind version of the image, comprises an action of computing a color entropy loss which measures the degree to which colors are lost between the image and the colorblind version of the image.

17. The process of claim 16, wherein a RGB color space of the image and the colorblind version thereof are equally quantized into Q bins, and wherein the process action of computing the color entropy loss, comprises an action of employing an equation

18. The process of claim 11, wherein the process action of estimating the amount quantifying the loss of color information between the image and the colorblind version of the image, comprises the actions of:

computing a color distinguish-ability loss which measures the degree to which a color difference between color pairs in the image are preserved in the corresponding color pairs in the colorblind version of the image;

computing a color entropy loss which measures the degree to which colors are lost between the image and the colorblind version of the image; and

combining the color distinguish-ability loss and the color entropy loss to produce said amount quantifying the loss of color information between the image and the colorblind version of the image.

19. The process of claim 18, wherein the process action of combining the color distinguish-ability loss and the color entropy loss, comprises computing a weighted linear combination of the losses using an equation Loss=α

Loss₁+(1−

α

)Loss₂, wherein Loss is the combined loss, Loss₁ is the color distinguish-ability loss, Loss₂ is the color entropy loss, and α

is a weighting factor.

20. A system for re-ranking the results of a relevance-ranked image search to account for the accessibility of the images to a colorblind user, comprising:

a general purpose computing device;

one or more image databases which are in communication with the computing device; and

a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,

input a designation of the type of colorblindness the user suffers from,

generate colorblind versions of images discovered in the search, said colorblind versions simulating how the associated images would appear to a person suffering from the designated the type of colorblindness,

compute a colorblind accessibility quantity for each images of interest in the search results based on the associated colorblind versions of the images,

re-rank the search results based on a combination of the relevance rankings and rankings derived from the colorblind accessibility quantities, and

provide the re-ranked results to the user.