Colorblind accessible image search

• US 8,412,694 B2
• Filed: 01/18/2009
• Issued: 04/02/2013
• Est. Priority Date: 01/18/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Meng, Yang, Linjun, Hua, Xian-Sheng
• Primary Examiner(s): MORRISON, JAY A
• Application Number:
  US12/355,795
• Publication Number:
  US 20100185624A1
• Time in Patent Office: 1,535 Days
• Field of Search: None
• US Class Current: 707/706
• CPC Class Codes: G06F 16/5838 using colour

Abstract

Colorblind accessible image search technique embodiments are presented that re-rank the results of a relevance-ranked image search to account for the accessibility of the images to a colorblind person. This is accomplished by first computing a colorblind accessibility quantity for each image of interest in the search results. A colorblind accessibility quantity quantizes the degree to which color information is preserved when an image is perceived by a colorblind person viewing the image. It is computed by generating a colorblind version of an image that simulates how the image would appear to the colorblind person. An amount quantifying the loss of color information between the image and the colorblind version of the image is then estimated. This estimate is used to compute the colorblind accessibility quantity for the image. Once the colorblind accessibility quantities have been computed, the image search results are re-ranked based on these quantities.

Claims

1. A computer-implemented process for re-ranking the results of a relevance-ranked image search to account for the accessibility of the images to a colorblind person, comprising using a computer to perform the following process actions:

inputting a designation of the type of colorblindness the colorblind person suffers from;

obtaining results of a relevance-ranked image search;

generating colorblind versions of images discovered in the search, said colorblind versions simulating how the associated images would appear to a person suffering from said type of colorblindness the colorblind person suffers from;

computing a colorblind accessibility quantity for each image of interest in the search results based on the associated colorblind versions of the images, said colorblind accessibility quantity quantifying the degree to which color information is preserved when an image is perceived by a colorblind person viewing the image; and

re-ranking the images of interest in the search results based at least in part on the computed colorblind accessibility quantities.

2. The process of claim 1, wherein the process action of computing a colorblind accessibility quantity for each image of interest in the search results, comprises the actions of:

estimating the fraction of the loss of color information between the image and a colorblind version of the image; and

defining the colorblind accessibility quantity for an image as one minus the estimated fraction of the loss of color information.

3. The process of claim 1, wherein the process action of re-ranking the images of interest in the search results based at least in part on the computed colorblind accessibility quantities, comprises an action of ordering the images of the search results by their associated colorblind accessibility quantity values in descending order.

4. The process of claim 1, wherein the process action of re-ranking the images of interest in the search results based at least in part on the computed colorblind accessibility quantities, comprises the actions of:

identifying a number of the top ranked images in the image search results, wherein the number is less than all the images; and

ordering the identified images of the search results by their associated colorblind accessibility quantity values in descending order.

5. The process of claim 4, wherein the number of top ranked images is user-specified.

6. The process of claim 1, wherein the process action of re-ranking the images of interest in the search results based at least in part on the computed colorblind accessibility quantities, comprises the actions of:

ranking the images of the search results by their associated colorblind accessibility quantity values; and

ordering the images of the search results in descending order based on a combined ranking score of each image, wherein the combined ranking score of an image is a combination of a relevance score of the image and the colorblind accessibility quantity of that image.

7. The process of claim 6, wherein said combination of the relevance score of the image and the colorblind accessibility quantity of that image is as a weighted linear combination computed using the equation NewRankScore=β

RelevanceScore+(1−

β

) Accessibility, wherein NewRankScore is the combined ranking score, RelevanceScore is a relevance score based on which an original search ranking list was generated, Accessibility is the colorblind accessibility quantity, and β

is a weighting factor.

8. The process of claim 6, wherein the process of ordering the images of the search results in descending order based on the combined ranking score of each image is performed whenever it is specified by a user.

9. The process of claim 1, wherein the process action of re-ranking the images of interest in the search results based at least in part on the computed colorblind accessibility quantities, comprises the actions of:

identifying a number of the top ranked images in the image search results, wherein the number is less than all the images;

ranking the identified images of the search results by their associated colorblind accessibility quantity values; and

ordering the identified images of the search results in descending order based on a combined ranking score of each image, wherein the combined ranking score of an image is a combination of a relevance score of the image and the colorblind accessibility quantity of that image.

10. The process of claim 9, wherein the number of top ranked images is user-specified.

11. A system for re-ranking the results of a relevance-ranked image search to account for the accessibility of the images to a colorblind user, comprising:

a general purpose computing device;

one or more image databases which are in communication with the computing device; and

a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,

input a designation of the type of colorblindness the user suffers from,

input results of a relevance-ranked image search,

generate colorblind versions of images discovered in the search, said colorblind versions simulating how the associated images would appear to a person suffering from the designated the type of colorblindness,

compute a colorblind accessibility quantity for each images of interest in the search results based on the associated colorblind versions of the images, said colorblind accessibility quantity quantifying the degree to which color information is preserved when an image is perceived by a colorblind person viewing the image,

re-rank the images of interest in the search results based on a combination of the relevance rankings and rankings derived from the colorblind accessibility quantities, and

provide the re-ranked results to the user.