Generating alternative descriptions for images

• US 9,098,532 B2
• Filed: 11/29/2012
• Issued: 08/04/2015
• Est. Priority Date: 11/29/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cragun, Brian J., Johlic, Marc K., Keohane, Susann M.
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US13/688,358
• Publication Number:
  US 20140146053A1
• Time in Patent Office: 978 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B

Abstract

Mechanisms are provided for generating alternative text descriptions for images in electronic documents. An original image embedded in an electronic document is analyzed to generate a data pattern for the image. A matching operation is performed to identify similar images in other electronic documents from sources of electronic documents based on the generated data pattern. Textual description information associated with the similar images is extracted from data associated with the similar image. An alternative text description for the original image is generated based on the extracted textual description information associated with the similar images. The alternative text description for the original image is stored in association with the original image.

Claims

1. A method, in a data processing system comprising at least one processor and at least one memory, for generating alternative text descriptions for images in electronic documents, comprising:

analyzing, by the data processing system, an original image embedded in an electronic document to generate a data pattern for the image;

performing, by the data processing system, a matching operation to identify one or more similar images in other electronic documents from one or more sources of electronic documents based on the generated data pattern;

extracting, by the data processing system, textual description information associated with the one or more similar images from data associated with the one or more similar images;

generating, by the data processing system, an alternative text description for the original image based on the extracted textual description information associated with the one or more similar images;

storing, by the data processing system, the alternative text description for the original image in association with the original image;

generating a confidence level value for the alternative text description, wherein the confidence level value identifies a level of confidence that the alternative text description accurately describes content of the original image; and

storing the confidence level value in association with the alternative text description, wherein the confidence level value is generated based on scoring each keyword in the alternative text description according to a frequency of occurrence of the keyword in textual description information associated with the one or more similar images and wherein the confidence level value is generated using the following relationship;

Confidence level value=(v/(v+m))*R+(m/(v+m))*C

where R is an average score for the keywords in the alternative text description, v is a number of keywords in the alternative text description, m is a minimum number of keywords in the alternative text description required, and C is a mean confidence level value.

2. The method of claim 1, wherein analyzing the original image further comprises classifying the image as either information or non-informative, wherein, in response to the image being classified as non-informative, the method is aborted before executing the performing, extracting, generating, and storing operations.

3. The method of claim 1, wherein the confidence level value is generated based on a comparison of the alternative text description to an existing text description associated with the original image.

4. The method of claim 1, further comprising:

generating a relevance rating value for the alternative text description, wherein the relevance rating value identifies a level of relevance of the alternative text description to a remaining portion of the electronic document other than the original image; and

storing the relevance rating value in association with the alternative text description.

5. The method of claim 4, wherein the relevance rating value is generated at least by:

comparing at least one of keywords or phrases in the alternative text description to portions of the electronic document;

determining a degree of matching between the keywords or phrases and text in the portions of the electronic document; and

generating the relevance rating value based on the degree of matching, wherein the degree of matching is based on a frequency of occurrence of a matching keyword or phrase in the text of the portions of the electronic document.

6. The method of claim 1, wherein generating the alternative text description for the original image based on the extracted textual description information associated with the one or more similar images comprises, for each similar image in the one or more similar images, collecting information regarding at least one of:

a type of source of the similar image;

a textual context in which the similar image is presented in the other documents;

user comments associated with the similar image;

or

a frequency of use of the similar image in the other electronic documents, and wherein generating the alternative text description comprises selecting textual description information associated with the one or more similar images based on a weighting of the textual description information of each similar image relative to textual description information of other similar images in the one or more similar images of the collected information, wherein the weighting is based on the collected information.

7. The method of claim 1, further comprising:

generating one of an audible or a tactile output of the alternative text description to assist a visually impaired person in obtaining information about the original image.

8. A computer program product, for generating alternative text descriptions for images in electronic documents, comprising a non-transitory computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

analyze an original image embedded in an electronic document to generate a data pattern for the image;

perform a matching operation to identify one or more similar images in other electronic documents from one or more sources of electronic documents based on the generated data pattern;

extract textual description information associated with the one or more similar images from data associated with the one or more similar images;

generate an alternative text description for the original image based on the extracted textual description information associated with the one or more similar images;

store the alternative text description for the original image in association with the original image;

generate a confidence level value for the alternative text description, wherein the confidence level value identifies a level of confidence that the alternative text description accurately describes content of the original image; and

store the confidence level value in association with the alternative text description, wherein the confidence level value is generated based on scoring each keyword in the alternative text description according to a frequency of occurrence of the keyword in textual description information associated with the one or more similar images and wherein the confidence level value is generated using the following relationship;

Confidence level value=(v/(v+m))*R+(m/(v+m))*C

where R is an average score for the keywords in the alternative text description, v is as number of keywords in the alternative text description, m is a minimum number of keywords in the alternative text description required, and C is a mean confidence level value.

9. The computer program product of claim 8, wherein the computer readable program to analyze the original image further causes the computing device to:

classify the image as either informative or non-informative, wherein, in response to the image being classified as non-informative, the computer readable program causes the computing device to abort before executing the performing, extracting, generating, and storing operations.

10. The computer program product of claim 8, wherein the confidence level value is generated based on a comparison of the alternative text description to an existing text description associated with the original image.

11. The computer program product of claim 8, wherein the computer readable program further causes the computing device to:

generate a relevance rating value for the alternative text description, wherein the relevance rating value identifies a level of relevance of the alternative text description to a remaining portion of the electronic document other than the original image; and

store the relevance rating value in association with the alternative text description.

12. The computer program product of claim 11, wherein the computer readable program to generate the relevance rating value further cause the computing device to:

compare at least one of keywords or phrases in the alternative text description to portions of the electronic document;

determine a degree of matching between the keywords or phrases and text in the portions of the electronic document; and

generate the relevance rating value based on the degree of matching, wherein the degree of matching is based on a frequency of occurrence of a matching keyword or phrase in the text of the portions of the electronic document.

13. The computer program product of claim 8, wherein the computer readable program to generate the alternative text description for the original image based on the extracted textual description information associated with the one or more similar images further causes the computing device to, for each similar image in the one or more similar images, collect information regarding at least one of:

a type of source of the similar image;

a textual context in which the similar image is presented in the other documents;

user comments associated with the similar image;

or

a frequency of use of the similar image in the other electronic documents, and wherein generating the alternative text description comprises selecting textual description information associated with the one or more similar images based on a weighting of the textual description information of each similar image relative to textual description information of other similar images in the one or more similar images of the collected information, wherein the weighting is based on the collected information.

14. An apparatus, comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to;

analyze an original image embedded in an electronic document to generate a data pattern for the image;

perform a matching operation to identify one or more similar images in other electronic documents from one or more sources of electronic documents based on the generated data pattern;

extract textual description information associated with the one or more similar images from data associated with the one or more similar images;

generate an alternative text description for the original image based on the extracted textual description information associated with the one or more similar images;

store the alternative text description for the original image in association with the original image;

generate a confidence level value for the alternative text description, wherein the confidence level value identifies a level of confidence that the alternative text description accurately describes content of the original image; and

store the confidence level value in association with the alternative text description, wherein the confidence level value is generated based on scoring each keyword in the alternative text description according to a frequency of occurrence of the keyword in textual description information associated with the one or more similar images and wherein the confidence level value is generated using the following relationship;

Confidence level value=(v/v+m))*R+(m/(v+m))*C

where R is an average score for the keywords in the alternative text description, v is a number of keywords hi the alternative text description, m is a minimum number of keywords in the alternative text description required, and C is a mean confidence level value.

15. The apparatus of claim 14, wherein the instructions to analyze the original image further causes the processor to:

classify the image as either informative or non-informative, wherein, in response to the image being classified as non-informative, the instructions cause the processor to abort before executing the performing, extracting, generating, and storing operations.

16. The apparatus of claim 14, wherein the confidence level value is generated based on a comparison of the alternative text description to an existing text description associated with the original image.

17. The apparatus of claim 14, wherein the instructions further cause the processor to:

generate a relevance rating value for the alternative text description, wherein the relevance rating value identifies a level of relevance of the alternative text description to a remaining portion of the electronic document other than the original image; and

store the relevance rating value in association with the alternative text description.

18. The apparatus of claim 17, wherein the instructions to generate the relevance rating value further causes the processor to:

compare at least one of keywords or phrases in the alternative text description to portions of the electronic document;

determine a degree of matching between the keywords or phrases and text in the portions of the electronic document; and

generate the relevance rating value based on the degree of matching, wherein degree of matching is based on a frequency of occurrence of a matching keyword or phrase in the text of the portions of the electronic document.

19. The apparatus of claim 14, wherein the instructions to generate the alternative text description for the original image based on the extracted textual description information associated with the one or more similar images further causes the processor to, for each similar image in the one or more similar images, collect information regarding at least one of:

a type of source of the similar image;

a textual context in which the similar image is presented in the other documents;

user comments associated with the similar image;

or

a frequency of use of the similar image in the other electronic documents, and wherein generating the alternative text description comprises selecting textual description information associated with the one or more similar images based on a weighting of the textual description information of each similar image relative to textual description information of other similar images in the one or more similar images of the collected information, wherein the weighting is based on the collected information.

20. The apparatus of claim 14, wherein the instructions further cause the processor to:

generate one of an audible or a tactile output of the alternative text description to assist a visually impaired person in obtaining information about the original image.