SCORING RELEVANCE OF A DOCUMENT BASED ON IMAGE TEXT

• US 20110087660A1
• Filed: 12/17/2010
• Published: 04/14/2011
• Est. Priority Date: 03/01/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ma, Wei-Ying, Shi, Shuming, Li, Zhiwei, Yu, Qing, Wen, Ji-Rong
• Granted Patent: US 8,645,370 B2
• Time in Patent Office: Days
• US Class Current: 707/728
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A method and system for determining relevance of a document having text and images to a text string is provided. A scoring system identifies image text associated with an image of the document. The scoring system calculates an image score indicating relevance of the image text to the text string. The image score may be used in many applications, such as searching, summary generation, and document classification, image search, and image classification.

Claims

1. A computing device for determining relevance of a document to a text string, the document containing text and images, the text and the text string having terms, comprising:

a memory storing computer-executable instructions implementing

a component that, for each of a plurality of images contained in the document, identifies text associated with the image by extracting text from the document that is determined to be adjacent to the image, the extracted text being the identified text that is associated with the image;

a component that, for each of the plurality of images, determines relevance of the identified text associated with the image to the text string based on comparison of terms of the identified text to terms of the text string; and

a component that determines relevance of the document to the text string based on the determined relevance of the identified text associated with each of the plurality of the images to the text string; and

a processor for executing the instructions stored in the memory.

2. The computing device of claim 1 wherein the document is a web page and the identified text is metadata of the image.

3. The computing device of claim 1 wherein the component that determines relevance of the identified text associated with the image to the text string factors in importance of the image to the document.

4. The computing device of claim 3 wherein the importance of the image is determined using an importance function that is trained to determine the importance of images to documents.

5. The computing device of claim 4 wherein the document is a web page and the importance of the image is based on image level features, document level features, and site level features.

6. The computing device of claim 1 wherein the component that determines relevance of the document to the text string is further based on relevance of text content of the document to the text string.

7. The computing device of claim 1 including a component that ranks documents based on their determined relevance.

8. The computing device of claim 7 wherein the documents that are ranked are results of a search with the text string as the search request.

9. A computer-readable storage device containing computer-executable instructions for controlling a computing device to determine relevance of a document to a text string, the document containing text and an image, the text and the text string having terms, by a method comprising:

identifying image text of the document, the image text of the document being a subset of text of the document that is determined to be associated with the image;

a component that, determines relevance of the image text to the text string based on comparison of terms of the image text to terms of the text string; and

a component that determines relevance of the document to the text string based on the determined relevance of the image text to the text string.

10. The computer-readable storage device of claim 9 wherein the image text is a caption for the image.

11. The computer-readable storage device of claim 9 wherein the image text is metadata associated with the image.

12. The computer-readable storage device of claim 9 wherein the image text is adjacent to the image.

13. The computer-readable storage device of claim 9 wherein the component that determines relevance of the image text to the text string factors in importance of the image to the document.

14. The computer-readable storage device of claim 9 wherein the document contains multiple images with each image having an importance to the document and wherein the image is the most important image of the document.

15. The computer-readable storage device of claim 14 wherein the importance of the image to the document is determined using an importance function that is trained to determine the importance of images to documents.

16. The computer-readable storage device of claim 15 wherein the document is a web page and the importance of the image is based on image level features, document level features, and site level features.

17. A method in a computing device for determining relevance of web pages to queries, the web pages having text and images, the method comprising:

for images of web pages, identifying image text associated with the image by extracting from the web page having the image text that is determined to be associated with the image;

receiving a query;

generating a search result of web pages for the received query;

for each web page in the generated search result, determining relevance of image text associated with an image of the web page to the query; and

ranking the web pages of the generated search result based on the determined relevance of the image text of the web pages to the query.

18. The method of claim 17 wherein a web page has multiple images and the web pages are ranked based on relevance of the most important image of the web page.

19. The method of claim 18 wherein the importance of the image is based on image level features, document level features, and site level features.

20. The method of claim 17 wherein the image text is metadata associated with an image.