Ranking search results using feature extraction

• US 7,716,198 B2
• Filed: 12/21/2004
• Issued: 05/11/2010
• Est. Priority Date: 12/21/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Hang, Meyerzon, Dmitriy
• Primary Examiner(s): Wong; Don
• Assistant Examiner(s): DANG, THANH HA T
• Application Number:
  US11/019,091
• Publication Number:
  US 20060136411A1
• Time in Patent Office: 1,967 Days
• Field of Search: 707/3, 707 5- 7
• US Class Current: 707/706
• CPC Class Codes: G06F 16/3344 using natural language analysis

Abstract

Methods and computer-readable media are provided for ranking search results using feature extraction data. Each of the results of a search engine query is parsed to obtain data, such as text, formatting information, metadata, and the like. The text, the formatting information and the metadata are passed through a feature extraction application to extract data that may be used to improve a ranking of the search results based on relevance of the search results to the search engine query. The feature extraction application extracts features, such as titles, found in any of the text based on formatting information applied to or associated with the text. The extracted titles, the text, the formatting information and the metadata for any given search results item are processed according to a field weighting application for determining a ranking of the given search results item. Ranked search results items may then be displayed according to ranking.

Claims

1. A method, comprising:

receiving on a computing device resource items generated by a search engine in response to a search request;

parsing each of the resource items to obtain data, wherein the data includes;

text, formatting information and metadata;

passing the data for each of the resource items that includes the text, the formatting information and the metadata through a feature extraction application for determining applicability of the obtained metadata by comparing the obtained metadata against known metadata for use in ranking search results;

comparing words parsed from the text that is separate from the obtained metadata against a database of known words that have been previously characterized;

using formatting characteristics of the data to determine when to extract a title from the resource item;

wherein the formatting characteristics that are used include;

a bold formatting characteristic;

an underlining formatting characteristic;

wherein the feature extraction application stores statistical information for each extracted title that provides a number of and a frequency of appearance of the extracted title in the data; and

extracting features from the one or more portions for each of the resource items;

passing extracted features through a ranking application for generating a ranking value for each of the resource items based on a relevance of each of the resource items to the search request; and

generating a list of the resource items in an order according to the ranking value for each of the resource items, whereby when the resource item has the ranking value associated with being ranked as most relevant to the search request received by the search engine is displayed first.

2. The method of claim 1, whereby parsing the resource items into data further includes parsing the data to include one or more text selections and associated formatting information applied to the one or more text selections.

3. The method of claim 2, whereby extracting the features from the data includes extracting statistical information for any extracted features including a frequency with which any features are found.

4. The method of claim 3, whereby passing the extracted features and the data through the ranking application for generating the ranking value for each of the resource items based on the relevance of each of the resource items to a search request received by the search engine includes passing the statistical information for the extracted features through the ranking application.

5. The method of claim 4, whereby extracting the features from the data for each of the resource items includes computing a score for a beginning point and an ending point for each extracted feature in a given resource item based on formatting information applied to a document.

6. The method of claim 5, whereby extracting the features from the data for each of the resource items includes extracting titles from the data for each of the resource items.

7. The method of claim 1, prior to passing the any extracted features and the data through the ranking application for generating the ranking value for each of the resource items, further comprising:

receiving a search request at the search engine; and

causing the list of the resource items to display in a user interface.

8. The method of claim 1, prior to parsing each of the resource items into data, receiving the resource items from one or more information sources accessed and parsed by the search engine.

9. A method, comprising:

receiving on a computing device resource items generated by a search engine in response to a search request;

obtaining resource items from an information source;

parsing a metadata source in each of the resource items for one or more metadata items, and parsing a content portion of each of the resource items into one or more text selections and associated formatting information applied to the one or more text selections;

passing the one or more metadata items and the one or more text selections and associated formatting information for each of the resource items through a feature extraction application;

extracting titles from the one or more text selections and associated formatting information for each of the resources including using formatting characteristics to determine when to extract a title from the resource item;

wherein the formatting characteristics that are used include;

a bold formatting characteristic;

an underlining formatting characteristic;

wherein the feature extraction application stores statistical information for each extracted title that provides a number of and a frequency of appearance of the extracted title;

determining applicability of the one or more metadata items by comparing the one or more metadata items against known metadata for use in ranking search results;

comparing words parsed from the text that is separate from the one or more metadata items against a database of known words that have been previously characterized;

processing the extracted titles, the one or more metadata items, the one or more text selections and the associated formatting information according to a ranking algorithm for generating a ranking value for each of the resource items based on a relevance of each of the resource items to the search request received by the search engine; and

generating a list of the resource items in an order according to the ranking value for each of the resource items.

10. The method of claim 9, further comprising displaying each of the resource items in a user interface in an order according to the ranking value for each of the resource items whereby the resource item having the ranking value associated with being ranked as most relevant to the search request received by the search engine is displayed first.

11. The method of claim 9,

whereby extracting titles from the one or more text selections and associated formatting information for each of the resource items includes extracting statistical information for any extracted titles including a frequency with which any titles are found in an associated resource item;

and

further comprising processing the statistical information for the any extracted titles according to a field weighting algorithm in association with processing the any extracted titles, the one or more metadata items, the one or more text selections and the associated formatting information according to the field weighting algorithm.

12. The method of claim 9, prior to processing the extracted titles, the one or more metadata items, the one or more text selections and the associated formatting information according to the ranking algorithm for generating the ranking value for each of the resource items based on the relevance of each of the resource items to the search request received by the search engine, receiving the search request at the search engine.

13. A computer-readable medium having stored thereon computer-executable instructions which when executed by a computer perform a method, comprising:

receiving resource items generated by a search engine in response to a search request, the resource items including one or more content portions and one or more metadata portions;

parsing each of the resource items into the one or more content portions and associated formatting information applied to the one or more content portions;

passing the one or more content portions and associated formatting information for each of the resource items through a feature extraction application;

determining applicability of the one or more metadata portions by comparing the one or more metadata portions against known metadata for use in ranking search results;

comparing words parsed from the content portions that is separate from the one or more metadata portions against a database of known words that have been previously characterized;

using formatting characteristics to determine when to extract a title from the resource item;

wherein the formatting characteristics that are used include;

a bold formatting characteristic;

an underlining formatting characteristic;

wherein the feature extraction application stores statistical information for each extracted title that provides a number of and a frequency of appearance of the extracted title in the data; and

extracts features from the one or more content portions and associated formatting information for each of the resource items;

passing the extracted features and the one or more content portions and associated formatting information through a ranking application for generating a ranking value for each of the resource items based on a relevance of each of the resource items to the search request received by the search engine; and

generating a list of the resource items in an order according to the ranking value for each of the resource items.

14. The computer-readable medium of claim 13, wherein parsing each of the resource items further comprises:

parsing the one or more metadata sources in each of the resource items for one or more metadata items; and

parsing the one or more content portions into one or more text selections and associated formatting information applied to the one or more text selections.

15. The computer-readable medium of claim 14, whereby extracting features from the one or more content portions and associated formatting information and the one or more metadata portions includes extracting statistical information for any extracted features including a frequency with which any features are found in an associated resource item.

16. The computer-readable medium of claim 15, whereby passing the extracted features and the one or more content portions and associated formatting information and one or more metadata portions through the ranking application for generating the ranking value for each of the resource items based on a relevance of each of the resource items to a search request received by the search engine includes passing the statistical information for the extracted features through the ranking application with the associated extracted features.

17. The computer-readable medium of claim 16, whereby extracting features from the one or more content portions and associated formatting information and the one or more metadata portions for each of the resource items includes computing a score for a beginning point and an ending point for each extracted feature in a given resource item based on formatting information applied to a document.

18. The computer-readable medium of claim 17, whereby extracting features from the one or more content portions and associated formatting information and the one or more metadata portions for each of the resource items includes extracting titles from the one or more content portions and associated formatting information for each of the resource items.

19. The computer-readable medium of claim 13, prior to passing the any extracted features and the one or more content portions and associated formatting information through the ranking application for generating the ranking value for each of the resource items, further comprising:

receiving a search request at the search engine; and

causing the list of the resource items to display in a user interface in an order according to the ranking value for each of the resource items whereby the resource item having the ranking value associated with being ranked as most relevant to the search request received by the search engine is displayed first.

20. The computer-readable medium of claim 13, prior to parsing each of the resource items into the one or more content portions and associated formatting information, receiving the resource items from one or more information sources accessed and parsed by the search engine.