SYSTEM AND METHOD FOR RANKING WEB SEARCHES WITH QUANTIFIED SEMANTIC FEATURES

• US 20100191740A1
• Filed: 01/26/2009
• Published: 07/29/2010
• Est. Priority Date: 01/26/2009
• Status: Abandoned Application

• Current Assignee: Oath Inc. (Verizon Communications Inc.)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Dumoulin, Benoit, Lu, Yumao
• Application Number:
  US12/360,016
• Publication Number:
  US 20100191740A1
• Time in Patent Office: Days
• US Class Current: 707/748
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

A system and method for ranking web searches with quantified semantic features. A query for a web search is received from a user. The query is segmented and tagged into one or more linguistic segments using linguistic analysis. At least some of the linguistic segments are tagged with a linguistic type. A query execution plan is generated comprising the linguistic segments and, for each of the linguistic segments tagged with a linguistic type, at least one tag attribute comprising at least one domain specific feature of the linguistic type. A search is performed for documents matching the query. Each of the documents is scored for each of the linguistic segments of the query execution plan using the tag attributes of the respective linguistic segment. The documents are ranked using a function that uses the scores of the documents. A ranked list of the documents is transmitted back to the user.

Claims

1. A method comprising the steps of:

receiving a query for a web search from a user, via a network, wherein the query comprises a plurality of query tokens;

segmenting and tagging the query, using linguistic analysis performed on at least one computing device, into one or more linguistic segments, wherein each linguistic segment comprises a term comprising one or more of the query tokens, and wherein at least some of the linguistics segments are further tagged with a linguistic type;

generating a query execution plan, on the at least one computing device, wherein the query execution plan comprises the one or more linguistic segments, and wherein for each of the one or more linguistic segments tagged with a linguistic type, the query execution plan further comprises at least one tag attribute comprising at least one domain specific feature of the linguistic type of its respective linguistic segment;

searching, using the at least one computing device, for a plurality of documents matching the query;

scoring the plurality of documents, using the at least one computing device, wherein each of the plurality of documents is scored for each of the one or more linguistic segments of the query execution plan using the at least one tag attribute of the respective linguistic segment;

ranking the plurality of documents, using the at least one computing device, wherein the plurality of documents are ranked by a function which uses the scores of the respective documents to determine the rank of the respective document;

transmitting a list of the plurality of documents in rank order, over the network, to the user.

2. The method of claim 1 wherein each of the features specific to the domain of the linguistic type of a linguistic segment are retrieved from one of a plurality of vertical databases, wherein each of the plurality of vertical databases comprises data related to a specific linguistic type.

3. The method of claim 2 wherein the at least one of the domain specific features is selected from the list:

confidence level, weight.

4. The method of claim 2 wherein the at least one of the domain specific features comprises confidence level and weight.

5. The method of claim 2 wherein for each linguistic segment having a domain specific feature of weight, the weight is expressed in the query execution plan as one or more inverse document frequencies, where each inverse document frequency is the inverse document frequency of one of the one or more query tokens comprising the term in the segment.

6. The method of claim 1 wherein each of the plurality of documents is scored using at least one scoring methodology selected from the list:

frequency of linguistic segments, normalized frequency of linguistic segments, semantic minimum coverage of linguistic segments, the semantic moving average BM25 of linguistic segments, the vertical moving average BM25 of linguistic segments.

7. The method of claim 1 wherein each of the plurality of documents is scored using semantic minimum coverage for each linguistic segment within the query execution plan, wherein the formula for calculating semantic minimum coverage is:

8. The method of claim 1 wherein each of the plurality of documents is scored using semantic moving average for each linguistic segment within the query execution plan, wherein the formula for calculating semantic moving average is:

9. The method of claim 2 wherein each of the plurality of documents is scored using vertical moving average for each linguistic segment within the query execution plan, wherein the formula for calculating vertical moving average is:

10. A system comprising:

a query receiving module that receives queries for a web searches from users, via a network, wherein each query comprises a plurality of query tokens;

a linguistic analysis module that segments and tags each query received by the query receiving module into one or more linguistic segments using linguistic analysis, wherein each linguistic segment comprises a term comprising one or more of the query tokens of the respective query, and wherein at least some of the linguistics segments are further tagged with a linguistic type;

a query execution plan generation module that generates query execution plans for each query processed by the linguistic analysis module, wherein each query execution plan comprises the one or more linguistic segments of the respective query, and wherein for each of the one or more linguistic segments tagged with a linguistic type, the query execution plan further comprises at least one tag attribute comprising at least one domain specific feature of the linguistic type of its respective linguistic segment;

a search module that searches, for each query processed by the query execution plan generation module, for a plurality of documents matching the respective query;

a document scoring module that scores, for every query processed by the search module, the respective plurality of documents, wherein each of the plurality of documents is scored for each of the one or more linguistic segments of the respective query execution plan using the at least one tag attribute of the respective linguistic segment;

a document ranking module that ranks, for every query processed by the search module, the respective plurality of documents, wherein the plurality of documents are ranked by a function which uses the scores calculated by the document scoring module of the respective documents to determine the rank of the respective documents;

a results transmission module that transmits, for each plurality of documents ranked by the document ranking module, a list of the respective plurality of documents in rank order, over the network, to a user that submitted the query.

11. The system of claim 10 wherein each of the features specific to the domain of the linguistic type of a linguistic segment are retrieved from one of a plurality of vertical databases, wherein each of the plurality of vertical databases comprises data related to a specific linguistic type.

12. The system of claim 11 wherein the at least one of the domain specific features is selected from the list:

confidence level, weight.

13. The system of claim 11 wherein the at least one of the domain specific features comprises confidence level and weight.

14. The system of claim 11 wherein for each linguistic segment having a domain specific feature of weight, the weight is expressed in the query execution plan as one or more inverse document frequencies, where each inverse document frequency is the inverse document frequency of the one of the one or more query tokens comprising the term in the respective segment.

15. The system of claim 10 wherein each of the plurality of documents is scored by the document scoring module using at least one scoring methodology selected from the list:

frequency of linguistic segments, normalized frequency of linguistic segments, semantic minimum coverage of linguistic segments, the semantic moving average BM25 of linguistic segments, the vertical moving average BM25 of linguistic segments within the document.

16. The system of claim 10 wherein each of the plurality of documents is scored by the document scoring module using semantic minimum coverage for each linguistic segment within the query execution plan, wherein the formula for calculating semantic minimum coverage is:

17. The system of claim 10 wherein each of the plurality of documents is scored by the document scoring module using semantic moving average for each linguistic segment within the query execution plan, wherein the formula for calculating semantic moving average is:

18. The system of claim 11 wherein each of the plurality of documents is scored by the document scoring module using vertical moving average for each linguistic segment within the query execution plan, wherein the formula for calculating vertical moving average is:

19. A computer-readable medium having computer-executable instructions for a method comprising the steps of:

receiving a query for a web search from a user, via a network, wherein the query comprises a plurality of query tokens;

segmenting and tagging the query, using linguistic analysis performed on at least one computing device, into one or more linguistic segments, wherein each linguistic segment comprises a term comprising one or more of the query tokens, and wherein at least some of the linguistics segments are further tagged with a linguistic type;

generating a query execution plan, on the at least one computing device, wherein the query execution plan comprises the one or more linguistic segments, and wherein for each of the one or more linguistic segments tagged with a linguistic type, the query execution plan further comprises at least one tag attribute comprising at least one domain specific feature of the linguistic type of its respective linguistic segment;

searching, using the at least one computing device, for a plurality of documents matching the query;

scoring the plurality of documents, using the at least one computing device, wherein each of the plurality of documents is scored for each of the one or more linguistic segments of the query execution plan using the at least one tag attribute of the respective linguistic segment;

ranking the plurality of documents, using the at least one computing device, wherein the plurality of documents are ranked by a function which uses the scores of the respective documents to determine the rank of the respective document;

transmitting a list of the plurality of documents in rank order, over the network, to the user.

20. The method of claim 19 wherein each of the features specific to the domain of the linguistic type of a linguistic segment are retrieved from one of a plurality of vertical databases, wherein each of the plurality of vertical databases comprises data related to a specific linguistic type.

21. The method of claim 20 wherein the at least one of the domain specific features is selected from the list:

confidence level, weight.

22. The method of claim 20 wherein the at least one of the domain specific features comprises confidence level and weight.

23. The method of claim 20 wherein for each linguistic segment having a domain specific feature of weight, the weight is expressed in the query execution plan as one or more inverse document frequencies, where each inverse document frequency is the inverse document frequency of the one of the one or more query tokens comprising the term in the segment.

24. The method of claim 19 wherein each of the plurality of documents is scored using at least one scoring methodology selected from the list:

frequency of linguistic segments, normalized frequency of linguistic segments, semantic minimum coverage of linguistic segments, the semantic moving average BM25 of linguistic segments, the vertical moving average BM25 of linguistic segments.

25. The method of claim 19 wherein each of the plurality of documents is scored using semantic minimum coverage for each linguistic segment within the query execution plan, wherein the formula for calculating semantic minimum coverage is:

26. The method of claim 19 wherein each of the plurality of documents is scored using semantic moving average for each linguistic segment within the query execution plan, wherein the formula for calculating semantic moving average is:

27. The method of claim 20 wherein each of the plurality of documents is scored using vertical moving average for each linguistic segment within the query execution plan, wherein the formula for calculating vertical moving average is: