Abbreviation handling in web search

• US 7,809,715 B2
• Filed: 04/15/2008
• Issued: 10/05/2010
• Est. Priority Date: 04/15/2008
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Dumoulin, Benoit, Peng, Fuchun, Wei, Xing
• Primary Examiner(s): Le; Uyen T.
• Application Number:
  US12/103,126
• Publication Number:
  US 20090259629A1
• Time in Patent Office: 903 Days
• Field of Search: None
• US Class Current: 707/713
• CPC Class Codes: G06F 16/3338 Query expansion

Abstract

A method for handling abbreviations in web queries includes building a dictionary of a plurality of possible word expansions for a plurality of potential abbreviations related to query terms received or anticipated to be received by a search engine; accepting a query including an abbreviation; expanding the abbreviation into one of the plurality of word expansions if a probability that the expansion is correct is above a threshold value, wherein the probability is determined by taking into consideration a context of the abbreviation within the query, wherein the context including at least anchor text; and sending the query with the expanded abbreviation to the search engine to generate a search results page related to the query.

Claims

1. A method for handling abbreviations in web queries, the method comprising:

building a dictionary of a plurality of possible word expansions for a plurality of potential abbreviations related to query terms received or anticipated to be received by a search engine;

accepting a query including an abbreviation;

expanding the abbreviation into one of the plurality of word expansions if a probability that the expansion is correct is above a threshold value, wherein the probability is determined by taking into consideration a context of the abbreviation within the query, wherein the context comprises at least anchor text; and

sending the query with the expanded abbreviation to the search engine to generate a search results page related to the query.

2. The method of claim 1, wherein the context is taken from one or more of anchor text, query logs, and click logs.

3. The method of claim 2, wherein if the probability of the expansion being correct is lower than a probability of the expansion being the abbreviation itself, the method further comprising:

not expanding the abbreviation; and

sending the query with the unexpanded abbreviation to the search engine to generate the search results page.

4. The method of claim 2, wherein the probability of the expansion being correct given the abbreviation and the context, P(E|AC), is proportional to

5. The method of claim 4, wherein P(E|A) is given by:

6. The method of claim 5, wherein when the context includes the anchor text, the method further comprising:

determining a probability that the context will co-occur given the expansion of the abbreviation as

7. The method of claim 6, wherein when the context includes query logs, the method further comprising:

linearly combining and smoothing the probability P(c_i|AE) with

8. The method of claim 7, wherein when the context includes click logs, the method further comprising:

linearly combining and smoothing the probability P(c_i|AE) with

9. The method of claim 1, wherein the dictionary is built offline, the method further comprising:

retrieving anchor texts that point to a same web page; and

extracting abbreviations and corresponding expansions within the anchor text that are formed as initialisms.

10. A method for disambiguation of abbreviations in web queries, the method comprising:

retrieving anchor texts that link to a same web page;

extracting a plurality of abbreviations and corresponding expansions from the anchor text;

building a dictionary of the plurality of the abbreviations and corresponding expansions in a database;

accepting a query including an abbreviation;

determining whether or not to expand the abbreviation in the query based on a probability that a corresponding expansion from the dictionary is correct is above a threshold value, wherein the probability is determined by taking into account a context comprising one or more of anchor text, query logs, and click logs; and

rewriting the query to include an expanded abbreviation if the probability that the expansion is correct is above the threshold value, wherein the results produced by a search engine in response to search queries are improved.

11. The method of claim 10, wherein if the probability of the expansion being correct is below the threshold value, the method further comprising:

not expanding the abbreviation; and

sending the query with the unexpanded abbreviation to the search engine to generate the search results page.

12. The method of claim 10, wherein a probability of a most probable expansion from the dictionary comprises a first probability, and a probability that the expansion is the abbreviation itself comprises a second probability, the method further comprising:

determining a ratio between the first and second probabilities;

expanding the abbreviation in accordance with the most probable expansion if the ratio is above a first threshold value;

highlighting the abbreviation with a suggested expansion of the most probable expansion for a searching user so that the user may accept the suggested expansion if the ratio is between a second, lower threshold value and the first threshold value; and

not expanding the abbreviation if the ratio is below the second threshold value.

13. The method of claim 12, further comprising:

rewriting the query to include the expansion of the abbreviation in response to user selection of the suggested expansion; and

regenerating the search results for the user based on the re-written query.

14. The method of claim 12, wherein a probability of the expansion being correct given the abbreviation and the context, P(E|AC), is proportional to

15. The method of claim 14, wherein P(E|A) is given by:

16. The method of claim 15, wherein when the context includes the anchor text, the method further comprising:

determining a probability that the context will co-occur given the expansion of the abbreviation as

17. The method of claim 16, wherein when the context includes query logs, the method further comprising:

linearly combining and smoothing the probability P(c_i|AE) with

18. The method of claim 17, wherein when the context includes click logs, the method further comprising:

linearly combining and smoothing the probability P(c_i|AE) with

19. A system for handling abbreviations in we queries, the system comprising:

a memory containing instructions for execution by a processor;

an abbreviation expansion database containing a dictionary of a plurality of possible word expansions for a plurality of potential abbreviations related to query search terms, wherein the abbreviations and corresponding potential expansions are derived from at least anchor text;

a communicator operable to accept a query including an abbreviation and operable to send search queries to a search engine;

a language modeler coupled with the memory, the processor, and the communicator, the language modeler operable to determine whether or not to expand the abbreviation in the query based on a probability that a corresponding expansion from the dictionary is correct is above a threshold value, wherein the probability is determined by taking into account a context comprising one or more of anchor text, query logs, and click logs; and

an expander coupled with the language modeler and operable to rewrite the query to include an expanded abbreviation if the probability that the expansion is correct is above the threshold value, wherein the results produced by the search engine in response to search queries are improved.

20. The system of claim 19, wherein if the probability of the expansion being correct is below the threshold value, the expander operable to not expand the abbreviation;

and the communicator is operable to send the query with the unexpanded abbreviation to the search engine to generate the search results page.

21. The system of claim 19, wherein a probability of a most probable expansion from the dictionary comprises a first probability, and a probability that the expansion is the abbreviation itself comprises a second probability, the language modeler and expander are operable to:

determine a ratio between the first and second probabilities;

expand the abbreviation in accordance with the most probable expansion if the ratio is above a first threshold value;

highlight the abbreviation with a suggested expansion of the most probable expansion for a searching user so that the user may accept the suggested expansion if the ratio is between a second, lower threshold value and the first threshold value; and

not expand the abbreviation if the ratio is below the second threshold value.

22. The system of claim 21, wherein the expander is operable to:

rewrite the query to include the expansion of the abbreviation in response to user selection of the suggested expansion; and

regenerate the search results for the user based on the rewritten query.

23. The system of claim 21, wherein a probability of the expansion being correct given the abbreviation and the context, P(E|AC), is proportional to

24. The system of claim 23, wherein P(E|A) is given by:

25. The system of claim 24, wherein when the context includes the anchor text, the modeler is operable to:

determine a probability that the context will co-occur given the expansion of the abbreviation as

26. The system of claim 25, wherein when the context includes query logs, the modeler is operable to:

linearly combining and smoothing the probability P(c_i|AE) with

27. The system of claim 26, wherein when the context includes click logs, the modeler is operable to:

linearly combining and smoothing the probability P(c_i|AE) with