SEARCH LEXICON EXPANSION

• US 20120158703A1
• Filed: 12/16/2010
• Published: 06/21/2012
• Est. Priority Date: 12/16/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Xiao, Liu, Jingjing, Acero, Alejandro, Wang, Ye-Yi
• Granted Patent: US 9,928,296 B2
• Time in Patent Office: Days
• US Class Current: 707/723
• CPC Class Codes: G06F 16/3349 Reuse of stored results of previous queries
• CPC Class Codes: G06F 16/374 Thesaurus
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/242 Dictionaries

Abstract

One or more techniques and/or systems are disclosed for creating an expanded or improved lexicon for use in search-based semantic tagging. A set of first documents can be identified using a set of first lexicon elements as queries, and one or more first document patterns can be extracted from the set of first documents. The document patterns can be used to find one or more second documents in a query log that comprise the document patterns, which are associated with query terms used to return the second documents. The query terms for the second documents can be extracted and used to expand the lexicon. Elements within the lexicon may be weighted based upon relevance to different query domains, for example.

Claims

1. A computer-based method for creating an expanded lexicon for use in searching, comprising:

identifying a second document from a query log using a document pattern extracted from a set of one or more first documents in the query log, using a computer-based processor, where the set of one or more first documents is identified using a set of one or more first lexicon elements in a set of one or more first queries; and

extracting a second lexicon element for the lexicon from a second query identified using the second document from the query log.

2. The method of claim 1, comprising extracting the document pattern from the first document in the query log.

3. The method of claim 2, extracting the document pattern comprising identifying one or more of the following in the first document:

one or more keywords in a document title;

one or more keywords in a document snippet;

one or more expressions in a document universal resource locator (URL); and

one or more identifiers in the document.

4. The method of claim 1, comprising identifying a second document in the query log using the extracted document pattern, for respective query domains.

5. The method of claim 4, identifying a second document in the query log comprising comparing the extracted document pattern to one or more of the following in a query log document:

a document title;

a document snippet;

a document URL; and

a body of the document.

6. The method of claim 4, the query domains comprising a category of entities described by a query term.

7. The method of claim 1, comprising identifying the second query using the second document.

8. The method of claim 7, identifying the second query comprising identifying one or more query terms in the query log, used to return the second document from a search.

9. The method of claim 1, comprising extracting a lexicon candidate from the second query, comprising parsing the second query into respective query elements.

10. The method of claim 1, comprising removing one or more lexicon elements comprising a contextual query term that is not relative to a query domain.

11. The method of claim 1, comprising one or more of assigning a weight to a lexicon element based on a popularity of the lexicon element in a query domain, and assigning a weight to a query domain based on a likelihood of the query domain for a lexicon element.

12. The method of claim 11, comprising using the query log to perform one or more of:

determining the popularity of the lexicon element given the query domain; and

determining the likelihood of the query domain given the lexicon element.

13. The method of claim 11, comprising one or more of:

determining the popularity of the lexicon element in a query domain using document selection counts for a set of query domain documents from the query log; and

determining the likelihood of the query domain for the lexicon element comprising identifying query domain documents returned by a search using the lexicon element.

14. The method of claim 11, comprising one or more of:

determining the popularity of the lexicon element given the query domain, comprising combining a probability of the lexicon element given a set of query domain documents from the query log with a probability of the query domain documents from the query log given the query domain; and

determining a probability of the query domain given the lexicon element, comprising combining a probability of the query domain given the set of query domain documents from the query log with a probability of the query domain documents from the query log given the lexicon element.

15. A system for creating an expanded lexicon for use in searching, comprising:

a computer-based processor configured to process data for the system;

a query log component, stored on one or more computer-based data storage components, comprising one or more queries and one or more documents selectively linked to the queries; and

a lexicon expansion component operably coupled with the processor and the query log component, and configured to;

identify a second document from the query log using a document pattern extracted from a set of one or more first documents in the query log, where the set of one or more first documents is identified using a set of one or more first lexicon elements in a set of one or more first queries; and

identify a second lexicon element to add to the lexicon from a second query identified using the second document.

16. The system of claim 15, comprising a weighting component configured to assign a weight comprising one or more of a popularity of a lexicon element in a query domain, and a likelihood of a query domain for a lexicon element.

17. The system of claim 16, the weighting component comprising one or more of:

a lexicon element probability component configured to determine a probability that the lexicon element belongs to a given query domain; and

a query domain probability component configured to determine a probability that a query domain corresponds to a given lexicon element.

18. The system of claim 15, comprising a lexicon cleaning component configured to remove one or more lexicon elements comprising a contextual query term not relative to a query domain.

19. The system of claim 15, comprising a pattern extraction component configured to extract the document pattern from the first document based on one or more of the following in the first document:

one or more keywords in a document title;

one or more keywords in a document snippet;

one or more expressions in a document universal resource locator (URL); and

one or more identifiers in the document.

20. A computer-based method for creating an expanded lexicon for use in searching, comprising:

extracting a document pattern from a first document in a query log;

identifying a second document in the query log using the extracted pattern;

identifying a second query using the second document, comprising identifying a query in the query log that yielded the second document;

extracting a second lexicon element comprising parsing the second query into respective query elements;

using the query log to determine one or more of;

a popularity for the second lexicon element given the query domain; and

determining a likelihood of a query domain given the second lexicon element; and

one or more of assigning a weight to the second lexicon element based on a popularity of the second lexicon element in a query domain, and assigning a weight to a query domain based on a likelihood of the query domain for the second lexicon element.