Search lexicon expansion

• US 9,928,296 B2
• Filed: 12/16/2010
• Issued: 03/27/2018
• Est. Priority Date: 12/16/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Li, Xiao, Liu, Jingjing, Acero, Alejandro, Wang, Ye-Yi
• Primary Examiner(s): Uddin, Mohammed R
• Application Number:
  US12/970,477
• Publication Number:
  US 20120158703A1
• Time in Patent Office: 2,658 Days
• Field of Search: 707708, 707726, 707739, 707771, 707999005
• CPC Class Codes: G06F 16/3349 Reuse of stored results of previous queries
• CPC Class Codes: G06F 16/374 Thesaurus
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 40/242 Dictionaries

Abstract

One or more techniques and/or systems are disclosed for creating an expanded or improved lexicon for use in search-based semantic tagging. A set of first documents can be identified using a set of first lexicon elements as queries, and one or more first document patterns can be extracted from the set of first documents. The document patterns can be used to find one or more second documents in a query log that comprise the document patterns, which are associated with query terms used to return the second documents. The query terms for the second documents can be extracted and used to expand the lexicon. Elements within the lexicon may be weighted based upon relevance to different query domains, for example.

Claims

1. A method performed by at least one processing unit, the method comprising:

identifying a seed lexicon term from a plurality of lexicon terms that are identified by a lexicon as members of a semantic class;

identifying result documents retrieved from a search engine by queries submitted to the search engine that include the seed lexicon term;

extracting a keyword that is shared by the result documents retrieved from the search engine by the queries that include the seed lexicon term; and

using the keyword shared by the result documents to expand the lexicon by;

identifying, in a query log, specific other result documents that also include the keyword shared by the result documents, wherein the query log does not indicate that the specific other result documents were retrieved from the search engine using the lexicon terms identified by the lexicon as members of the semantic class;

identifying another member of the semantic class in a particular query that has been submitted to the search engine to retrieve the specific other result documents; and

adding to the lexicon, as a new lexicon term, the another member of the semantic class.

2. The method of claim 1, further comprising:

cleaning contextual terms that appear in the particular query with the new lexicon term.

3. The method of claim 1, further comprising:

weighting the new lexicon term relative to another new lexicon term based at least on popularity of the new lexicon term and the another new lexicon term in a particular query domain associated with the semantic class, the weighting being based at least on counts of the new lexicon term with respect to the particular query domain.

4. The method of claim 1, wherein the keyword appears in both:

titles of the result documents retrieved from the search engine by the queries that include the seed lexicon term, and

other titles of the specific other result documents that the query log does not indicate were retrieved from the search engine using the lexicon terms identified by the lexicon as members of the semantic class.

5. The method of claim 1, wherein the keyword appears in both:

URLs of the result documents retrieved from the search engine by the queries that include the seed lexicon term, and

other URLs of the specific other result documents that the query log does not indicate were retrieved from the search engine using the lexicon terms identified by the lexicon as members of the semantic class.

6. The method of claim 1, wherein the keyword appears in both:

snippets of the result documents retrieved from the search engine by the queries that include the seed lexicon term, and

other snippets of the specific other result documents that the query log does not indicate were retrieved from the search engine using the lexicon terms identified by the lexicon as members of the semantic class.

7. The method of claim 1, further comprising:

parsing the particular query into one or more elements, the one or more elements including the another member of the semantic class.

8. A method performed by one or more processing units, the method comprising:

accessing a query log reflecting queries that users have submitted to a search engine to retrieve documents;

identifying, within the query log, first documents returned by the search engine in response to first queries that include a dictionary term that appears in a dictionary;

extracting a document pattern shared by the first documents; and

using the extracted document pattern to expand the dictionary by;

identifying second documents within the query log that also share the extracted document pattern shared by the first documents;

identifying a specific query term within the query log that the users submitted to the search engine to retrieve the second documents; and

expanding the dictionary by adding the specific query term to the dictionary as a new dictionary term.

9. The method of claim 8, wherein the dictionary comprises a lexicon of entities of a particular semantic class, the dictionary term and the new dictionary term being different entities of the particular semantic class.

10. The method of claim 9, wherein the particular semantic class is a restaurant class, the dictionary term identifies a first restaurant, and the new dictionary term identifies a second restaurant.

11. The method of claim 10, wherein the extracted document pattern shared by the first documents and the second documents includes the word “

restaurant”

.

12. The method of claim 11, wherein the word “

restaurant”

appears in both;

first titles of the first documents, and

second titles of the second documents.

13. The method of claim 11, wherein the word “

restaurant”

appears in both;

first URLs of the first documents, and

second URLs of the second documents.

14. The method of claim 11, wherein the word “

restaurant”

appears in both;

first snippets of the first documents, and

second snippets of the second documents.

15. A system comprising:

one or more processing units; and

memory comprising instructions which, when executed by the one or more processing units, cause the one or more processing units to;

access a query log reflecting queries that users have submitted to a search engine to retrieve documents;

identify, in the query log, first documents that were returned by the search engine in response to users submitting lexicon terms to the search engine, the lexicon terms being identified by a lexicon as members of particular query domain;

extract a document pattern from the first documents that were returned by the search engine in response to the users submitting the lexicon terms to the search engine; and

use the extracted document pattern to expand the lexicon by;

identifying, in the query log, second documents that include the extracted document pattern and that the query log does not indicate were retrieved by submitting the lexicon terms to the search engine;

identifying, in the query log, another member of the particular query domain that the users have submitted to the search engine to retrieve the second documents; and

adding the another member of the particular query domain to the lexicon as a new lexicon term.

16. The system of claim 15, wherein the instructions, when executed by the one or more processing units, cause the one or more processing units to:

identify the new lexicon term by parsing a particular query that the users submitted to retrieve the second documents.

17. The system of claim 15, wherein the extracted document pattern comprises a specific word that is shared by both the first documents that were retrieved by submitting the lexicon terms to the search engine and the second documents that the query log does not indicate were retrieved by submitting the lexicon terms to the search engine.

18. The system of claim 17, wherein the instructions, when executed by the one or more processing units, cause the one or more processing units to:

weight the lexicon term relative to the new lexicon term based at least on relative popularity in the particular query domain.

19. The system of claim 17, wherein the particular query domain relates to locations, the lexicon term identifies a first location, and the new lexicon term that is added to the lexicon identifies a second location.

20. The system of claim 17, wherein the query domain relates to movies, the lexicon term identifies a first movie title, and the new lexicon term that is added to the lexicon identifies a second movie title.

21. A method performed by at least one processing unit, the method comprising:

obtaining a lexicon of previously-identified members of a semantic class;

processing a query log to identify result documents that have been retrieved using the previously-identified members of the semantic class;

identifying a characteristic that is shared by the result documents that have been retrieved using the previously-identified members of the semantic class;

identifying further result documents that also have the characteristic that is shared by the result documents that have been retrieved using the previously-identified members of the semantic class; and

identifying a new member of the semantic class that has been used to retrieve the further result documents, the new member of the semantic class not being previously identified by the lexicon as belonging to the semantic class.