METHODS AND SYSTEMS FOR CREATING AND USING AN ADAPTIVE THESAURUS

• US 20100198821A1
• Filed: 01/30/2009
• Published: 08/05/2010
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: RELX Inc. (RELX PLC)
• Original Assignee: LexisNexis Group Inc. (RELX PLC)
• Inventors: LORITZ, Donald, SILVER, Harry, CHEN, Shian-jung, EDALA, Narasimha
• Granted Patent: US 8,463,806 B2
• Time in Patent Office: Days
• US Class Current: 707/728
• CPC Class Codes: G06F 16/3338 Query expansion
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms

Abstract

Methods and systems are provided for creating an adaptive thesaurus. A term pair including an index term and an expansion term is received. A recall gain, an expansion independence, and a semantic similarity of the term pair are calculated using a processor. Whether to store the term pair is determined based on the recall gain, the expansion independence, and the semantic similarity. The term pair is stored based on the determination. Methods and systems are provided for searching using an adaptive thesaurus. A search query including a query term is received. An expansion term stored in association with an index term matching the query term in the adaptive thesaurus is retrieved. Using a processor, the search query is expanded using the expansion term based on a recall gain, an expansion independence, and a semantic similarity.

Claims

1. A computer-implemented method for creating an adaptive thesaurus, comprising:

receiving a term pair including an index term and an expansion term;

calculating, using a processor, a recall gain, an expansion independence, and a semantic similarity of the term pair;

determining whether to store the term pair based on the recall gain, the expansion independence, and the semantic similarity; and

storing the term pair based on the determination.

2. The method of claim 1, further comprising:

storing the recall gain, the expansion independence, and the semantic similarity in association with the term pair.

3. The method of claim 1, further comprising:

determining a first feature vector of the index term and a second feature vector of the expansion term based on a corpus of semantically annotated texts; and

storing the first feature vector in association with the index term and the second feature vector in association with the expansion term.

4. The method of claim 3, wherein the first feature vector, the second feature vector, the recall gain, the expansion independence, and the semantic similarity are calculated based on the results of:

searching the index term in the corpus of semantically annotated texts;

searching the expansion term in the corpus of semantically annotated texts;

searching the index term and the expansion term, conjoined by a Boolean OR, in the corpus of semantically annotated texts; and

searching the index term and the expansion term, conjoined by a Boolean AND, in the corpus of semantically annotated texts.

5. The method of claim 1, wherein the determining comprises:

comparing the recall gain with a recall gain threshold, the expansion independence with an expansion independence threshold, and the semantic similarity with a semantic similarity threshold.

6. A system for creating an adaptive thesaurus, comprising:

a computer comprising a processor, the processor executing instructions to;

receive a term pair including an index term and an expansion term;

calculate a recall gain, an expansion independence, and a semantic similarity of the term pair;

determine whether to store the term pair based on the recall gain, the expansion independence, and the semantic similarity; and

store the term pair based on the determination.

7. The system of claim 6, wherein the processor further executes instructions to store the recall gain, the expansion independence, and the semantic similarity in association with the term pair.

8. The system of claim 6, wherein the processor further executes instructions to:

determine a first feature vector of the index term and a second feature vector of the expansion term based on a corpus of semantically annotated texts; and

store the first feature vector in association with the index term and the second feature vector in association with the expansion term.

9. The system of claim 8, wherein the processor further executes instructions to:

search the index term in the corpus of semantically annotated texts;

search the expansion term in the corpus of semantically annotated texts;

search the index term and the expansion term, conjoined by a Boolean OR, in the corpus of semantically annotated texts; and

search the index term and the expansion term, conjoined by a Boolean AND, in the corpus of semantically annotated texts,

wherein the first feature vector, the second feature vector, the recall gain, the expansion independence, and the semantic similarity are calculated based on results of the searches.

10. The system of claim 6, wherein the processor further executes instructions to compare the recall gain with a recall gain threshold, the expansion independence with an expansion independence threshold, and the semantic similarity with a semantic similarity threshold.

11. A computer-readable storage medium including instructions which, when executed by a processor, perform a method for creating an adaptive thesaurus, the method comprising:

receiving a term pair including an index term and an expansion term;

calculating, using a processor, a recall gain, an expansion independence, and a semantic similarity of the term pair;

determining whether to store the term pair based on the recall gain, the expansion independence, and the semantic similarity; and

storing the term pair based on the determination.

12. The computer-readable storage medium of claim 11, wherein the method further comprises:

storing the recall gain, the expansion independence, and the semantic similarity in association with the term pair.

13. The computer-readable storage medium of claim 11, wherein the method further comprises:

determining a first feature vector of the index term and a second feature vector of the expansion term based on a corpus of semantically annotated texts; and

storing the first feature vector in association with the index term and the second feature vector in association with the expansion term.

14. The computer-readable storage medium of claim 13, wherein the first feature vector, the second feature vector, the recall gain, the expansion independence, and the semantic similarity are calculated based on the results of:

searching the index term in the corpus of semantically annotated texts;

searching the expansion term in the corpus of semantically annotated texts;

searching the index term and the expansion term, conjoined by a Boolean OR, in the corpus of semantically annotated texts; and

searching the index term and the expansion term, conjoined by a Boolean AND, in the corpus of semantically annotated text.

15. The computer-readable storage medium of claim 11, wherein the determining comprises:

comparing the recall gain with a recall gain threshold, the expansion independence with an expansion independence threshold, and the semantic similarity with a semantic similarity threshold.

16. A computer-implemented method for searching using an adaptive thesaurus, comprising:

receiving a search query including a query term;

retrieving an expansion term stored in association with an index term matching the query term in the adaptive thesaurus; and

expanding, using a processor, the search query using the expansion term based on a recall gain, an expansion independence, and a semantic similarity.

17. The method of claim 16, further comprising:

submitting the expanded search query to a search engine.

18. The method of claim 16, further comprising:

using the recall gain, the expansion independence, and the semantic similarity to rank a search result.

19. The method of claim 16, further comprising:

retrieving the recall gain, the expansion independence, and the semantic similarity from the adaptive thesaurus, the recall gain, the expansion independence, and the semantic similarity being stored in the adaptive thesaurus in association with the query term and the expansion term.

20. The method of claim 16, further comprising:

comparing the recall gain with a recall gain threshold, the expansion independence with an expansion independence threshold, and the semantic similarity with a semantic similarity threshold.

21. The method of claim 16, further comprising:

calculating, before expanding the search query, the semantic similarity; and

comparing the semantic similarity with a semantic similarity threshold.

22. The method of claim 16, further comprising:

displaying the expansion term to a user; and

enabling the user to select the displayed expansion term for expanding the search query.

23. The method of claim 22, wherein the expansion independence of the displayed expansion term is lower than an expansion independence threshold.

24. The method of claim 20, further comprising:

enabling a user to adjust one or more of the recall gain threshold, the expansion independence threshold, and the semantic similarity threshold.

25. A system for searching using an adaptive thesaurus, comprising:

a computer comprising a processor, the processor executing instructions to;

receive a search query including a query term;

retrieve an expansion term stored in association with an index term matching the query term in the adaptive thesaurus; and

expand, using a processor, the search query using the expansion term based on a recall gain, an expansion independence, and a semantic similarity.

26. The system of claim 25, wherein the processor further executes instructions to submit the expanded search query to a search engine.

27. The system of claim 25, wherein the processor further executes instructions to use the recall gain, the expansion independence, and the semantic similarity to rank a search result.

28. The system of claim 25, wherein the processor further executes instructions to retrieve the recall gain, the expansion independence, and the semantic similarity from the adaptive thesaurus, the recall gain, the expansion independence, and the semantic similarity being stored in the adaptive thesaurus in association with the query term and the expansion term.

29. The system of claim 25, wherein the processor further executes instructions to compare the recall gain with a recall gain threshold, the expansion independence with an expansion independence threshold, and the semantic similarity with a semantic similarity threshold.

30. The system of claim 25, wherein the processor further executes instructions to:

calculate, before expanding the search query, the semantic similarity; and

compare the semantic similarity with a semantic similarity threshold.

31. The system of claim 25, wherein the processor further executes instructions to:

display the expansion term to a user; and

enable the user to select the displayed expansion term for expanding the search query.

32. The system of claim 31, wherein the expansion independence of the displayed expansion term is lower than an expansion independence threshold

33. The system of claim 29, wherein the processor further executes instructions to enable a user to adjust one or more of the recall gain threshold, the expansion independence threshold, and the semantic similarity threshold.

34. A computer-readable storage medium including instructions which, when executed by a processor, perform a method for searching using an adaptive thesaurus, the method comprising:

receiving a search query including a query term;

retrieving an expansion term stored in association with an index term matching the query term in the adaptive thesaurus; and

expanding, using a processor, the search query using the expansion term based on a recall gain, an expansion independence, and a semantic similarity.

35. The computer-readable storage medium of claim 34, wherein the method further comprises:

submitting the expanded search query to a search engine.

36. The computer-readable storage medium of claim 34, wherein the method further comprises:

using the recall gain, the expansion independence, and the semantic similarity to rank a search result.

37. The computer-readable storage medium of claim 34, wherein the method further comprises:

retrieving the recall gain, the expansion independence, and the semantic similarity from the adaptive thesaurus, the recall gain, the expansion independence, and the semantic similarity being stored in the adaptive thesaurus in association with the query term and the expansion term.

38. The computer-readable storage medium of claim 34, wherein the method further comprises:

comparing the recall gain with a recall gain threshold, the expansion independence with an expansion independence threshold, and the semantic similarity with a semantic similarity threshold.

39. The computer-readable storage medium of claim 34, wherein the method further comprises:

calculating, before expanding the search query, the semantic similarity; and

comparing the semantic similarity with a semantic similarity threshold.

40. The computer-readable storage medium of claim 39, wherein the method further comprises:

displaying the expansion term to a user; and

enabling the user to select the displayed expansion term for expanding the search query.

41. The computer-readable storage medium of claim 40, wherein the expansion independence of the displayed expansion term is lower than an expansion independence threshold.

42. The computer-readable storage medium of claim 38, wherein the method further comprises:

enabling a user to adjust one or more of the recall gain threshold, the expansion independence threshold, and the semantic similarity threshold.

42-1. A computer-implemented method for creating an adaptive thesaurus, comprising:

receiving a term pair including an index term and an expansion term;

calculating, using a processor, a recall gain, an expansion independence, and a semantic similarity of the term pair;

determining whether to store the index term and the expansion term based on the recall gain, the expansion independence, and the semantic similarity;

storing the index term and the expansion term based on the determination; and

storing the recall gain, the expansion independence, and the semantic similarity in association with the index term and the expansion term.