Semantic search by means of word sense disambiguation using a lexicon

• US 9,317,589 B2
• Filed: 08/07/2008
• Issued: 04/19/2016
• Est. Priority Date: 08/07/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Leyba, Todd
• Primary Examiner(s): Chen, Susan
• Application Number:
  US12/188,047
• Publication Number:
  US 20100036829A1
• Time in Patent Office: 2,812 Days
• Field of Search: 704/260, 704/26
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3344 using natural language analysis

Abstract

Techniques are disclosed for analyzing a “context window” of a search query to determine a semantic meaning of a search word and to filter search results based upon the semantic meaning. Generally, a lexicon may be used to store forms, meanings, and usages of words and phrases. When a user specifies a query, a semantic analyzer obtains all of the word senses for a search word. The semantic analyzer applies lexical analysis techniques to the search word and context window to obtain a total score for each word sense and selects the word sense with the highest total score. After query results such as documents containing the search words are obtained, the semantic analyzer applies lexical analysis techniques to filter the results so that only documents which use the search terms, according to the selected word sense are returned.

Claims

1. A computer-implemented method of filtering search results returned for a search query, wherein the search results include one or more documents, the method comprising:

identifying at least a first search term included in the search query;

determining, by operation of one or more computer processors, a plurality of word usage senses associated with the first search term;

selecting a first word usage sense from the plurality of word usage senses associated with the first search term for use in filtering the search results;

filtering the search results to include only documents in which the first search term is used in a word usage sense matching the selected first word usage sense, wherein at least one document in which the first search term is used in a different word usage sense is filtered out of the search results; and

returning the set of filtered search results as the search results for the search query.

2. The method of claim 1, wherein selecting the first word usage sense from the plurality of word usage senses associated with the first search term, comprises:

for each of the plurality of word usage senses;

determining a plurality of lexical component scores by applying a plurality of lexical analysis techniques using a respective word usage sense and a context window of words surrounding the first search term,

applying a weighting factor to each lexical component score, and

adding the lexical component scores to determine a total score for a respective word usage sense; and

selecting the word usage sense with a highest total score as the first word usage sense.

3. The method of claim 2, wherein one of the lexical analysis techniques comprises:

identifying words in a definition of one of the respective word usage senses; and

calculating a value based upon the number of words in the definition that match words in the context window of words.

4. The method of claim 2, wherein one of the lexical analysis techniques comprises calculating a value based upon how often one of the respective word usage senses is used in a language, relative to other word usage senses of the plurality.

5. The method of claim 2, wherein one of the lexical analysis techniques comprises:

identifying a domain associated with one of the respective word usage senses;

identifying one or more domains of the plurality of word usage senses associated with the words in the context window of words;

determining a value based upon a number of words in the context window of words that belong to the same domain as the first word usage sense.

6. The method of claim 2, wherein one of the lexical analysis techniques comprises:

identifying a first hypernym tree for the first word usage sense;

identifying a second hypernym tree for at least one word in the context window of words; and

calculating a value based upon the degree of overlap between the first and second hypernym trees.

7. The method of claim 2, wherein one of the lexical analysis techniques comprises calculating a value based upon the number of words in the context window of words that are hyponyms of the first word usage sense.

8. The method of claim 2, wherein one of the lexical analysis techniques comprises calculating a value based upon the number of words in the context window of words that are a meronym or holonym of the first word usage sense.

9. The method of claim 2, further comprising, generating a summary display of search results for each word usage sense, wherein the summary for each word usage sense includes:

a respective definition of the word usage sense; and

a total number of the documents that use the first search term according to the respective word usage sense.

10. A computer readable storage medium containing a program which, when executed, performs an operation for filtering search results returned for a search query based upon a word usage sense, wherein the search results include one or more documents, the operation comprising:

identifying at least a first search term included in the search query;

determining a plurality of word usage senses associated with the first search term;

selecting a first word usage sense from the plurality of word usage senses associated with the first search term for use in filtering the search results;

filtering the search results to include only documents in which the first search term is used in a word usage sense matching the selected first word usage sense, wherein at least one document in which the first search term is used in a different word usage sense is filtered out of the search results; and

returning the set of filtered search results as the results for the search query.

11. The computer readable storage medium of claim 10, wherein selecting the first word usage sense from the plurality of word usage senses associated with the first search term, comprises:

for each of the plurality of word usage senses;

determining a plurality of lexical component scores by applying a plurality of lexical analysis techniques using a respective word usage sense and a context window of words surrounding the first search term,

applying a weighting factor to each lexical component score, and

adding the lexical component scores to determine a total score for a respective word usage sense; and

selecting the word usage sense with a highest total score as the first word usage sense.

12. The computer readable storage medium of claim 11, wherein one of the lexical analysis techniques comprises:

identifying words in a definition of one of the respective word usage senses; and

calculating a value based upon the number of words in the definition that match words in the context window of words.

13. The computer readable storage medium of claim 11, wherein one of the lexical analysis techniques comprises calculating a value based upon how often one of the respective word usage senses is used in a language, relative to other word usage senses of the plurality.

14. The computer readable storage medium of claim 11, wherein one of the lexical analysis techniques comprises:

identifying a domain associated with one of the respective word usage senses;

identifying one or more domains of the plurality of word usage senses associated with the words in the context window of words;

determining a value based upon a number of words in the context window of words that belong to the same domain as the first word usage sense.

15. The computer readable storage medium of claim 11, wherein one of the lexical analysis techniques comprises:

identifying a first hypernym tree for the first word usage sense;

identifying a second hypernym tree for at least one word in the context window of words; and

calculating a value based upon the degree of overlap between the first and second hypernym trees.

16. The computer readable storage medium of claim 11, wherein one of the lexical analysis techniques comprises calculating a value based upon the number of words in the context window of words that are hyponyms of the first word usage sense.

17. The computer readable storage medium of claim 11, wherein one of the lexical analysis techniques comprises calculating a value based upon the number of words in the context window of words that are a meronym or holonym of the first word usage sense.

18. A system, comprising:

a processor; and

a memory containing a program, which, when executed by the processor is configured to perform an operation for filtering search results returned for a search query based upon a word usage sense, wherein the search results include one or more documents, the operation comprising;

identifying at least a first search term included in the search query;

determining a plurality of word usage senses associated with the first search term;

selecting a first word usage sense from the plurality of word usage senses associated with the first search term for use in filtering the search results;

filtering the search results to include only documents in which the first search term is used in a word usage sense matching the selected first word usage sense, wherein at least one document in which the first search term is used in a different word usage sense is filtered out of the search results; and

returning the set of filtered search results as the results for the search query.

19. The system of claim 18, wherein selecting the first word usage sense from the plurality of word usage senses associated with the first search term, comprises:

for each of the plurality of word usage senses;

determining a plurality of lexical component scores by applying a plurality of lexical analysis techniques using a respective word usage sense and a context window of words surrounding the first search term,

applying a weighting factor to each lexical component score, and

adding the lexical component scores to determine a total score for a respective word usage sense; and

selecting the word usage sense with a highest total score as the first word usage sense.

20. The system of claim 19, wherein one of the lexical analysis techniques comprises:

identifying words in a definition of one of the respective word usage senses; and

calculating a value based upon the number of words in the definition that match words in the context window of words.

21. The system of claim 19, wherein one of the lexical analysis techniques comprises calculating a value based upon how often one of the respective word usage senses is used in a language, relative to other word usage senses of the plurality.

22. The system of claim 19, wherein one of the lexical analysis techniques comprises:

identifying a domain associated with one of the respective word usage senses;

identifying one or more domains of the plurality of word usage senses associated with the words in the context window of words;

determining a value based upon a number of words in the context window of words that belong to the same domain as the first word usage sense.

23. The system of claim 19, wherein one of the lexical analysis techniques comprises:

identifying a first hypernym tree for the first word usage sense;

identifying a second hypernym tree for at least one word in the context window of words; and

calculating a value based upon the degree of overlap between the first and second hypernym trees.

24. The system of claim 19, wherein one of the lexical analysis techniques comprises calculating a value based upon the number of words in the context window of words that are hyponyms of the first word usage sense.

25. The system of claim 19, wherein one of the lexical analysis techniques comprises calculating a value based upon the number of words in the context window of words that are a meronym or holonym of the first word usage sense.