Natural Language Hypernym Weighting For Word Sense Disambiguation

• US 20090089047A1
• Filed: 08/29/2008
• Published: 04/02/2009
• Est. Priority Date: 08/31/2007
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Powerset Inc (Microsoft Corporation)
• Inventors: Snow, Rion, Waterman, Scott A., Pell, Barney
• Granted Patent: US 8,463,593 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

Technologies are described herein for probabilistically assigning weights to word senses and hypernyms of a word. The weights can be used in natural language processing applications such as information indexing and querying. A word hypernym weight (WHW) score can be determined by summing word sense probabilities of word senses from which the hypernym is inherited. WHW scores can be used to prune away hypernyms prior to indexing, to rank query results, and for other functions related to information indexing and querying. A semantic search technique can use WHW scores to retrieve an entry related to a word from an index in response to matching an indexed hypernym of the word with a query term applied to the index. More refined and accurate query results may be provided based on reduced user inputs.

Claims

1. A method for indexing information with word sense disambiguation, the method comprising:

determining a first word sense associated with a word;

determining a hypernym associated with the first word sense;

establishing a word hypernym weight for the hypernym;

selecting the hypernym based on the word hypernym weight; and

storing a second word sense of the selected hypernym into a semantic index.

2. The method of claim 1, further comprising storing the first word sense in the semantic index.

3. The method of claim 1, further comprising storing the selected hypernym into a semantic index.

4. The method of claim 3, wherein pruning comprises applying a heuristic for cost to value ratio to the hypernyms.

5. The method of claim 1, further comprising storing the word hypernym weight in the semantic index.

6. The method of claim 1, wherein establishing the word hypernym weight comprises summing word sense probabilities of word senses from which the hypernym is inherited.

7. The method of claim 1, further comprising querying the semantic index.

8. The method of claim 7, further comprising ranking query results using the word hypernym weight.

9. The method of claim 7, further comprising retrieving an entry related to the word from the semantic index in response to matching an indexed hypernym of the word with a query term applied to the semantic index.

10. The method of claim 7, further comprising providing support for selecting or filtering query results based on the word hypernym weight.

11. A computer storage medium having computer executable instructions stored thereon which, when executed by a computer, cause the computer to:

identify a word to be entered into an index;

determine a word sense associated with the word;

determine a hypernym associated with the word sense;

establish a word hypernym weight for the hypernym; and

store information associated with the hypernym into the index based on the word hypernym weight.

12. The computer storage medium of claim 11, further causing the computer to store the word sense into the index.

13. The computer storage medium of claim 11, further causing the computer to prune hypernyms in response to the established hypernym weight.

14. The computer storage medium of claim 11, further causing the computer to applying a heuristic for cost to value ratio to the hypernym.

15. The computer storage medium of claim 11, further causing the computer to store the word hypernym weight into the index.

16. The computer storage medium of claim 11, wherein establishing the word hypernym weight comprises summing word sense probabilities of word senses from which the hypernym is inherited.

17. The computer storage medium of claim 11, further causing the computer to query the index.

18. The computer storage medium of claim 17, further causing the computer to rank query results using the word hypernym weight.

19. The computer storage medium of claim 17, further causing the computer to retrieve an entry related to the word from the index in response to matching an indexed hypernym of the word with a query term applied to the index.

20. A method for natural language indexing and query with word sense disambiguation, the method comprising:

identifying a word to be stored into a semantic index;

determining a first word sense associated with the word;

determining a hypernym associated with the first word sense;

establishing a word hypernym weight for the hypernym by summing word sense probabilities of word senses from which the hypernym is inherited;

pruning hypernyms based on the word hypernym weight;

determining a second word sense associated with the hypernym;

entering the second word sense into the semantic index;

storing the word hypernym weight into the semantic index; and

retrieving an entry related to the word from the index in response to matching an indexed hypernym of the word with a query term applied to the index.