Method and apparatus for parsing text using mutual information

• US 7,475,006 B2
• Filed: 07/11/2001
• Issued: 01/06/2009
• Est. Priority Date: 07/11/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Weise, David N.
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US09/903,055
• Publication Number:
  US 20030083861A1
• Time in Patent Office: 2,736 Days
• Field of Search: 704/9, 704/2, 704/250
• US Class Current: 704/9
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/268 Morphological analysis
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking

Abstract

A method and parser are provided that generate a score for a node identified during a parse of a text segment. The score is based on a mutual information score that measures the mutual information between a phrase level for the node and a word class of at least one word in the text segment.

Claims

1. A method of generating a score for a node identified during a parse of a text segment, the method comprising:

identifying a phrase level for the node;

identifying a word class for at least one word that neighbors a text spanned by the node by;

identifying a word class for a word to the left of the text spanned by the node; and

identifying a word class for a word to the right of the text spanned by the node; and

generating a score by determining a mutual information metric based on the phrase level and the word class, wherein determining a mutual information metric comprises determining a mutual information metric based on the phrase level of the node, the word class of the word to the right of the text spanned by the node and the word class of the word to the left of the text spanned by the node.

2. The method of claim 1 wherein identifying a word class further comprises:

identifying all possible word classes for a word to the left of the text spanned by the node; and

identifying all possible word classes for a word to the right of the text spanned by the node.

3. The method of claim 2 wherein generating a score comprises generating a score based in part on all of the identified word classes.

4. The method of claim 1 wherein identifying a word class comprises identifying all possible word classes for at least one word.

5. A method of generating a score for a node identified during a parse of a text segment, the method comprising:

identifying a phrase level for the node;

identifying a word class for at least one word that neighbors a text spanned by the node by;

identifying all possible word classes for a word to the left of the text spanned by the node; and

identifying all possible word classes for a word to the right of the text spanned by the node; and

generating a score by determining a mutual information metric based on the phrase level and the word class, wherein generating a score comprises generating a score based in part on all of the identified word classes.

6. The method of claim 5 wherein determining a mutual information metric comprises determining a mutual information metric based on the phrase level of the node, the word class of the word to the right of the text spanned by the node and the word class of the word to the left of the text spanned by the node.