Semi-supervised part-of-speech tagging

• US 8,275,607 B2
• Filed: 12/12/2007
• Issued: 09/25/2012
• Est. Priority Date: 12/12/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Johnson, Mark Edward, Toutanova, Kristina Nikolova
• Primary Examiner(s): Godbold, Douglas
• Assistant Examiner(s): Villena, Mark
• Application Number:
  US11/954,212
• Publication Number:
  US 20090157384A1
• Time in Patent Office: 1,749 Days
• Field of Search: 704/4, 704/9, 704/240
• US Class Current: 704/9
• CPC Class Codes: G06F 40/268 Morphological analysis

Abstract

A word is selected from a received text and features are identified from the word. The features are applied to a model to identify probabilities for sets of part-of-speech tags. The probabilities for the sets of part-of-speech tags are used to weight scores for possible part-of-speech tags for the selected word to form weighted scores. The weighted scores are used to select a part-of-speech tag for the word and the selected part of speech tag is stored or output. The scores for the possible part-of-speech tags are based on variational approximation parameters trained from a sparse prior over probability distributions describing the probability of a part-of-speech tag given a word.

Claims

1. A method comprising:

receiving a text comprising a sequence of words;

selecting a word from the text;

identifying features of the selected word, the features comprising a suffix of the selected word;

applying the features of the selected word to a model to identify probabilities for sets of part-of-speech tags, at least one set of part-of-speech tags comprising at least two part-of-speech tags, each part-of-speech tag representing a part-of-speech;

with a processor, using the probabilities for sets of part-of-speech tags to weight scores for possible part-of-speech tags for the selected word to form weighted scores by performing steps for each set of part-of speech tags, the steps comprising;

selecting a variational approximation parameter that is dependent on the selected word, an occurrence number for the word and the set of part of speech tags wherein the variational parameter is trained from a sparse prior distribution of probability distributions that describe a probability of a part-of-speech tag given a word;

determining a separate value for each part-of-speech tag in the set of part-of-speech tags by using the selected variational approximation parameter;

selecting from the set of part-of-speech tags the part-of-speech tag with the largest value;

computing a score using the selected part-of-speech tag; and

weighting the score by the probability of the set of part-of-speech tags;

using the weighted scores to select a part-of-speech tag for the selected word; and

storing the selected part-of-speech tag for the selected word.

2. The method of claim 1 wherein the features of the selected word further comprise whether the selected word is capitalized in the text, whether the selected word contains a hyphen and whether the selected word contains a digit character.

3. The method of claim 1 wherein using the weighted scores to select a part-of-speech tag comprises selecting the set of part-of-speech tags that produces the largest weighted score and selecting the part-or-speech tag in the selected set of part-of-speech tags that is associated with the largest value in the set of part-of-speech tags.

4. The method of claim 1 wherein the model is trained based on entries in a dictionary, each entry identifying features of a word and a set of part-of-speech tags for the word, the dictionary lacking an entry for the selected word in the text.

5. The method of 4 wherein the model is trained by forming partial counts of part-of-speech tags based on a probability of a part-of-speech tag given a set of features.

6. A method comprising:

receiving a text;

selecting a first word in the text;

retrieving an entry for the first word from a dictionary stored on a computer-readable storage medium, the entry indicating a set of part-of-speech tags associated with the first word;

using the set of part-of-speech tags from the entry to identify a part-of-speech tag for the first word wherein using the set of part-of-speech tags from the entry to identify a part-of-speech tag for the first word comprises selecting a part-of-speech tag from the set of part-of-speech tags and computing a value for the selected part-of-speech tag using a variational approximation parameter that is selected based on an occurrence number of the first word and that describes a probability distribution of the part-of-speech tag, wherein the variational approximation parameter is trained based in part on a sparse prior distribution of probability distributions that provide a probability of a part-of-speech tag given a word;

storing the part-of-speech tag for the first word on a computer-readable storage medium;

selecting a second word in the text;

determining that the dictionary does not have an entry for the second word;

with a processor, selecting a part-of-speech tag for the second word based in part on probabilities of sets of part-of-speech tags given features of the second word; and

storing the part-of-speech tag for the second word on a computer-readable storage medium.

7. The method of claim 6 wherein selecting a part-of-speech tag for the second word based in part on probabilities of sets of part-of-speech tags given features of the second word comprises determining a score for each part-of-speech tag in a set of part-of-speech tags, determining which score is a maximum score, weighting the maximum score by the probability of the set of part-of-speech tags given the features of the second word to form a set score for the set of part-of-speech tags, selecting a set of part-of-speech tags based on the set score, and selecting the part-of-speech tag associated with the maximum score of the selected set of part-of-speech tags.

8. The method of claim 7 wherein features of the second word comprise whether the word is capitalized, whether the word contains a hyphen, whether the word contains a digit, and the suffix of the word.