Method and apparatus for distribution-based language model adaptation

• US 20060009965A1
• Filed: 09/13/2005
• Published: 01/12/2006
• Est. Priority Date: 10/13/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Mingjing, Gao, Jianfeng
• Granted Patent: US 7,254,529 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

A method and apparatus are provided for adapting a language model to a task-specific domain. Under the method and apparatus, the relative frequency of n-grams in a small training set (i.e. task-specific training data set) and the relative frequency of n-grams in a large training set (i.e. out-of-domain training data set) are used to weight a distribution count of n-grams in the large training set. The weighted distributions are then used to form a modified language model by identifying probabilities for n-grams from the weighted distributions.

Claims

1. A method of forming a language model, the method comprising:

selecting out-of-task training data having n-gram counts;

selecting task-specific training data having n-gram counts;

modifying an n-gram count in the out-of-task training data by applying an n-gram-specific weight that is based in part on an n-gram count in the task-specific training data to form modified training data; and

identifying probabilities for the language model based on the modified training data.

2. The method of claim 1 wherein applying a weight comprises forming a weight based on a relative frequency of an n-gram in the task-specific training data.

3. The method of claim 2 wherein forming a weight further comprises forming a weight based on a relative frequency of the n-gram in the out-of-task data.

4. The method of claim 1 wherein forming modified data comprises using the n-gram count from the task-specific training data only for forming the n-gram-specific weight.

5. The method of claim 4 wherein identifying probabilities for the language model comprises using only the modified data.

6. A computer-readable medium having computer-executable instructions for forming a language model through steps comprising:

determining a distribution of entities in a small set of training data;

changing a distribution of entities in a large set of training data based on an entity-specific weight that is formed in part from a distribution of entities in the small set of training data to form a modified distribution of entities; and

using the modified distribution of entities to identify probabilities for the language model.

7. The computer-readable medium of claim 6 wherein the entities comprise n-grams.

8. The computer-readable medium of claim 6 wherein the entity-specific weight is formed based in part on a relative frequency of entities in the small set of training data.

9. The computer-readable medium of claim 8 wherein the entity-specific weight is further formed based on a relative frequency of entities in the large set of training data.

10. The computer-readable medium of claim 9 wherein changing a distribution further comprises applying the entity-specific weight to a count of entities in the large set of training data.

11. A method of adapting a general language model to cover a task-specific domain, the method comprising:

determining a weight based on the relative frequency of an n-gram in the task-specific domain;

multiplying the weight by a count of the n-gram in a distribution domain associated with the general language model to form a modified count, the modified count formed without adding counts of n-grams in the task-specific domain;

using only the modified counts of n-grams to determine a probability that forms part of an adapted language model.

12. The method of claim 11 wherein determining a weight comprises determining a separate weight for each of a plurality of n-grams.