Language model adaptation using semantic supervision

• US 7,478,038 B2
• Filed: 03/31/2004
• Issued: 01/13/2009
• Est. Priority Date: 03/31/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tam, Yik-Cheung, Chelba, Ciprian, Acero, Alejandro, Mahajan, Milind
• Primary Examiner(s): Vo; Huyen X.
• Application Number:
  US10/814,906
• Publication Number:
  US 20050228641A1
• Time in Patent Office: 1,749 Days
• Field of Search: 704/9, 704/10, 704/1, 704/4, 704/251, 704/255, 704/257, 704/250
• US Class Current: 704/10
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

A method and apparatus are provided for adapting a language model. The method and apparatus provide supervised class-based adaptation of the language model utilizing in-domain semantic information.

Claims

1. A method of adapting an n-gram language model for a new domain, the method comprising:

receiving background data indicative of general text phrases not directed to the new domain;

receiving a set of semantic entities used in the new domain and organized in classes;

generating background n-gram class count data based on the background data and the semantic entities and classes thereof;

receiving adaptation data indicative of text phrases used in the new domain;

generating adaptation n-gram class count data based on the adaptation data and the semantic entities and classes thereof;

training a language model based on the background n-gram class count data and the adaptation n-gram class count data; and

embodying the language model in tangible form.

2. The method of claim 1 and further comprising:

generating background n-gram word data based on the background n-gram class count data and the semantic entities and classes thereof;

generating adaptation n-gram word data based on the adaptation n-gram class count data and the semantic entities and classes thereof; and

wherein training the language model based on the background n-gram class count data and the adaptation n-gram class count data comprises using background n-gram word data and adaptation n-gram word data.

3. The method of claim 2 wherein generating background n-gram word data comprises generating background n-gram word data for multi-word semantic entities with each data entry comprising a selected number of words.

4. The method of claim 3 wherein generating adaptation n-gram word data comprises generating adaptation n-gram word data for multi-word semantic entities with each data entry comprising a selected number of words.

5. The method of claim 4 wherein generating adaptation n-gram class count data based on the adaptation data and the semantic entities and classes thereof comprises tagging word level adaptation data based on the semantic entities and classes thereof.

6. The method of claim 5 wherein generating adaptation n-gram class count data based on the adaptation data and the semantic entities and classes thereof comprises counting unique class level n-grams of the tagged adaptation data.

7. The method of claim 6 wherein generating adaptation n-gram class count data based on the adaptation data and the semantic entities and class thereof comprises discarding some class n-grams from the tagged adaptation data.

8. The method of claim 3 wherein generating background n-gram class count data based on the background data and the semantic entities and classes thereof comprises tagging word level background data based on the semantic entities and classes thereof.

9. The method of claim 8 wherein generating background n-gram class count data based on the background data and the semantic entities and classes thereof comprises counting unique class level n-grams of the tagged background data.

10. The method of claim 9 wherein generating background n-gram class count data based on the background data and the semantic entities and classes thereof comprises discarding some class n-grams from the tagged background data.

11. The method of claim 2 and further comprising:

wherein generating background n-gram word data comprises generating background n-gram word data for multi-word semantic entities with each data entry comprising a selected number of words;

wherein generating adaptation n-gram word data comprises generating adaptation n-gram word data for multi-word semantic entities with each data entry comprising a selected number of words;

wherein generating background n-gram class count data based on the background data and the semantic entities and classes thereof comprises tagging word level background data based on the semantic entities and classes thereof; and

wherein generating adaptation n-gram class count data based on the adaptation data and the semantic entities and classes thereof comprises tagging word level adaptation data based on the semantic entities and classes thereof.

12. A computer-readable storage medium having computer-executable instructions for performing steps to generate a language model, the steps comprising:

receiving a set of semantic entities used in a selected domain and organized in classes;

receiving background n-grams class count data correlated to classes of the set of semantic entities and based on background data indicative of general text;

receiving adaptation n-gram class count data correlated to classes of the set of semantic entities and based on adaptation data indicative of a selected domain to be modeled;

training a language model based on the background n-gram class count data, the adaptation n-gram class count data and the set of semantic entities; and

wherein training the language model comprises computing background word count data based on the background n-gram class count data and the set of semantic entities, computing adaptation word count data based on the adaptation n-gram class count data and the set of semantic entities, and smoothing the n-gram relative frequencies.

13. The computer-readable storage medium of claim 12 wherein smoothing comprises using a deleted-interpolation algorithm.