Apparatus and method for model adaptation for spoken language understanding

• US 8,731,924 B2
• Filed: 08/08/2011
• Issued: 05/20/2014
• Est. Priority Date: 03/21/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Tur, Gokhan
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US13/205,057
• Publication Number:
  US 20110295602A1
• Time in Patent Office: 1,016 Days
• Field of Search: 704/244
• US Class Current: 704/244
• CPC Class Codes: G10L 15/065 Adaptation

Abstract

An apparatus and a method are provided for building a spoken language understanding model. Labeled data may be obtained for a target application. A new classification model may be formed for use with the target application by using the labeled data for adaptation of an existing classification model. In some implementations, the existing classification model may be used to determine the most informative examples to label.

Claims

1. A method comprising:

receiving a first language model for a first domain having a first training set;

based on a second language model, which is smaller than the first language model, selectively sampling training data associated with a second domain from a second training set, to yield sampled training data;

labeling, via a processor, the sampled training data, to yield second domain labeled data; and

concatenating, via the processor, the first language model by replacing a portion of the first training set in the first language model with the second domain labeled data.

2. The method of claim 1, wherein concatenating the first language model further comprises:

determining a distance from the first language model to the second domain labeled data; and

modifying the first language model using the distance.

3. The method of claim 2, wherein the first language model is a speech processing model and wherein the distance comprises a logistic loss function.

4. The method of claim 1, further comprising:

obtaining confidence scores from the first language model and a modified first language model; and

engaging in active learning using the confidence scores.

5. The method of claim 1, wherein modifying the first language model utilizes one of a Boosting algorithm, a Naï

ve Bayes classifier, a linear model interpolation, and a Bayesian adaptation.

6. The method of claim 5, wherein probability distributions correspond to an existing language model probability distribution and a modified language model probability distribution.

7. The method of claim 1, further comprising labeling future utterances using a modified first language model.

8. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

receiving a first language model for a first domain having a first training set;

based on a second language model, which is smaller than the first language model, selectively sampling training data associated with a second domain from a second training set, to yield sampled training data;

labeling the sampled training data, to yield second domain labeled data; and

concatenating the first language model by replacing a portion of the first training set in the first language model with the second domain labeled data.

9. The system of claim 8, wherein concatenating the first language model further comprises:

determining a distance from the first language model to the second domain labeled data; and

modifying the first language model using the distance.

10. The system of claim 9, wherein the first language model is a speech processing model and wherein the distance comprises a logistic loss function.

11. The system of claim 8, the computer-readable storage medium having additional instructions stored which result in the operations further comprising:

obtaining confidence scores from the first language model and a modified first language model; and

engaging in active learning using the confidence scores.

12. The system of claim 8, wherein modifying the first language model utilizes one of a Boosting algorithm, a Naï

ve Bayes classifier, a linear model interpolation, and a Bayesian adaptation.

13. The system of claim 12, wherein probability distributions correspond to an existing language model probability distribution and a modified language model probability distribution.

14. The system of claim 8, the computer-readable storage medium having additional instructions stored which result in the operations further comprising labeling future utterances using a modified first language model.

15. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving a first language model for a first domain having a first training set;

based on a second language model, which is smaller than the first language model, selectively sampling training data associated with a second domain from a second training set, to yield sampled training data;

labeling the sampled training data, to yield second domain labeled data; and

concatenating the first language model by replacing a portion of the first training set in the first language model with the second domain labeled data.

16. The computer-readable storage device of claim 15, wherein concatenating the existing language model further comprises:

determining a distance from the first language model to the second domain labeled data; and

modifying the first language model using the distance.

17. The computer-readable storage device of claim 16, wherein the first language model is a speech processing model and wherein the distance comprises a logistic loss function.

18. The computer-readable storage device of claim 15, the computer-readable storage medium having additional instructions stored which result in the operations further comprising:

obtaining confidence scores from the first language model and a modified first language model; and

engaging in active learning using the confidence scores.

19. The computer-readable storage device of claim 15, wherein modifying the first language model utilizes one of a Boosting algorithm, a Naï

ve Bayes classifier, a linear model interpolation, and a Bayesian adaptation.

20. The computer-readable storage device of claim 19, wherein probability distributions correspond to an existing language model probability distribution and a modified language model probability distribution.