SYSTEMS AND METHODS FOR PROVIDING METADATA-DEPENDENT LANGUAGE MODELS

• US 20140324434A1
• Filed: 04/25/2013
• Published: 10/30/2014
• Est. Priority Date: 04/25/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Vozila, Paul J., Lenke, Nils, Tam, Wilson
• Granted Patent: US 9,626,960 B2
• Time in Patent Office: Days
• US Class Current: 704/257
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/226 using non-speech characteristics

Abstract

Techniques for generating language models. The techniques include: obtaining language data comprising training data and associated values for one or more metadata attributes, the language data comprising a plurality of instances of language data, an instance of language data comprising an instance of training data and one or more metadata attribute values associated with the instance of training data; identifying, by processing the language data using at least one processor, a set of one or more of the metadata attributes to use for clustering the instances of training data into a plurality of clusters; clustering the training data instances based on their respective values for the identified set of metadata attributes into the plurality of clusters; and generating a language model for each of the plurality of clusters.

Claims

1. A method for generating a plurality of language models, the method comprising acts of:

(A) obtaining language data comprising training data and associated values for one or more metadata attributes, the language data comprising a plurality of instances of language data, an instance of language data comprising an instance of training data and one or more metadata attribute values associated with the instance of training data;

(B) identifying, by processing the language data using at least one processor, a set of one or more of the metadata attributes to use for clustering the instances of training data into a plurality of clusters;

(C) clustering the training data instances based on their respective values for the identified set of metadata attributes into the plurality of clusters; and

(D) generating a language model for each of the plurality of clusters.

2. The method of claim 1, wherein the one or more metadata attributes comprise a plurality of metadata attributes, wherein the act (B) comprises:

automatically evaluating multiple of the plurality of metadata attributes, the automatically evaluating a first of the multiple metadata attributes comprising generating at least one language model for at least one group of training data instances obtained by dividing the training data instances based on their respective values for the first metadata attribute; and

identifying the set of metadata attributes based on results of the evaluation.

3. The method of claim 2, wherein automatically evaluating the first metadata attribute comprises:

dividing the training data instances, based on their respective values for the first metadata attribute, to obtain two or more groups of training data instances;

generating a child language model for each of the obtained groups of training data instances;

generating a parent language model using all the training data instances; and

calculating a score for the first metadata attribute based at least in part how a measure of the goodness of fit of the generated child language models compares with a measure of the goodness of fit of the generated parent language model.

4. The method of claim 3, wherein generating the child language model for each of the obtained groups comprises generating a unigram language model for each of the obtained groups.

5. The method of claim 3, wherein the measure of the goodness of fit of the generated parent language model is a likelihood, according to the generated parent language model, of the training data instances used to generate the parent language model.

6. The method of claim 1, wherein the set of metadata attributes comprises a first subset of metadata attributes and a second subset of metadata attributes, and wherein:

the act (C) comprises dividing the training data instances based on their respective values for the first subset of metadata attributes into a plurality of groups; and

the act (D) comprises generating a basis language model for each of the plurality of groups.

7. The method of claim 6, wherein:

the act (C) further comprises dividing at least one of the plurality of groups of training data instances based on their respective values for the second subset of metadata attributes to obtain at least a first cluster of the plurality of clusters; and

the act (D) further comprises generating a mixture language model for the first cluster at least in part by calculating mixture weights for at least two of the plurality of generated basis language models.

8. A system comprising:

at least one processor configured to perform acts of;

(A) obtaining language data comprising training data and associated values for one or more metadata attributes, the language data comprising a plurality of instances of language data, an instance of language data comprising an instance of training data and one or more metadata attribute values associated with the instance of training data;

(B) identifying, by processing the language data, a set of one or more of the metadata attributes to use for clustering the instances of training data into a plurality of clusters;

(C) clustering the training data instances based on their respective values for the identified set of metadata attributes into the plurality of clusters; and

(D) generating a language model for each of the plurality of clusters.

9. The system of claim 8, wherein the one or more metadata attributes comprise a plurality of metadata attributes, wherein the act (B) comprises:

automatically evaluating multiple of the plurality of metadata attributes, the automatically evaluating a first of the multiple metadata attributes comprising generating at least one language model for at least one group of training data instances obtained by dividing the training data instances based on their respective values for the first metadata attribute; and

identifying the set of metadata attributes based on results of the evaluation.

10. The system of claim 9, wherein automatically evaluating the first metadata attribute comprises:

dividing the training data instances, based on their respective values for the first metadata attribute, to obtain two or more groups of training data instances;

generating a child language model for each of the obtained groups of training data instances;

generating a parent language model using all the training data instances; and

calculating a score for the first metadata attribute based at least in part how a measure of the goodness of fit of the generated child language models compares with a measure of the goodness of fit of the generated parent language model.

11. The system of claim 10, wherein generating the child language model for each of the obtained groups comprises generating a unigram language model for each of the obtained groups.

12. The system of claim 10, wherein the measure of the goodness of fit of the generated parent language model is a likelihood, according to the generated parent language model, of the training data instances used to generate the parent language model.

13. The system of claim 8, wherein the set of metadata attributes comprises a first subset of metadata attributes and a second subset of metadata attributes, and wherein:

the act (C) comprises dividing the training data instances based on their respective values for the first subset of metadata attributes into a plurality of groups; and

the act (D) comprises generating a basis language model for each of the plurality of groups.

14. The system of claim 13, wherein:

the act (C) further comprises dividing at least one of the plurality of groups of training data instances based on their respective values for the second subset of metadata attributes to obtain at least a first cluster of the plurality of clusters; and

the act (D) further comprises generating a mixture language model for the first cluster at least in part by calculating mixture weights for at least two of the plurality of generated basis language models.

15. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to perform a method comprising acts of:

(A) obtaining language data comprising training data and associated values for one or more metadata attributes, the language data comprising a plurality of instances of language data, an instance of language data comprising an instance of training data and one or more metadata attribute values associated with the instance of training data;

(B) identifying, by processing the language data, a set of one or more of the metadata attributes to use for clustering the instances of training data into a plurality of clusters;

(C) clustering the training data instances based on their respective values for the identified set of metadata attributes into the plurality of clusters; and

(D) generating a language model for each of the plurality of clusters.

16. The at least one non-transitory computer-readable storage medium of claim 15, wherein the one or more metadata attributes comprise a plurality of metadata attributes, wherein the act (B) comprises:

automatically evaluating multiple of the plurality of metadata attributes, the automatically evaluating a first of the multiple metadata attributes comprising generating at least one language model for at least one group of training data instances obtained by dividing the training data instances based on their respective values for the first metadata attribute; and

identifying the set of metadata attributes based on results of the evaluation.

17. The at least one non-transitory computer-readable storage medium of claim 16, wherein automatically evaluating the first metadata attribute comprises:

dividing the training data instances, based on their respective values for the first metadata attribute, to obtain two or more groups of training data instances;

generating a child language model for each of the obtained groups of training data instances;

generating a parent language model using all the training data instances; and

calculating a score for the first metadata attribute based at least in part how a measure of the goodness of fit of the generated child language models compares with a measure of the goodness of fit of the generated parent language model.

18. The at least one non-transitory computer-readable storage medium of claim 17, wherein generating the child language model for each of the obtained groups comprises generating a unigram language model for each of the obtained groups.

19. The at least one non-transitory computer-readable storage medium of claim 18, wherein the set of metadata attributes comprises a first subset of metadata attributes and a second subset of metadata attributes, and wherein:

the act (C) comprises dividing the training data instances based on their respective values for the first subset of metadata attributes into a plurality of groups; and

the act (D) comprises generating a basis language model for each of the plurality of groups.

20. The at least one non-transitory computer-readable storage medium of claim 19, wherein:

the act (C) further comprises dividing at least one of the plurality of groups of training data instances based on their respective values for the second subset of metadata attributes to obtain at least a first cluster of the plurality of clusters; and

the act (D) further comprises generating a mixture language model for the first cluster at least in part by calculating mixture weights for at least two of the plurality of generated basis language models.