Language Modeling Using Entities

• US 20150340024A1
• Filed: 05/11/2015
• Published: 11/26/2015
• Est. Priority Date: 05/23/2014
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Schogol, Vladislav, Mengibar, Pedro J. Moreno
• Application Number:
  US14/708,987
• Publication Number:
  US 20150340024A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0633 using lexical or orthographic knowledge sources

Abstract

Among other things, this document describes a computer-implemented method. The method can include obtaining a plurality of text samples. For each of one or more text samples in the plurality of text samples, the text sample can be annotated with one or more labels that indicate respective classes to which one or more terms in the text sample are assigned, wherein annotating the text sample comprises determining that at least one term in the text sample corresponds to a first entity in a data structure of interconnected entities and determining a classification of the first entity within the data structure of interconnected entities. The method can include generating a class-based training set of text samples. A class-based language model can be trained using the class-based training set of text samples. A plurality of class-specific language models can be trained.

Claims

1. A computer-implemented method, comprising:

obtaining a plurality of text samples;

for each of one or more text samples in the plurality of text samples;

determining that at least one term in the text sample corresponds to a first entity in a data structure of entities, wherein the data structure includes representations of a plurality of entities and defines relationships among particular ones of the plurality of entities;

determining classes to which the first entity within the data structure of entities belongs; and

annotating the text sample with one or more labels that indicate respective classes to which the first entity corresponding to the at least one term belongs;

generating a class-based training set of text samples by substituting the one or more terms in the one or more text samples with respective class identifiers for the one or more terms that correspond to the respective labels for the one or more terms;

training a class-based language model using the class-based training set of text samples;

training a plurality of class-specific language models; and

performing speech recognition on an utterance using the class-based language model and at least one class-specific language model from among the plurality of class-specific language models.

2. The computer-implemented method of claim 1, wherein the data structure of entities is represented by a graph of interconnected nodes that correspond to respective entities represented in the data structure.

3. The computer-implemented method of claim 1, wherein annotating the text sample comprises identifying multiple classifications for the first entity, and selecting a particular classification from among the multiple classifications that the first entity is most strongly associated with.

4. The computer-implemented method of claim 1, wherein the data structure of entities maps relationships among entities in the data structure and identifies one or more attributes of particular ones of the entities in the data structure.

5. The computer-implemented method of claim 4, further comprising determining that a second term in a first text sample being annotated corresponds to a first attribute of one or more entities in the data structure of entities, wherein annotating the first text sample comprises determining a label for the second term in the first text sample based on the first attribute of the one or more entities in the data structure.

6. The computer-implemented method of claim 1, further comprising generating a plurality of class-specific training sets of text samples using terms from the one or more text samples that were substituted out for the class identifiers in the class-based training set of text samples,

wherein one or more class-specific language models from among the plurality of class-specific language models are trained using class-specific training sets of text samples.

7. The computer-implemented method of claim 1, further comprising repeatedly re-training the plurality of class-specific language models using dynamically updated training sets of text samples.

8. The computer-implemented method of claim 7, wherein the dynamically updated training sets of text samples used to repeatedly re-train the plurality of class-specific language models are generated using entities identified from a data structure of entities.

9. The computer-implemented method of claim 8, wherein the data structure of entities is an emergent data structure that reflects updated knowledge over time such that additional entities are identified from the data structure for at least some of the times that the updated training sets of text samples are generated.

10. The computer-implemented method of claim 1, wherein performing speech recognition on the utterance using the class-based language model and the at least one class-specific language model comprises:

transcribing, using the class-based language model, one or more sequences of terms in the utterance;

identifying a particular term in the utterance that is adjacent to the one or more sequences of terms in the utterance that have been transcribed;

determining, based on the one or more sequences of terms in the utterance that have been transcribed, one or more classes to which the particular term likely belongs; and

transcribing the particular term using the at least one class-specific language model, wherein the at least one class-specific language model is selected based on the one or more classes to which the particular term is determined to likely belong.

11. The computer-implemented method of claim 10, wherein the one or more classes to which the particular term likely belongs are determined further based on one or more contextual signals associated with the utterance other than content of the utterance.

12. The computer-implemented method of claim 10, wherein transcribing the particular term using the at least one class-specific language model comprises determining that the particular term is an entity or an attribute of an entity in a data structure of entities.

13. The computer-implemented method of claim 1, wherein performing speech recognition on the utterance comprises generating a transcription of the utterance and labeling one or more terms in the transcription based on one or more class-specific language models that were used to transcribe respective ones of the one or more terms.

14. The computer-implemented method of claim 13, wherein the one or more terms in the transcription are labeled with respective classes for the one or more terms that correspond to classes of entities in a data structure of entities.

15. One or more computer-readable devices having instructions stored thereon that, when executed by one or more processors, cause performance of operations comprising:

obtaining a plurality of text samples;

for each of one or more text samples in the plurality of text samples;

determining that at least one term in the text sample corresponds to a first entity in a data structure of entities, wherein the data structure includes representations of a plurality of entities and defines relationships among particular ones of the plurality of entities;

determining classes to which the first entity within the data structure of entities belongs; and

annotating the text sample with one or more labels that indicate respective classes to which the first entity corresponding to the at least one term belongs;

generating a class-based training set of text samples by substituting the one or more terms in the one or more text samples with respective class identifiers for the one or more terms that correspond to the respective labels for the one or more terms;

training a class-based language model using the class-based training set of text samples;

training a plurality of class-specific language models; and

performing speech recognition on an utterance using the class-based language model and at least one class-specific language model from among the plurality of class-specific language models.

16. The one or more computer-readable devices of claim 15, wherein the data structure of entities is represented by a graph of interconnected nodes that correspond to respective entities represented in the data structure.

17. The one or more computer-readable devices of claim 15, wherein performing speech recognition on the utterance using the class-based language model and the at least one class-specific language model comprises:

transcribing, using the class-based language model, one or more sequences of terms in the utterance;

identifying a particular term in the utterance that is adjacent to the one or more sequences of terms in the utterance that have been transcribed;

determining, based on the one or more sequences of terms in the utterance that have been transcribed, one or more classes to which the particular term likely belongs; and

transcribing the particular term using the at least one class-specific language model, wherein the at least one class-specific language model is selected based on the one or more classes to which the particular term is determined to likely belong.

18. The one or more computer-readable devices of claim 17, wherein transcribing the particular term using the at least one class-specific language model comprises determining that the particular term is an entity or an attribute of an entity in a data structure of entities.

19. A system, comprising:

one or more computers configured to provide;

a data structure that includes representations of a plurality of entities and that maps relationships among particular ones of the plurality of entities;

an entity classifier that assigns particular entities from among the plurality of entities in the data structure to one or more respective classes;

one or more corpora of text samples;

a named-entity recognition engine that identifies particular terms in a first set of text samples that correspond to entities represented in the data structure;

a training sample generator that generates a training set of text samples by replacing the particular terms in the first set of text samples with class identifiers that indicate respective classes for the particular terms that are determined based on the classes that the entity classifier has assigned to the entities represented in the data structure that correspond to the particular terms; and

a training engine that generates one or more language models using the training set of text samples.

20. The system of claim 19, wherein the training engine generates a class-based language model using the training set of text samples and one or more class-specific language models using the particular terms that were substituted out of the training set of text samples.