Large Scale Distributed Syntactic, Semantic and Lexical Language Models
First Claim
1. A composite language model comprising a composite word predictor, wherein:
- the composite word predictor is stored in one or more memories, and comprises a first language model and a second language model that are combined according to a directed Markov random field;
the composite word predictor predicts, automatically with one or more processors that are communicably coupled to the one or more memories, a next word based upon a first set of contexts and a second set of contexts;
the first language model comprises a first word predictor that is dependent upon the first set of contexts;
the second language model comprises a second word predictor that is dependent upon the second set of contexts; and
composite model parameters are determined by multiple iterations of a convergent N-best list approximate Expectation-Maximization algorithm and a follow-up Expectation-Maximization algorithm applied in sequence, wherein the convergent N-best list approximate Expectation-Maximization algorithm and the follow-up Expectation-Maximization algorithm extracts the first set of contexts and the second set of contexts from a training corpus.
2 Assignments
0 Petitions
Accused Products
Abstract
A composite language model may include a composite word predictor. The composite word predictor may include a first language model and a second language model that are combined according to a directed Markov random field. The composite word predictor can predict a next word based upon a first set of contexts and a second set of contexts. The first language model may include a first word predictor that is dependent upon the first set of contexts. The second language model may include a second word predictor that is dependent upon the second set of contexts. Composite model parameters can be determined by multiple iterations of a convergent N-best list approximate Expectation-Maximization algorithm and a follow-up Expectation-Maximization algorithm applied in sequence, wherein the convergent N-best list approximate Expectation-Maximization algorithm and the follow-up Expectation-Maximization algorithm extracts the first set of contexts and the second set of contexts from a training corpus.
-
Citations
7 Claims
-
1. A composite language model comprising a composite word predictor, wherein:
-
the composite word predictor is stored in one or more memories, and comprises a first language model and a second language model that are combined according to a directed Markov random field; the composite word predictor predicts, automatically with one or more processors that are communicably coupled to the one or more memories, a next word based upon a first set of contexts and a second set of contexts; the first language model comprises a first word predictor that is dependent upon the first set of contexts; the second language model comprises a second word predictor that is dependent upon the second set of contexts; and composite model parameters are determined by multiple iterations of a convergent N-best list approximate Expectation-Maximization algorithm and a follow-up Expectation-Maximization algorithm applied in sequence, wherein the convergent N-best list approximate Expectation-Maximization algorithm and the follow-up Expectation-Maximization algorithm extracts the first set of contexts and the second set of contexts from a training corpus. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
Specification