Large Scale Distributed Syntactic, Semantic and Lexical Language Models

• US 20130325436A1
• Filed: 05/29/2012
• Published: 12/05/2013
• Est. Priority Date: 05/29/2012
• Status: Abandoned Application

• Current Assignee: Wright State University
• Original Assignee: Wright State University
• Inventors: Wang, Shaojun, Tan, Ming
• Application Number:
  US13/482,529
• Publication Number:
  US 20130325436A1
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A composite language model may include a composite word predictor. The composite word predictor may include a first language model and a second language model that are combined according to a directed Markov random field. The composite word predictor can predict a next word based upon a first set of contexts and a second set of contexts. The first language model may include a first word predictor that is dependent upon the first set of contexts. The second language model may include a second word predictor that is dependent upon the second set of contexts. Composite model parameters can be determined by multiple iterations of a convergent N-best list approximate Expectation-Maximization algorithm and a follow-up Expectation-Maximization algorithm applied in sequence, wherein the convergent N-best list approximate Expectation-Maximization algorithm and the follow-up Expectation-Maximization algorithm extracts the first set of contexts and the second set of contexts from a training corpus.

Claims

1. A composite language model comprising a composite word predictor, wherein:

the composite word predictor is stored in one or more memories, and comprises a first language model and a second language model that are combined according to a directed Markov random field;

the composite word predictor predicts, automatically with one or more processors that are communicably coupled to the one or more memories, a next word based upon a first set of contexts and a second set of contexts;

the first language model comprises a first word predictor that is dependent upon the first set of contexts;

the second language model comprises a second word predictor that is dependent upon the second set of contexts; and

composite model parameters are determined by multiple iterations of a convergent N-best list approximate Expectation-Maximization algorithm and a follow-up Expectation-Maximization algorithm applied in sequence, wherein the convergent N-best list approximate Expectation-Maximization algorithm and the follow-up Expectation-Maximization algorithm extracts the first set of contexts and the second set of contexts from a training corpus.

2. The composite language model of claim 1, wherein:

the composite word predictor further comprises a third language model that is combined with the first language model and the second language model according to the directed Markov random field;

the composite word predictor predicts the next word based upon a third set of contexts;

the third language model comprises a third word predictor that is dependent upon the third set of contexts; and

the convergent N-best list approximate Expectation-Maximization algorithm and the follow-up Expectation-Maximization algorithm extracts the third set of contexts from the training corpus.

3. The composite language model of claim 2, wherein the first language model is a Markov chain source model, the second language model is a probabilistic latent semantic analysis model, and the third language model is a structured language model.

4. The composite language model of claim 1, wherein the convergent N-best list approximate Expectation-Maximization algorithm and the follow-up Expectation-Maximization algorithm are stored and executed by a plurality of machines.

5. The composite language model of claim 1, wherein the first language model is a Markov chain source model, and the second language model is a probabilistic latent semantic analysis model.

6. The composite language model of claim 1, wherein the first language model is a Markov chain source model, and the second language model is a structured language model.

7. The composite language model of claim 1, wherein the first language model is a probabilistic latent semantic analysis model, and the second language model is a structured language model.