Discriminative training of automatic speech recognition models with natural language processing dictionary for spoken language processing

• US 10,140,976 B2
• Filed: 12/14/2015
• Issued: 11/27/2018
• Est. Priority Date: 12/14/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Suzuki, Masayuki, Itoh, Nobuyasu, Kurata, Gakuto, Nagano, Tohru
• Primary Examiner(s): Zhu, Richard
• Application Number:
  US14/968,439
• Publication Number:
  US 20170169813A1
• Time in Patent Office: 1,079 Days
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/0638 Interactive procedures

Abstract

Methods and systems for language processing includes training one or more automatic speech recognition models using an automatic speech recognition dictionary. A set of N automatic speech recognition hypotheses for an input is determined, based on the one or more automatic speech recognition models, using a processor. A best hypothesis is selected using a discriminative language model and a list of relevant words. Natural language processing is performed on the best hypothesis.

Claims

1. A method for language processing, comprising:

training one or more automatic speech recognition models using an automatic speech recognition dictionary and speech recognition training data;

determining a set of N automatic speech recognition hypotheses that characterize a spoken input, based on the one or more automatic speech recognition models, using a processor;

selecting a hypothesis from the set of N automatic speech recognition hypotheses using a discriminative language model and a first natural language processing dictionary that excludes words having little discriminatory value according to an error rate of only words other than words having little likely effect on the natural language outcome in each hypothesis; and

performing natural language processing on the selected hypothesis using a second natural language processing dictionary that is different from the automatic speech recognition dictionary and the first natural language processing dictionary.

2. The method for language processing of claim 1, further comprising determining the first natural language processing dictionary using natural language processing training data and the automatic speech recognition dictionary by trimming out words having little likely effect on the natural language outcome from the automatic speech recognition dictionary.

3. The method for language processing of claim 2, wherein determining the first natural language processing dictionary comprises concatenating all words in the natural language processing training data to generate raw natural language processing text.

4. The method for language processing of claim 3, further comprising tokenizing the raw natural language processing text using the automatic speech recognition dictionary.

5. The method for language processing of claim 4, further comprising collecting tokenized words that appear more than a threshold number of times.

6. The method for language processing of claim 1, wherein selecting the hypothesis comprises determining a word error rate for each hypothesis that considers only errors in words of the respective hypothesis that are in the first natural language processing dictionary using the discriminative language model.

7. The method for language processing of claim 6, further comprising selecting the hypothesis having the lowest word error rate.

8. The method for language processing of claim 1, wherein training the one or more automatic speech recognition models comprises training an acoustic model and a language mode.

9. A computer readable storage medium comprising a computer readable program for language processing, wherein the computer readable program when executed on a computer causes the computer to perform the steps of claim 1.

10. A method for language processing, comprising:

training one or more automatic speech recognition models using an automatic speech recognition dictionary and speech recognition training data;

determining a set of N automatic speech recognition hypotheses that characterize a spoken input, based on the one or more automatic speech recognition models, using a processor, comprising;

concatenating all words in the natural language processing training data to generate raw natural language processing text;

tokenizing the raw natural language processing text using the automatic speech recognition dictionary;

collecting tokenized words that appear more than a threshold number of times; and

forming a first natural language processing dictionary that excludes words having little discriminatory value using the collected tokenized words;

selecting a hypothesis from the set of N automatic speech recognition hypotheses using a discriminative language model and the first natural language processing dictionary input according to an error rate of only words other than words having little likely effect on the natural language processing outcome in each hypothesis, said selection comprising;

determining a word error rate for each hypothesis that considers only the first natural language processing dictionary using the discriminative language model; and

selecting the hypothesis having the lowest word error rate; and

performing natural language processing on the selected hypothesis using a second natural language processing dictionary that is different from the automatic speech recognition dictionary and the first natural language processing dictionary.

11. A system for language processing, comprising:

an automatic speech recognition module comprising a processor configured to train one or more automatic speech recognition models using an automatic speech recognition dictionary and speech recognition training data, to determine a set of N automatic speech recognition hypotheses that characterize a spoken input based on the one or more automatic speech recognition models, and to select a hypothesis from the set of N automatic speech recognition hypotheses using a discriminative language model and first natural language processing dictionary that excludes words having little discriminatory value according to an error rate of only words other than words having little likely effect on the natural language processing outcome in each hypothesis; and

a natural language processing module configured to perform natural language processing on the selected hypothesis using a second natural language processing dictionary that is different from the automatic speech recognition dictionary and the first natural language processing dictionary.

12. The system of claim 11, wherein the automatic speech recognition module is further configured to determine the first natural language processing dictionary using natural language processing training data and the automatic speech recognition dictionary by trimming out words having little likely effect on the natural language outcome from the automatic speech recognition dictionary.

13. The system for language processing of claim 12, wherein the automatic speech recognition module is further configured to concatenate all words in the natural language processing training data to generate raw natural language processing text.

14. The system for language processing of claim 13, wherein the automatic speech recognition module is further configured to tokenize the raw natural language processing text using the automatic speech recognition dictionary.

15. The system for language processing of claim 14, wherein the automatic speech recognition module is further configured to collect tokenized words that appear more than a threshold number of times.

16. The system for language processing of claim 11, wherein the automatic speech recognition module is further configured to determine a word error rate for each hypothesis that considers only errors in words of the respective hypothesis that are in the first natural language processing dictionary using the discriminative language model.

17. The system for language processing of claim 16, wherein the automatic speech recognition module is further configured to select the hypothesis having the lowest word error rate.

18. The system for language processing of claim 11, wherein the automatic speech recognition module is further configured to train an acoustic model and a language mode.