Adapting input speech and reference patterns for changing speaker and environment

• US 5,778,340 A
• Filed: 09/07/1995
• Issued: 07/07/1998
• Est. Priority Date: 09/08/1994
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Hattori, Hiroaki
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Storm, Donald L.
• Application Number:
  US08/524,807
• Time in Patent Office: 1,034 Days
• Field of Search: 395/2.53, 395/2.61, 395/2.31, 395/2.43, 395/2.47, 395/2.52, 395/2.6, 395/2.54, 395/2.64, 395/2.42, 395/2.4, 395/2.48, 395/2.33, 704/265
• US Class Current: 704/244
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence

Abstract

Speech input is converted into a feature vector series, which is held as an input pattern in an input pattern memory and which is also fed to a preliminary recognizer. A reference pattern memory stores reference patterns of recognition subject words. The preliminary recognizer executes preliminary recognition by calculating similarity measures between the input pattern and the reference patterns stored in a reference pattern memory, and obtains top N candidates using the calculated similarity measures. A reference pattern adapter executes adaptation of the reference patterns based on the reference patterns, the input pattern, the top N candidates, and newly stores the adapted reference patterns in the reference pattern memory. A final recognizer then executes speech recognition of the input pattern by using the newly-stored reference patterns corresponding to the top N candidates.

Claims

1. A speech recognition system comprising:

a speech waveform analyzer for converting an input speech waveform into feature vectors;

an input pattern memory for storing the feature vectors as an input pattern;

a reference pattern memory for storing reference patterns of recognition subject words;

a preliminary recognizer for obtaining a similarity measure between the input pattern and each of the reference patterns, determining at least one reference pattern as at least one candidate recognition result based on the similarity measures, and outputting top N candidates, N being an integer greater than zero, using the obtained similarity measures;

a reference pattern adapter for;

(a) executing adaptation of the reference patterns based on (i) the reference patterns, (ii) the input pattern, and (iii) the top N candidates, and

(b) newly storing the adapted reference patterns as reference patterns in the reference pattern memory; and

a final recognizor for executing speech recognition to obtain one top candidate by using the adapted reference patterns and the input pattern.

2. The speech recognition system according to claim 1, further comprising a switch for inputting the top N candidates obtained in the preliminary recognizor and the result of recognition in the final or second recognizer to the reference pattern adapter, the adaptation of the reference patterns being executed by using the input pattern in the input pattern memory and the top N candidates or the result of recognition.

3. The speech recognition system according to claim 2, which further comprises an input unit for inputting a result of user'"'"'s confirmation and/or correction of the result of recognition in the final or second recognizer, the reference pattern adapter executing adaptation of the reference patterns by using the top N candidates obtained in the preliminary recognizer or the result of recognition in the final or second recognizor, the input pattern in the input pattern memory, and the reference patterns in the reference pattern memory.

4. The speech recognition system according to claim 1, wherein the adaptation of the reference patterns is once again executed by using the input pattern in the input pattern memory, the reference patterns in the reference pattern memory, and the result of recognition in the final or second recognizer.

5. The speech recognition system according to claim 4, which further comprises an input unit for inputting the result of user'"'"'s confirmation and/or correction of the result of recognition in the final or second recognizer, the reference pattern adapter executing adaptation once again of the reference patterns by using the top N candidates obtained in the preliminary recognizer or the result of recognition in the final or second recognizor, the input pattern in the input pattern memory, and the reference patterns in the reference pattern memory.

6. A speech recognition system comprising:

a speech waveform analyzer for converting an input speech waveform into feature vectors;

an input pattern memory for storing the feature vectors as an input pattern;

a reference pattern memory for storing reference patterns of recognition subject words;

a preliminary recognizer for obtaining a similarity measure between the input pattern and each of the reference patterns and outputting top N candidates, N being an integer greater than zero, using the obtained similarity measures;

an input pattern adapter for executing adaptation of the input pattern based on (i) the reference patterns, (ii) the input pattern, and (iii) the top N candidates; and

a final or second recognizer for executing speech recognition to obtain one top candidate based on (i) the reference patterns and (ii) the adapted input pattern.

7. A speech recognition system comprising:

a speech waveform analyzer for converting an input speech waveform into feature vectors;

an input pattern memory for storing the feature vectors as an input pattern;

a reference pattern memory for storing reference patterns of recognition subject words;

a preliminary recognizer for obtaining a similarity measure between the input pattern and each of the reference patterns and outputting top N candidates, N being an integer greater than zero, using the obtained similarity measures;

a reference pattern adapter for;

(a) executing adaptation of the reference patterns based on (i) the reference patterns, (ii) the input pattern, and (iii) the top N candidates, and

(b) newly storing the adapted reference patterns as reference patterns in the reference pattern memory;

an input pattern adapter for executing adaptation of the input pattern based on (i) the reference patterns, (ii) the input pattern, and (iii) the top N candidates; and

a final recognizor for executing speech recognition to obtain one top candidate based on (i) the reference patterns and (ii) the adapted input pattern.

8. The speech recognition system according to claim 7, wherein the adaptation of the reference patterns is once again executed based on the input pattern in the input pattern memory, the reference patterns in the reference pattern memory, and the recognition result in the final or second recognizer.

9. The speech recognition system according to claim 8, which further comprises an input unit for inputting the result of user'"'"'s confirmation and/or correction of the result of recognition in the final or second recognizer, the reference pattern adapter executing adaptation of the reference patterns by using the top N candidates obtained in the preliminary recognizer or the result of recognition in the final or second recognizor, the input pattern in the input pattern memory, and the reference patterns in the reference pattern memory.

10. The speech recognition system comprising:

an analyzer for converting a speech signal input into feature vector series;

an input pattern memory for storing the feature vector series as an input pattern;

a reference pattern memory for storing reference patterns;

a preliminary recognizor for executing preliminary recognition by using (i) the input pattern and (ii) all reference patterns stored in the reference pattern memory to obtain a plurality of candidates in an order of higher similarities;

a reference pattern adapter for;

(a) executing adaptation of the reference patterns by using (i) the input pattern, (ii) a plurality of candidates as a result of the preliminary recognition, and (iii) the reference patterns corresponding to the plurality of candidates, and

(b) storing the adapted reference patterns in the reference pattern memory; and

a final recognizor for executing re-recognition of the input pattern by using the adapted reference patterns corresponding to the plurality of candidates obtained as a result of the preliminary recognition.

11. The speech recognition system comprising:

an analyzer for converting a speech signal input into feature vector series;

an input pattern memory for storing the feature vector series as an input pattern;

a reference pattern memory for storing reference patterns;

a preliminary recognizer for executing a preliminary recognition based on (i) the input pattern and (ii) all reference patterns read out from the reference pattern memory to obtain top N candidates in an order of higher similarities;

a reference pattern adapter for;

(a) executing adaptation of the reference patterns based on (i) the input pattern, (ii) top N candidate reference patterns as a result of the preliminary recognition, and (iii) the reference patterns, and

(b) storing the reference patterns in a reference pattern memory; and

a final recognizer for obtaining similarity measure through (i) a matching process for the input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns corresponding to the top N candidates.

12. A speech recognition system comprising:

an analyzer for converting a speech signal input into feature vector series;

an input pattern memory for storing the feature vector series as an input pattern;

a reference pattern memory for storing reference patterns;

a preliminary recognizer for executing preliminary recognition based on (i) the input pattern and (ii) all reference patterns read out from the reference pattern memory to obtain top N candidates in an order of higher similarities;

an input pattern adapter for executing adaptation of the input pattern based on (i) the input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns; and

a final recognizer for;

(a) reading out (i) the adapted input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns corresponding to the top N candidates, and

(b) executing the recognition by a matching process.

13. A speech recognition system comprising:

an analyzer for converting a speech signal input into feature vector series;

an input pattern memory for storing the feature vector series as an input pattern;

a reference pattern memory for storing reference patterns;

a preliminary recognizer for executing a preliminary recognition based on (i) the input pattern and (ii) all reference patterns read out from the reference pattern memory to obtain top N candidates in an order of higher similarities;

an input pattern adapter for executing adaptation of the input pattern based on (i) the input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns;

a reference pattern adapter for;

(a) executing adaptation of the reference patterns based on (i) the input pattern, (ii) the top N candidates as a result of the preliminary recognition, and (iii) the reference patterns, and

(b) storing the adapted reference patterns in the reference pattern memory; and

a final recognizer for obtaining similarity measure through a matching process by reading out (i) the adapted input pattern, (ii) the top N candidate reference patterns as a result of preliminary recognition, and (iii) the reference patterns corresponding to the top N candidates.

14. A speech recognition system comprising:

an analyzer for converting a speech signal input into feature vector series;

an input pattern memory for storing the feature vector series as an input pattern;

a reference pattern memory for storing reference patterns;

a preliminary recognizer for executing a preliminary recognition based on (i) the input pattern and (ii) all reference patterns read out from the reference pattern memory to obtain top N candidates in an order of higher similarities;

a reference pattern adapter for executing adaptation of the reference patterns by;

(a) reading out (i) the top N candidates as a result of preliminary recognition, (ii) the input pattern, and (iii) the reference patterns, and

(b) storing the adapted reference patterns in the reference pattern memory;

a final recognizer for obtaining similarity measure through matching for (i) the input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns corresponding to the top N candidates; and

a reference pattern adapter for;

(a) executing the adaptation of the reference patterns once again by reading out (i) the recognition result in the final or second recognizor, (ii) the input pattern, and (iii) the reference patterns, and

(b) storing the adapted reference patterns in the reference pattern memory to be ready for the next input.

15. A speech recognition system comprising:

an analyzer for converting a speech signal input into feature vector series;

an input pattern memory for storing the feature vector series as an input pattern;

a reference pattern memory for storing reference patterns;

a preliminary recognizer for executing a preliminary recognition based on (i) the input pattern and (ii) all reference patterns read out from the reference pattern memory to obtain top N candidates in an order of higher similarities;

an input pattern adapter for executing adaptation of the input pattern based on (i) the input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns;

a final recognizer for;

(a) receiving (i) the input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns corresponding to the top N candidates, and

(b) executing a matching process for the recognition; and

a reference pattern adapter for;

(a) executing the adaptation of the reference patterns based on (i) the recognition result in the final or second recognizor, (ii) the input pattern, and (iii) the reference patterns, and

(b) storing the adapted reference patterns in the reference pattern memory to be ready for the next input.

16. A speech recognition system comprising:

an analyzer for converting a speech signal input into feature vector series;

an input pattern memory for storing the feature vector series as an input pattern;

a reference pattern memory for storing reference patterns;

a preliminary recognizer for executing a preliminary recognition based on (i) the input pattern and (ii) all reference patterns read out from the reference pattern memory to obtain top N candidates in an order of higher similarities;

an input pattern adapter for executing adaptation of the input pattern based on (i) the input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns;

a reference pattern adapter for;

(a) executing adaptation of the reference patterns by reading out (i) the input pattern, (ii) the top N candidates as a result of preliminary recognitions and (iii) the reference patterns, and

(b) storing the adapted reference patterns in the reference pattern memory;

a final recognizer for;

(a) receiving (i) the adapted input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns corresponding to the top N candidates, and

(b) executing matching for the recognition; and

a reference pattern adapter for;

(a) executing the adaptation by reading out (i) the result of recognition in the final or second recognizer, (ii) the input pattern, and (iii) the reference patterns, and

(b) storing the adapted reference patterns in the reference pattern memory to be ready for the next input.

17. A speech recognition system comprising:

an analyzer for converting a speech signal input into feature vector series;

an input pattern memory for storing the feature vector series as an input pattern;

a reference pattern memory for storing reference patterns;

a preliminary recognizer for executing a preliminary recognition based on (i) the input pattern and (ii) all reference patterns read out from the reference pattern memory to obtain top N candidates in an order of higher similarities;

an input pattern adapter for executing an adaptation of the input pattern based on (i) the input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns;

a reference pattern adapter for;

(a) reading out (i) the input pattern, (ii) the top N candidates as a result of preliminary recognitions and (iii) the reference patterns, and

(b) executing adaptation of the reference patterns based on (i) the input pattern, (ii) the top N candidates, and (iii) the reference patterns, and

(c) storing the adapted reference patterns in the reference pattern memory;

a final recognizer for;

(a) receiving (i) the adapted input pattern, (ii) the top N candidates as a result of preliminary recognition, and (iii) the reference patterns corresponding to the top N candidates, and

(b) executing matching for the recognition;

a confirmer for;

(a) receiving the final or second recognition result and a confirmation result for the recognition result in the final or second recognizer,

(b) correcting the final or second recognition result if the final or second recognition result is wrong, and

(c) outputting the correct recognition result; and

a reference pattern adapter for;

(a) executing the adaptation of the reference patterns once again by reading out (i) the correct recognition result, (ii) the input pattern, and (iii) the reference patterns, and

(b) storing the once-again adapted reference patterns in the reference pattern memory to be ready for the next input.