Feature-based speech recognizer having probabilistic linguistic processor providing word matching based on the entire space of feature vectors

• US 5,822,729 A
• Filed: 06/05/1996
• Issued: 10/13/1998
• Est. Priority Date: 06/05/1996
• Status: Expired due to Term

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Glass, James Robert
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Smits, Talivaldis Ivars
• Application Number:
  US08/658,690
• Time in Patent Office: 860 Days
• Field of Search: 704/240, 704/243, 704/253, 704/254, 704/255, 704/257
• US Class Current: 704/255
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 2015/022 Demisyllables, biphones or triphones being the recognition units

Abstract

A feature-based speech recognizer having a probabilistic linguistic processor provides word matching based on the entire space of feature vectors. In this manner, the errors and inaccuracies associated with the heretofore known feature-based speech recognizers, which provided word matching on less than the entire space of feature vectors, are overcome, thereby resulting in improved-accuracy speech recognition. The word matching may be on feature vectors computed either from segments or from landmarks or from both segments and landmarks. For word matching on segment-based feature vectors, acoustic likelihoods may be normalized by extra-acoustic likelihoods defined by at least one extra-acoustic ("not" or "anti") model. Context-dependent and context-independent acoustic models may be employed.

Claims

1. A feature-based speech recognizer having a probabilistic linguistic processor providing word matching based on the entire space of segment-based feature vectors, comprising:

a segmenter responsive to acoustic evidence O in form of frames of speech-coded data representative of the speech to be recognized and operative (1) to parse said acoustic evidence into plural segments in such a way that each segment represents another way that said acoustic evidence may be partitioned framewise into segments, where all segments together define a segment space that accounts for all of the ways said frames of said acoustic evidence may be partitioned framewise, and (2) to combine said segments into plural segmentations S in such a way that each segmentation represents another way that said segments of said segment space may be combined segmentwise to account for all of said acoustic evidence;

a feature extractor, coupled to the segmenter and responsive to the frames of the acoustic evidence, and operative to extract, for each segment of a possible segmentation, a feature vector X having predetermined dimensions defined by linguistic units of an acoustic model that is representative of the presence of those linguistic units in the frames of the acoustic evidence underlying each such segment;

a classifier operable over the entire space of feature vectors, coupled to the feature extractor and to the segmenter, responsive to the extracted feature vectors, and to the segmentations S, and operative to classify the segment-based feature vectors X of every segmentation in terms of the different sequences of one or more predetermined linguistic units to provide a joint likelihood, P(XY.linevert split.SW), that is a measure of how well each feature vector X of every segmentation fits all the sequences of one or more predetermined linguistic units, and in such a way as to take into account, for every segmentation, the feature vectors Y of all of the other segments of the segment space belonging to other segmentations; and

a probabilistic word matcher, coupled to the classifier operable over the entire space of feature vectors, and operative to search the feature vectors classified over the entire space of segment-based feature vectors in terms of the ways the different sequences of one or more linguistic units fit the competing segmentations to find that word string that best matches the acoustic evidence.

2. The invention of claim 1, wherein the acoustic model includes in addition to the class, α

, of the linguistic units, at least one extra-linguistic class, α

, defined to map to all segments of the acoustic evidence which do not correspond to one of the linguistic units of the class α

.

3. The invention of claim 2, wherein acoustic-modelling is done at the phonetic level, and wherein probabilistic models are provided for individual phones of the class of linguistic units, α

, as well as for non-phones, of the extra-linguistic class, α

.

4. The invention of claim 2, wherein the classifier operable over the entire space of feature vectors of the linguistic processor of the instant invention (1) classifies the feature vectors of the segments of the entire segment space in terms of the non-linguistic units of the class of extra-linguistic units, α

, (2) computes from the feature vectors of the segments of some segmentation the acoustic likelihood of a linguistic unit of the linguistic class a for some sequence of one or more linguistic units of different possible sequences of linguistic units, (3) computes from the feature vectors of the same segments of some segmentation the likelihood of a non-linguistic unit over the class of extra-linguistic units, α

, (4) normalizes for each segment of some segmentation the acoustic likelihood by the likelihood of a non-linguistic unit, and (4) repeats the hereinabove steps (2) and (3) until the sequences of one or more linguistic units and competing segmentations have been gone through.

5. The invention of claim 1, further including an acoustic processor to provide said acoustic evidence O in response to human speech to be recognized.

6. The invention of claim 5, wherein the acoustic processor includes an analog-to-digital (A/D) converter responsive to human speech to sample the same, at the Nyquist rate, and to digitally encode the same providing sampled and digitally encoded data that is representative of the incoming speech signal, and a digital signal processor responsive to the sampled and digitally encoded data to provide said acoustic evidence O, consisting of a sequence of frames of coded speech data that occurs at a frame rate, that is representative of predetermined linguistic units present in the sampled and digitally encoded data.

7. The invention of claim 6, wherein the digital signal processor includes a spectral analyzer.

8. The invention of claim 6, wherein the digital signal processor includes a cepstral analyzer.

9. The invention of claim 7, wherein said spectral analyzer is a Mel Frequency Spectral Coefficients (MFSC) spectral analyzer.

10. The invention of claim 8, wherein said cepstral analyzer is a Mel Frequency Cepstral Coefficients (MFCC) cepstral analyzer.

11. The invention of claim 1, wherein said word matcher coupled to the classifier operable over the entire space of feature vectors, is responsive to the joint likelihoods, P(XY.linevert split.SW), and to the plural segmentations S, to compute, for every possible sequence of one or more linguistic units in its vocabulary, and for every segmentation S, (1) the joint likelihoods, P(XY.linevert split.SW), computed for each segmentation over the entire space of feature vectors and for the different sequences of one or more linguistic units, (2) the probability of every segmentation given a possible sequence of one or more lexical units, P(S.linevert split.W), and (3) the probability of the sequences of one or more lexical units, P(W), and outputs that string of linguistic units W* which maximizes the probability of having been spoken.

19. The invention of claim 1, wherein said word matcher coupled to the classifier operable over the entire space of feature vectors, is responsive to the likelihoods, P(Z.linevert split.SW), and to the plural segmentations S, to compute, for every possible sequence of one or more linguistic units in its vocabulary, and for every segmentation S, (1) the likelihoods, P(Z.linevert split.SW), computed for each segmentation over the entire space of feature vectors and for the different sequences of one or more linguistic units, (2) the probability of every segmentation given a possible sequence of one or more lexical units, P(S.linevert split.W), and (3) the probability of the sequences of one or more lexical units, P(W), and outputs that string of linguistic units W* which maximizes the probability of having been spoken.

12. A feature-based speech recognizer having a probabilistic linguistic processor providing word matching based on the entire space of landmark-based feature vectors, comprising:

a segmenter responsive to acoustic evidence O in form of frames of speech-coded data representative of the speech to be recognized and operative (1) to parse said acoustic evidence into plural segments in such a way that each segment represents another way that said acoustic evidence may be partitioned framewise into segments and landmarks, where all segments together define a segment space that accounts for all of the ways said frames of said acoustic evidence may be partitioned framewise, and (2) to combine said segments into plural segmentations S in such a way that each segmentation represents another way that said segments of said segment space may be combined segmentwise to account for all of said acoustic evidence;

a feature extractor, coupled to the segmenter and responsive to the frames of the acoustic evidence, and operative to extract, for each landmark of a possible segmentation, a feature vector Z having predetermined dimensions defined by linguistic units of an acoustic model that is representative of the presence of those linguistic units in the frames of the acoustic evidence underlying each such landmark;

a classifier operable over the entire space of feature vectors, coupled to the feature extractor and to the segmenter, responsive to the extracted feature vectors, and to the segmentations S, and operative to classify the landmark-based feature vectors Z of every segmentation in terms of the different sequences of one or more predetermined linguistic units to provide a likelihood, P(Z ISW), that is a measure of how well each feature vector Z of every landmark fits all the sequences of one or more predetermined linguistic units; and

a probabilistic word matcher, coupled to the classifier operable over the entire space of feature vectors, and operative to search the feature vectors classified over the entire space of landmark-based feature vectors in terms of the ways the different sequences of one or more linguistic units fit the competing segmentations to find that word string that best matches the acoustic evidence.

13. The invention of claim 12, further including an acoustic processor to provide said acoustic evidence A in response to human speech to be recognized.

14. The invention of claim 13, wherein the acoustic processor includes an analog-to-digital (A/D) converter responsive to human speech to sample the same, at the Nyquist rate, and to digitally encode the same providing sampled and digitally encoded data that is representative of the incoming speech signal, and a digital signal processor responsive to the sampled and digitally encoded data to provide said acoustic evidence O, consisting of a sequence of frames of coded speech data that occurs at a frame rate, that is representative of predetermined linguistic units present in the sampled and digitally encoded data.

15. The invention of claim 13, wherein the digital signal processor includes a spectral analyzer.

16. The invention of claim 13, wherein the digital signal processor includes a cepstral analyzer.

17. The invention of claim 15, wherein said spectral analyzer is a Mel Frequency Spectral Coefficients (MFSC) spectral analyzer.

18. The invention of claim 16, wherein said cepstral analyzer is a Mel Frequency Cepstral Coefficients (MFCC) cepstral analyzer.

20. A method for decoding speech from encoded speech data, comprising the steps of:

segmenting said encoded speech data into plural segments of a segment space and into plural segmentations each having at least one constitutive segments;

extracting for acoustic analysis from said data a network of feature vectors defined over said plural segmentations;

providing at least one model to be used during word matching; and

performing word matching for competing word hypotheses in each case over the entire network of feature vectors.

21. The method of claim 20, wherein said extracting step includes the step of extracting a network of segment-based feature vectors.

22. The method of claim 20, wherein said segmenting step further includes the step of defining landmarks associated with said plural segments and wherein said extracting step includes the step of extracting a network of landmark-based feature vectors.

23. The method of claim 20, wherein said segmenting step further includes the step of defining landmarks associated with said plural segments and wherein said extracting step includes the steps of extracting a network of landmark-based feature vectors Z and a network of segment-based feature vectors XY.

24. The method of claim 20, wherein said at least one model includes an extra-acoustic model representative of extra-linguistic units in said encoded speech data.

25. A method for decoding speech from encoded speech data, comprising the steps of:

segmenting said encoded speech data into plural segments of a segment space and into plural segmentations each having at least one constitutive segments;

providing at least one model to be used during word matching including at least one extra-linguistic model and including an acoustic model representative of predetermined linguistic units; and

performing word matching for competing word hypotheses using both said extra-linguistic and said acoustic models.