Apparatus and method for recognizing spoken words

• US 4,336,421 A
• Filed: 04/08/1980
• Issued: 06/22/1982
• Est. Priority Date: 04/08/1980
• Status: Expired due to Term

• Current Assignee: Siemens Corporate Research & Support, Inc. (Siemens AG)
• Original Assignee: Threshold Technology, Inc.
• Inventors: Welch, John R., Oxenberg, Sheldon C.
• Primary Examiner(s): Krass, Errol A.
• Assistant Examiner(s): KEMENY, EMANUEL
• Application Number:
  US06/138,643
• Time in Patent Office: 805 Days
• Field of Search: 179/1 SD, 179/1 SB, 179/1 VC, 340/146.3 WD, 340/146.3 SG, 340/148, 364/410, 364/419, 364/513
• US Class Current: 704/243
• CPC Class Codes: G10L 15/05 Word boundary detection

Abstract

A speech recognition technique is disclosed for recognizing words that are spoken at speeds that approach the speed of continuous speech.

To avoid the cost of a rigorous approach wherein all possible words of stored vocabulary are correlated against each group of speech samples, this invention uses relatively long duration inter-string pauses to detect string boundaries, and relatively short duration inter-segment pauses to determine speech segment boundaries which are used as a limited number of start or end points.

Claims

1. Apparatus for receiving input spoken vocabulary words during a training phase of operation and for subsequently recognizing strings of received input spoken command words, comprising:

feature extraction means for generating digital feature signals dependent on the features present in input words;

reference array formation means for forming and storing, for each vocabulary word, a time-dependent reference array dependent on the feature signals present during the vocabulary word as spoken during the training phase;

boundary detection means for detecting relatively long duration pauses between speech sounds as inter-string pauses from which string boundaries are determined, the string boundaries defining the beginning and end of a string of words, and for detecting relatively short duration pauses between speech sounds as inter-segment pauses from which speech segment boundaries are determined within a string of words;

candidate feature array formation means for forming and storing, for each command word candidate consisting of each segment and each sequence of up to a predetermined number of segments, a time-dependent candidate feature array dependent upon the feature signals present during said command word candidate;

array correlation means for comparing each candidate feature array with each reference array and for storing, for each command word candidate, the vocabulary word whose reference array results in the highest correlation and a record of the score of said highest correlation;

string matching means responsive to the stored correlation scores for determining the sequence of vocabulary words that yields the highest overall correlation score for the string; and

means for generating output indications of the determined sequence of vocabulary words.

2. Apparatus as defined by claim 1 wherein said reference array formation means and said candidate feature array formation means are operative to form time-normalized arrays.

3. Apparatus as defined by claim 1 wherein said candidate feature array formation means is operative to form a candidate feature array for each of the following:

a command word candidate consisting of each individual speech segment considered as a one segment word;

a command word candidate consisting of each individual speech segment together with the previous speech segment, if any, considered as a two segment word;

a command word candidate consisting of each individual speech segment together with the previous two speech segments, if any, considered as a three segment word; and

a command word candidate consisting of each individual speech segment together with the previous three speech segments, if any, considered as a four segment word.

4. Apparatus as defined by claim 2 wherein said candidate feature array formation means is operative to form a candidate feature array for each of the following:

a command word candidate consisting of each individual speech segment considered as a one segment word;

a command word candidate consisting of each individual speech segment together with the previous speech segment, if any, considered as a two segment word;

a command word candidate consisting of each individual speech segment together with the previous two speech segments, if any, considered as a three segment word; and

a command word candidate consisting of each individual speech segment together with the previous three speech segments, if any, considered as a four segment word.

5. Apparatus as defined by claim 1 wherein said string matching means is operative to identify, at each segment of a string, the sequence of vocabulary words which provides the highest overall correlation up to that particular segment of the string when considering said particular segment and all previous segments of the string as being part of sequences of different word lengths.

6. Apparatus as defined by claim 2 wherein said string matching means is operative to identify, at each segment of a string, the sequence of vocabulary words which provides the highest overall correlation up to that particular segment of the string when considering said particular segment and all previous segments of the string as being part of sequences of different word lengths.

7. Apparatus as defined by claim 4 wherein said string matching means is operative to identify, at each segment of a string, the sequence of vocabulary words which provides the highest overall correlation up to that particular segment of the string when considering said particular segment and all previous segments of the string as being part of sequences of different word lengths.

8. Apparatus as defined by claim 5 wherein said string matching means is also operative to store the highest overall correlation scores up to each segment, said scores being used in determining the highest overall correlation scores to subsequent segments.

9. Apparatus as defined by claim 6 wherein said string matching means is also operative to store the highest overall correlation scores up to each segment, said scores being used in determining the highest overall correlation scores to subsequent segments.

10. Apparatus as defined by claim 7 wherein said string matching means is also operative to store the highest overall correlation scores up to each segment, said scores being used in determining the highest overall correlation scores to subsequent segments.

11. Apparatus for receiving input spoken vocabulary words during a training phase of operation and for subsequently recognizing strings of received input spoken command words, comprising:

feature extraction means for generating digital feature signals dependent on the features present in input words;

reference array formation means for forming and storing, for each vocabulary word, a time-dependent reference array dependent on the feature signals present during the vocabulary word spoken during the training phase;

a feature buffer memory for storing feature signals which occur over a period of time;

candidate feature array formation means, responsive to the stored feature signals, for forming, for each command word candidate consisting of a portion of the speech of a received string, a time dependent command word candidate feature array dependent on the feature signals present during said command word candidate;

a first-in-first-out feature array memory stack for storing said command word candidate feature arrays;

array correlation means responsive to feature arrays in said feature array memory stack for comparing, in order, each candidate feature array with each reference array to obtain the vocabulary word whose reference array results in the highest correlation;

a correlation result memory for storing a record of the vocabulary words which result in the highest correlation with each candidate feature array, and for storing a record of the score of said highest correlation;

string matching means responsive to the correlation scores and vocabulary words stored in said correlation result memory for determining the sequence of vocabulary words that yields the highest correlation score for the string; and

means responsive to said string matching means for generating output indications of the determined sequence of vocabulary words;

said array formation means, array correlation means, and string matching means being operative to function one at a time on a priority basis, with said array formation means having a higher operational priority than said array correlation means, and said array correlation means having a higher operational priority than said string matching means.

12. Apparatus as defined by claim 11 further comprising feature sampling means for periodically sampling said digital feature signals and storing said feature signals in said feature buffer memory, said sampling being implemented on an interrupt basis that has highest operational priority.

13. Apparatus as defined by claim 11 further comprising:

boundary detection means for detecting boundaries defining the beginning and end of a string of input words and for detecting boundaries between speech segments within a string; and

a first-in-first-out boundary stack memory for storing the detected boundaries;

said array formation means being operative to define command word candidates in terms of the boundaries stored in said boundary stack memory.

14. Apparatus as defined by claim 12 further comprising:

boundary detection means for detecting boundaries defining the beginning and end of a string of input words and for detecting boundaries between speech segments within a string; and

a first-in-first-out boundary stack memory for storing the detected boundaries;

said array formation means being operative to define command word candidates in terms of the boundaries stored in said boundary stack memory.

15. Apparatus as defined by claim 12 wherein said boundary stack memory is operative to store said boundaries as the addresses of feature signal samples stored in said feature buffer memory.

16. Apparatus as defined by claim 14 wherein said boundary stack memory is operative to store said boundaries as the addresses of feature signal samples stored in said feature buffer memory.

17. Apparatus as defined by claim 15 wherein said reference array formation means and said candidate feature array formation means are operative to form time-normalized arrays.

18. Apparatus as defined by claim 16 wherein said reference array formation means and said candidate feature array formation means are operative to form time-normalized arrays.

19. Apparatus as defined by claim 17 wherein said candidate feature array formation means is operative to form a candidate feature array for each of the following:

a command word candidate consisting of each individual speech segment considered as a one segment word;

a command word candidate consisting of each individual speech segment together with the previous speech segment, if any, considered as a two segment word;

a command word candidate consisting of each individual speech segment together with the previous two speech segments, if any, considered as a three segment word; and

a command word candidate consisting of each individual speech segment together with the previous three speech segments, if any, considered as a four segment word.

20. Apparatus as defined by claim 18 wherein said candidate feature array formation means is operative to form a candidate feature array for each of the following:

a command word candidate consisting of each individual speech segment considered as a one segment word;

a command word candidate consisting of each individual speech segment together with the previous speech segment, if any, considered as a two segment word;

a command word candidate consisting of each individual speech segment together with the previous two speech segments, if any, considered as a three segment word; and

a command word candidate consisting of each individual speech segment together with the previous three speech segments, if any, considered as a four segment word.

21. Apparatus as defined by claim 19 wherein said string matching means is operative to identify, at each segment of a string, the sequence of vocabulary words which provides the highest overall correlation up to that particular segment of the string when considering said particular segment and all previous segments of the string as being part of sequences of different word lengths.

22. Apparatus as defined by claim 20 wherein said string matching means is operative to identify, at each segment of a string, the sequence of vocabulary words which provides the highest overall correlation up to that particular segment of the string when considering said particular segment and all previous segments of the string as being part of sequences of different word lengths.

23. Apparatus as defined by claim 21 wherein said string matching means is also operative to store the highest overall correlation scores up to each segment, said scores being used in determining the highest overall correlation scores to subsequent segments.

24. Apparatus as defined by claim 22 wherein said string matching means is also operative to store the highest overall correlation scores up to each segment, said scores being used in determining the highest overall correlation scores to subsequent segments.

25. Apparatus as defined by claim 13 wherein said boundary detection means is operative to detect relatively long duration pauses between speech sounds as inter-string pauses from which the string boundaries are determined, and is operative to detect relatively short duration pauses between speech sounds as inter-segment pauses from which the speech segment boundaries are determined.

26. Apparatus as defined by claim 14 wherein said boundary detection means is operative to detect relatively long duration pauses between speech sounds as inter-string pauses from which the string boundaries are determined, and is operative to detect relatively short duration pauses between speech sounds as inter-segment pauses from which the speech segment boundaries are determined.

27. Apparatus as defined by claim 23 wherein said boundary detection means is operative to detect relatively long duration pauses between speech sounds as inter-string pauses from which the string boundaries are determined, and is operative to detect relatively short duration pauses between speech sounds as inter-segment pauses from which the speech segment boundaries are determined.

28. Apparatus as defined by claim 24 wherein said boundary detection means is operative to detect relatively long duration pauses between speech sounds as inter-string pauses from which the string boundaries are determined, and is operative to detect relatively short duration pauses between speech sounds as inter-segment pauses from which the speech segment boundaries are determined.

29. A method for receiving input spoken vocabulary words during a training phase of operation and for subsequently recognizing strings of received input spoken command words, comprising the steps of:

generating digital feature signals dependent on the features present in input words;

forming and storing, for each vocabulary word, a time-dependent reference array dependent on the feature signals present during the vocabulary word as spoken during the training phase;

detecting relatively long duration pauses between speech sounds as inter-string pauses from which string boundaries are determined, the string boundaries defining the beginning and end of a string of words, and detecting relatively short duration pauses between speech sounds as inter-segment pauses from which speech segment boundaries are determined within a string of words;

forming and storing, for each command word candidate consisting of each segment and each sequence of up to a predetermined number of segments, a time-dependent candidate feature array dependent upon the feature signals present during said command word candidate;

comparing each candidate feature array with each reference array and storing, for each command word candidate, the vocabulary word whose reference array results in the highest correlation and a record of the score of said highest correlation;

determining, using the stored correlation scores and words, the sequence of vocabulary words that yields the highest overall correlation score for the string; and

generating output indications of the determined sequence of vocabulary words.

30. The method as defined by claim 29 wherein said reference array formation and said candidate feature array formation comprise forming time-normalized arrays.

31. The method as defined by claim 29 wherein said candidate feature array formation includes forming a candidate feature array for each of the following:

a command word candidate consisting of each individual speech segment considered as a one segment word;

a command word candidate consisting of each individual speech segment together with the previous speech segment, if any, considered as a two segment word;

a command word candidate consisting of each individual speech segment together with the previous two speech segments, if any, considered as a three segment word; and

a command word candidate consisting of each individual speech segment together with the previous three speech segments, if any, considered as a four segment word.

32. The method as defined by claim 30 wherein said candidate feature array formation includes forming a candidate feature array for each of the following:

a command word candidate consisting of each individual speech segment considered as a one segment word;

a command word candidate consisting of each individual speech segment together with the previous speech segment, if any, considered as a two segment word;

a command word candidate consisting of each individual speech segment together with the previous two speech segments, if any, considered as a three segment word; and

a command word candidate consisting of each individual speech segment together with the previous three speech segments, if any, considered as a four segment word.

33. The method as defined by claim 29 wherein said string matching includes identifying, at each segment of a string, the sequence of vocabulary words which provides the highest overall correlation up to that particular segment of the string when considering said particular segment and all previous segments of the string as being part of sequences of different word lengths.

34. The method as defined by claim 30 wherein said string matching includes identifying, at each segment of a string, the sequence of vocabulary words which provides the highest overall correlation up to that particular segment of the string when considering said particular segment and all previous segments of the string as being part of sequences of different word lengths.

35. The method as defined by claim 32 wherein said string matching includes identifying, at each segment of a string, the sequence of vocabulary words which provides the highest overall correlation up to that particular segment of the string when considering said particular segment and all previous segments of the string as being part of sequences of different word lengths.

36. The method as defined by claim 33 wherein said string matching also includes storing the highest overall correlation scores up to each segment, said scores being used in determining the highest overall correlation scores to subsequent segments.

37. The method as defined by claim 34 wherein said string matching also includes storing the highest overall correlation scores up to each segment, said scores being used in determining the highest overall correlation scores to subsequent segments.

38. The method as defined by claim 35 wherein said string matching also includes storing the highest overall correlation scores up to each segment, said scores being used in determining the highest overall correlation scores to subsequent segments.