Process for the recognition of a continuous flow of spoken words

• US 4,947,438 A
• Filed: 07/11/1988
• Issued: 08/07/1990
• Est. Priority Date: 07/11/1987
• Status: Expired due to Fees

• Current Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Original Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Inventors: Paeseler, Annedore
• Primary Examiner(s): Harkcom, Gary V.
• Assistant Examiner(s): Knepper, David D.
• Application Number:
  US07/217,535
• Time in Patent Office: 757 Days
• Field of Search: 381/41-43, 364/513, 364/513.5
• US Class Current: 704/252
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks

Abstract

Continuous speech recognition assigns predetermined words to syntactic categories and defines the syntactic categories which can follow and precede each predetermined word. The recognition process is achieved by comparing the input sequence of speech signals to reference values and summing those which are syntactically permissible until they form a valid word. Subsequent speech values to previouly calculated valid words are compared to reference values listed in syntactic categories which can follow the predetermined word. For each word, values are updated indicating the current word'"'"'s sequence number, syntax category, cumulative comparison sum, and the current list of compared words. Values are also stored for each word which identify the previous word, the following word and their syntax categories. This process is repeated until all input values have been processed. The results are then checked to verify valid syntax and the words with the closest match are read out.

Claims

1. A process for the recognition of speech signal derived from a continuous flow of spoken words, which speech signal comprises a temporal sequence of speech values, each of which values specifies a section of the speech signal;

comprising;

comparing the speech values with predetermined stored reference values, a group of which reference values represents one word of a predetermined vocabulary for forming an initial evaluation value;

summing the comparison results over various sequences of combinations of reference values and speech values per sequence whose order is permissible in accordance with a predetermined stored first list containing, for predetermined syntactic categories, at least one assignment per category to a combination of further syntactic categories and/or words for forming a cumulative evaluation value;

generating a second list and a third list the second list including references to the reference values of all those words which are compared with the respective next speech value as well as a sequence number per word, and the third list including, for each speech value which has been compared with the last reference value of at least one word, a plurality of entries, each entry including a current sequence number and;

(a) a reference to a syntactic category of the first list,

(b) a first specification for a sequence of compared words and/or syntactic categories which are assigned to sequences of already compared speech values,

(c) a second specification for a sequence of words and/or syntactic categories which can be assigned to subsequent speech values on the basis of the first list,

(d) a further sequence number assigned to the respective entry,

(e) a first cumulative evaluation value,

(f) a second initial evaluation value and

(g) a sequence of compared words;

determining a new sequence number at least after every comparison of a new speech value with the last reference value of at least one word, and after each such comparison, searching through the group of entries of the third list associated with the sequence number stored in the second list at this word for such entries in which the sequence contained in the second specification begins with the compared word, and deriving a new entry for each such entry present, for the new group of the third list associated with the new sequence number;

making a first further entry in the new group for each new entry in which the abbreviated sequence contained in the second specification begins with a syntactic category, for which at least one assignment is present in the first list, and, deriving a second further entry for the new group for each of the new and first further entries of the new group for which the second specification contains an empty sequence;

repeating the steps of deriving and making the first and second further entries alternately until, after at least one first further entry, no second further entry occurs;

entering a reference to the reference data of the first word of each entry of the new group in which the second sequence begins with a word to be recognized;

comparing the next speech value with the reference values of all words contained in the second list;

repeating the process steps until the last speech value of the speech signal to be recognized has been processed;

checking the last group of the third list for all entries containing;

a reference to the syntactic initial category, an empty sequence, and a sequence number; and

reading out the sequence of compared words from those entries having the smallest first evaluation value.

2. A process according to claim 1, wherein:

before the comparison of the first speech value, the first group contains;

a plurality of first entries each entry including;

a reference to a syntactic initial category, an empty sequence as a first specification, another of the combinations assigned to the initial category as a second specification, and an initial value for the two evaluation values; and

further entries including all the catagories which can be derived from each of the combinations of the second specification with a corresponding combination; and

wherein each new entry in the first specification contains a sequence extended to include the compared word, a sequence abbreviated to exclude the compared word in the second specification, the evaluation value incremented by the sum of the comparison results of the word as a first evaluation value, the sequence extended to include the compared word as a sequence of compared words, and the values of the entry present;

wherein each of the first further entries includes a reference to the syntactic category of the new entry, from which this first further entry is derived, an empty sequence in the first specification, another combination assigned to the syntactic category as a sequence in the second specification, the new sequence number as a further sequence number, the evaluation value of the new entry for both evaluation values, and an empty sequence as a sequence of compared words;

reading out the earlier entry for each second further entry, from that group specified in the further sequence number of the relevant new or further first entry, where the sequence associated with the second specification begins with the new syntactic category to which this new or first further entry of the new group contains a reference, wherein the second entry contains;

the reference to the syntactic category of the earlier entry, a sequence extended to include the syntactic category of the current new or first further entry in the first specification, a sequence abbreviated to exclude the same syntactic category in the second specification, the sequence number of the earlier entry as a further sequence number, the sum of the first evaluation value of the earlier entry and the difference between the two evaluation values of the current entry as a first evaluation value, the corresponding evaluation value of the earlier entry and the sequence of compared words of the earlier entry extended to include the sequence of compared words of the entry of the current group as a second evaluation value; and

entering into the second list each reference to the reference data, the associated first evaluation value, and the sequence number of the relevant entry.

3. A process according to claim 2 comprising making each first and second further entry only if no entry is present in the new group, which entry contains the same reference, the same first and second specification and the same further sequence number and in which the first evaluation value is smaller than the first evaluation value of the intended further entry;

and if such an entry is already present but with a greater evaluation value, deleting such entry.

4. A process according to claim 2 comprising entering a reference to reference data into the second list only if no reference to the reference data of the same word and the same sequence number with a smaller evaluation value is already present in said list;

and if such entry is already present but with a greater evaluation value, deleting such entry.

5. A process according to claim 2, comprising making each new and each first and second further entry only if its first evaluation value is smaller than a threshold value which is equal to the smallest first evaluation value, extended by a constant, of all entries currently contained in the second list.

6. A process according to claim 5, comprising deleting every entry in the second list whose first evaluation value is greater than the threshold value.

7. A process according to claim 6, comprising determining from the evaluation value of the preceding speech value, the threshold value as a smallest first evaluation value of the entries of the second list.

8. A process according to claim 2, comprising deleting a word from the second list if the number of sequence numbers lying between the new sequence number and the sequence number stored at the word is greater than a limit value contained in the reference values for this word.

9. A process according to claim 1, comprising assigning predetermined syntactic categories to other syntactic categories and/or word classes in said first list;

and entering a reference to a word class in said second list for all entries of the new group where the second sequence begins with a word class.

10. A process according to claim 9, comprising calling up an auxiliary list with each reference in the second list, an auxiliary list is called up containing for each word class the reference to the reference data of the words belonging to this class, and these references call up the corresponding reference values from the further list.

11. Apparatus for carrying out the process according to claim 1, comprising input means for receiving a spoken sentence in the form of an electrical speech signal;

conversion means connected to said input means for forming speech values;

a first memory containing specifications on syntactic categories of natural language and their assignment to further syntactic categories and/or specifications for words or word classes;

a further memory for reference values formed analogously to the speech values from sentences spoken earlier;

comparison means connected to an output of the conversion means and to a data output of the further memory for supplying comparison results from the comparison of speech values with reference values;

a second memory for storing the entries for the second list specifying at least a part of the address of the further memory; and

a third memory for storing the entries for the third list;

controller means for addressing the first, the second and the third memory and for recording data in the second and the third memory and reading out of the first, second and third memory and on receiving a word end signal for at least one word, forming the new first and second further entries for the third memory and subsequently the entries for the second memory and recording in these; and

, output means for outputting after processing the last speech signal the complete word string contained in the third memory with the smallest evaluation thereof.

12. Apparatus according to claim 11, wherein the controller comprises a programmed microprocessor.

13. Apparatus according to claim 11, comprising an auxiliary memory having an address input coupled to an output of the second memory and an output coupled to a partial address input of the further memory.