System for processing a succession of utterances spoken in continuous or discrete form

• US 5,526,463 A
• Filed: 04/09/1993
• Issued: 06/11/1996
• Est. Priority Date: 06/22/1990
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Dragon Systems, Inc. (Microsoft Corporation)
• Inventors: Roth, Robert S., Gillick, Laurence S.
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): Doerrler, Michelle
• Application Number:
  US08/045,991
• Time in Patent Office: 1,159 Days
• Field of Search: 381/41-43, 395/2.4, 395/2.5, 395/2.56, 395/2.61, 395/2.6, 395/2.59
• US Class Current: 704/251
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates

Abstract

The system of the invention relates to continuous speech pre-filtering systems for use in discrete and continuous speech recognition computer systems. The speech to be recognized is converted from utterances to frame data sets, which frame data sets are smoothed to generate a smooth frame model over a predetermined number of frames. A resident vocabulary is stored within the computer as clusters of word models which are acoustically similar over a succession of frame periods. A cluster score is generated by the system, which score includes the likelihood of the smooth frames evaluated using a probability model for the cluster against which the smooth frame model is being compared. Cluster sets having cluster scores below a predetermined acoustic threshold are removed from further consideration. The remaining cluster sets are unpacked for determination of a word score for each unpacked word. These word scores are used to identify those words which are above a second predetermined threshold to define a word list which is sent to a recognizer for a more lengthy word match. Control means enable the system to initialize times corresponding to the frame start time for each frame data set, defining a sliding window.

Claims

1. A system for processing speech, said speech including a succession of utterances spoken in any of continuous and discrete form, comprising:

A. means for storing a plurality of word models;

B. means for identifying a succession of temporal segments of said utterances spoken in any of continuous and discrete form;

C. means selectively operable on ones of said segments for identifying a subset of said plurality of word models meeting predetermined criteria, said subset defining a list of candidate words;

D. control means for determining arbitrarily selected frame start times t during said utterances spoken in any of continuous and discrete form, said frame start times being independent of identification of an initial anchor; and

E. means for generating a signal representative of said list of candidate words for selected ones of said frame start times t determined in step D.

2. A prefiltering system for processing speech, said speech including a succession of utterances spoken in any of continuous and discrete form, comprising:

A. cluster data storage means for storing a plurality of M cluster data sets, C₁, . . . , C_M, where M is an integer greater than 1, each of said cluster data sets including data representative of a plurality of word models;

B. frame data means for generating a succession of w frame data sets v_t, v_t+1, . . . v_t+w-1, beginning at a frame start time t during said succession of utterances spoken in any of continuous and discrete form, where w is an integer greater than 1, said succession of frame data sets being representative of a corresponding succession of temporal segments of said utterances spoken in any of continuous and discrete form, each of said frame data sets including k values representative of different frame parameters, where k≧

1;

C. data reduction means selectively operable on said w frame data sets for generating s reduced frame data sets Y₁, Y₂, . . . , Y₃, where s<

w, each of said reduced frame data sets being related to an associated plurality of said frame data sets and including j values representative of different reduced frame data set parameters;

D. scoring means for evaluating each of said reduced frame data sets against succession of said cluster data sets to generate a cluster score S_Y for each of said cluster data sets;

E. selectively operable identifying means for identifying each of said word models of said cluster data sets having a cluster score bearing a predetermined relation to at least one threshold score T, said identified word models defining a candidate word list;

F. control means for determining said frame start times t, where successive start times t are spaced apart arbitrarily, said frame start times being independent of identification of an initial anchor; and

G. means for generating a signal representative of said candidate word list for preselected ones of said frame start times t determined by said control means.

3. A system according to claim 2 wherein said cluster data storage means and said frame data means are adapted whereby each of said frame data sets are associated with duration D₁, and wherein said cluster data sets are each associated with duration D₂, such that:

space="preserve" listing-type="equation">D.sub.1 ≦

D.sub.2.

4. A system according to claim 2 wherein said cluster data storage means is adapted whereby said cluster data sets are wordstart cluster data sets.

5. A system according to claim 2 wherein said cluster data storage means is adapted whereby said word models of each of said cluster data sets correspond to acoustically similar utterances spoken in any of continuous and discrete form over a succession of no more than w frame data sets.

6. A system according to claim 2 wherein said data reduction means includes smooth frame means for processing said frame data sets whereby said reduced frame data sets are smoothed frame data sets.

7. A system according to claim 6 wherein said smooth frame means is adapted for determining smooth frame data sets in accordance with:

##EQU4## wherein Y₁, Y₂ . . . Y_s are said smooth frame data sets, and whereby each of said smooth frame data sets is associated with b of said frame data sets, and wherein said b are integers greater than 1, a_i are predetermined weighting coefficients, and c defines an offset of each smooth frame data set with respect to the next previous smooth frame data set.

8. A system according to claim 7 wherein said data reduction means is adapted whereby w=b+(s-1)c.

9. A system according to claim 8 wherein said data reduction means is adapted whereby w=12.

10. A system according to claim 8 wherein said data reduction means is adapted whereby k=8.

11. A system according to claim 10 wherein said data reduction means is adapted whereby s=3.

12. A system according to claim 11 wherein said data reduction means is adapted whereby b=4.

13. A system according to claim 12 wherein said data reduction means is adapted whereby c=4.

14. A system according to claim 8 wherein w=12, k=8, s=3, b=4, c=4, and a_i =1/b.

15. A system according to claim 7 wherein said data reduction means is adapted whereby said weighting coefficients a_i =1/b.

16. A system according to claim 2 wherein said cluster data storage means is adapted whereby each of said word models includes r node data vectors f₁, . . . , f_r, where r≦

s, each of said node data vectors being representative of a characteristic related to the occurrence of a selected one of acoustic segments from each word of a set of words associated with said cluster data sets in said acoustic segment of said utterances spoken in any of continuous and discrete form.

17. A system according to claim 16 wherein said cluster data storage means is adapted whereby said characteristic related to the occurrence of a selected one of acoustic segments comprises a probability distribution.

18. A system according to claim 17 wherein said cluster data storage means is adapted whereby said cluster score S_Y corresponds to:

##EQU5## wherein Y_i are ones of Y₁, Y₂, . . . , Y_s, and r are said node data vectors f₁, . . . , f_r for corresponding ones of said word models.

19. A system according to claim 2 wherein said data reduction means is adapted whereby j=k.

20. A system according to claim 2 wherein said identifying means includes a first identifying means for identifying each of said cluster data sets having a cluster score measured with respect to a first predetermined threshold T₁, said identified cluster data sets defining preliminary cluster data sets.

21. A system according to claim 20 further comprising means for generating a word score representative of the sum of said cluster score S_y of said cluster data set associated with each of said word models and a language model score S_L and wherein said identifying means further comprises a selectively operable second identifying means for identifying each of said word models of each of said cluster data sets having a word score S_W measured with respect to a second threshold T₂, said sum represented by:

space="preserve" listing-type="equation">S.sub.W =S.sub.Y +S.sub.L.

22. A speech processing method for processing speech including a succession of utterances spoken in any of continuous and discrete form comprising the steps of:

A. storing a plurality of M cluster data sets, C₁, . . . , C_M, where M is an integer greater than 1, each of said cluster data sets including data representative of a plurality of word models;

B. generating a succession of w frame data sets v_t, v_t+1, . . . v_t+w-1, beginning at a frame start time t during said succession of utterances spoken in any of continuous and discrete form, where w is an integer greater than 1, each of said frame data sets being representative of successive acoustic segments of utterances spoken in any of continuous and discrete form for a frame period, each of said frame data sets including k values representative of different frame parameters where k≧

1;

C. reducing w of said frame data sets to generate s reduced frame data sets, Y₁, Y₂, . . . , Y₃ where s<

w, each of said reduced frame data sets being related to an associated plurality of said frame data sets and including j values related to the k values of said associated frame data sets, where j≦

k;

D. evaluating said reduced frame data sets with a succession of said cluster data sets to generate a cluster score S_Y for each of said cluster data sets;

E. identifying each of said word models having a cluster score bearing a predetermined relation to at least one threshold score T, said identified word models defining a word list;

F. determining said frame start times t, where successive start times t are identified at arbitrarily selected intervals;

said frame start times being independent of identification of an initial anchor; and

G. generating a signal representative of said candidate word list for selected ones of said frame start times determined in step F.

23. A method according to claim 22 wherein said reducing step C further Comprises the substep of smoothing said frame data sets.

24. A method according to claim 23 wherein said smoothing substep includes smoothing in accordance with:

##EQU6## wherein Y₁, Y₂, . . . Y_s are said smooth frame data sets, and wherein said b are integers greater than 1, a_i are predetermined weighting coefficients, and c defines an offset of each smooth frame data set with respect to the next previous smooth frame data set.

25. A method according to claim 24 wherein said smoothing substep includes setting w=12, k=8, s=3, b=4, c=4, and a_i =1/b.

26. A method according to claim 22 wherein said storing step includes the substep of generating for each of said word models, r node data vectors f₁, . . . , f_r, where r is less than or equal to s, each of said node data vectors being representative of a characteristic related to the occurrence of a selected one of acoustic segments from each word of a set of words associated with said cluster data sets in said acoustic segment of said utterances spoken in any of continuous and discrete form, and

wherein said evaluating step includes the substep of determining said cluster score S_Y in accordance with:

##EQU7## wherein Y_i are ones of Y₁, Y₂, . . . , Y_s, and f_i are said node data vectors f₁, . . . , f_r for corresponding ones of said word models.

27. A prefiltering method for processing speech, said speech including a succession of utterances spoken in any of continuous and discrete form, comprising the steps of:

A. storing a plurality of word models;

B. identifying a succession of temporal segments of said utterances spoken in any of continuous and discrete form;

C. operating on ones of said segments and selectively identifying a subset of said plurality of word models meeting predetermined criteria, said subset defining a list of candidate words;

D. determining arbitrarily selected frame start times t during said utterances spoken in any of continuous and discrete form, said frame start times being independent of identification of an initial anchor; and

E. generating a signal representative of said list of candidate words for selected ones of said frame start times t determined in step D.