Apparatus and method for determining articulatory-orperation speech parameters

• US 5,913,188 A
• Filed: 09/11/1995
• Issued: 06/15/1999
• Est. Priority Date: 09/26/1994
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tzirkel-Hancock, Eli
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US08/526,386
• Time in Patent Office: 1,373 Days
• Field of Search: 395/2.1, 395/2.2, 395/2.22, 395/2.23, 395/2.29, 395/2.3, 395/2.33, 395/2.34, 395/2.35, 395/2.4, 395/2.43, 395/2.47, 395/2.64, 395/2.65
• US Class Current: 704/223
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 25/09 the extracted parameters being zero crossing rates
• CPC Class Codes: G10L 25/15 the extracted parameters being formant information
• CPC Class Codes: G10L 25/93 Discriminating between voiced and unvoiced parts of speech signals G10L25/90 takes precedence

Abstract

In an apparatus for extracting information from an input speech signal, a preprocessor, a buffer, a segmenter, an acoustic classifier and a feature extractor are provided. The preprocessor generates formant related information for consecutive time frames of the input speech signal. This formant related information is fed into the buffer, which can store signals representative of a plurality of frames. The segmenter monitors the signals representative of the incoming frames and identifies segments in the input speech signal during which variations in the formant related information remain within prespecified limits. The acoustic classifier then determines classification information for each segment identified by the segmenter, based on acoustic classes found in training data. The feature estimator then determines, for each segment, the information required, based on the input speech signal during that segment, training data and the classification information determined by the acoustic classifier.

Claims

1. An apparatus for determining, from an input speech signal, parameters defining the articulatory operation of a speech production system that generated the input speech signal, the apparatus comprising:

dividing means operable for dividing the input speech signal into a succession of frames;

a feature analyzer operable for determining acoustic features of the input speech signal during each frame;

a segmenter operable for defining a succession of segments as a number of frames by (i) comparing signals representative of the acoustic features for a current frame with signals representative of the acoustic features of previous frames in a current segment;

(ii) including he current frame in the current segment if said signals differ by less than a threshold value; and

(iii) beginning a new segment with the current frame if said signals differ by more than said threshold value; and

determining means operable for determining, for each segment, said articulatory speech production parameters using the acoustic features of the input speech signal during the segment and using stored reference data which relates said acoustic features to said articulatory speech production parameters.

2. An apparatus according to claim 1, wherein said difference is determined using a weighted Euclidean distance measure.

3. An apparatus according to claim 1, wherein said segmenter comprises smoothing means arranged to generate smoothed signals representative of the acoustic features of each frame, by smoothing variations between the acoustic features of adjacent frames and wherein said segmenter is operable to perform said comparison using smoothed signals representative of the current frame and smoothed signals representative of the previous frames in the current segment.

4. An apparatus according to claim 1, wherein said segmenter is arranged to limit the number of frames allowed within each segment.

5. An apparatus according to claim 1, wherein said feature analyse is arranged to determine formant related information of the input speech signal during each frame.

6. An apparatus according to claim 1, wherein said reference data comprises information that groups a selection of segments of input speech into different acoustic classes representative of acoustic characteristics of speech signals, and wherein said apparatus comprises a classifier for determining classification information for each segment defined by said segmenter and wherein said determining means is arranged to determine said parameters using said classification information.

7. An apparatus according to claim 6, wherein said classifier is arranged to determine, for each class, the probability that the current segment belongs to that class and wherein said classification information comprises each said determined probability.

8. An apparatus according to claim 6, wherein said classification information comprises the most likely class to which the current segment belongs.

9. An apparatus according to claim 6, wherein said information that groups said selection of segments into different classes comprises a plurality of mathematical models, one for each acoustic class, each model modelling the distribution of the acoustic features of the input speech during the selected segments in the class.

10. An apparatus according to claim 9, wherein each said mathematical model is an average.

11. An apparatus according to claim 9 or 10, wherein said classifier is arranged to determine said classification information by comparing signals representative of the current segment with each said mathematical model.

12. An apparatus according to claim 11, wherein said signals representative of the current segment are the average of the acoustic features of the input speech signal during the current segment.

13. An apparatus according to claim 6, wherein each of said parameters has associated therewith a number of discrete parameter values and wherein said reference data comprises a mathematical model for each parameter value in each acoustic class, relating that parameter value to said acoustic features.

14. An apparatus according to claim 13, wherein said mathematical model for each parameter value in each acoustic class is a Gaussian density function.

15. An apparatus according to claim 13, wherein said reference data further comprises the probability that a specific parameter defining the operation of the speech production system has one of said associated discrete parameter values given that a segment is in a specific acoustic class, for each parameter and parameter value in each acoustic class.

16. An apparatus according to claim 15, wherein said reference data further comprises the probability that a specific parameter defining the operation of the speech production system has one of said associated discrete parameter values, for each parameter and each parameter value.

17. An apparatus according to claim 13, wherein said mathematical model for each parameter value in each acoustic class is arranged to estimate the probability of receiving an input speech signal corresponding to the current frame, given that the current frame is in a segment in the acoustic class for that model, and assuming that the parameter has the parameter value for that model.

18. An apparatus according to claim 1, wherein said determining means is arranged to determine said parameters for each frame of input speech.

19. An apparatus according to claim 17, wherein said determining means is arranged to determine the probability of receiving an input speech signal corresponding to the current frame assuming that a specific parameter defining the operation of the speech production system has one of said associated parameter values, for each parameter and parameter value in dependence upon said reference data.

20. An apparatus according to claim 1, wherein said determining means is arranged to determine said parameters once per segment, in dependence upon signals representative of the segment.

21. An apparatus according to claim 6, wherein said classifier is arranged to determine said classification information in dependence upon previously determined classification information.

22. An apparatus according to claim 1, wherein said segmenter is arranged to compare the signals representative of the acoustic features of previous frames in the current segment with the signals representative of the acoustic features for the current frame as soon as said feature analyser has determined the acoustic features for the current frame.

23. An apparatus according to claim 1, wherein said determining means is arranged to determine, for the current segment, said parameters as soon as the last frame in the segment has been divided from the input speech signal.

24. An apparatus according to claim 1, further comprising a buffer for storing acoustic features corresponding to a plurality of frames, and wherein said segmenter is arranged to define a plurality of segments within the stored acoustic features corresponding to said plurality of frames, after said acoustic features have been determined by said feature analyser.

25. An apparatus according to claim 24, wherein said determining means is arranged to determine, for each segment defined by said segmenter, said parameters as soon as each said segment is defined by said segmenter.

26. An apparatus according to claim 19, wherein said determining means is arranged to decide the value of each parameter, based on the probabilities determined for that parameter.

27. An apparatus according to claim 26, wherein said determining means is arranged to decide the value of each parameter in dependence upon past decisions about the parameter values.

34. An apparatus according to claim 1, wherein said reference data representative of training speech signals is determined by a method in accordance with any one of claims 29 to 33.

62. An apparatus according to claim 1, wherein said threshold value is determined using a method of determining a threshold value for use in segmenting an input speech signal, the method comprising the steps of:

providing a set of training speech signals having known phonetic boundaries and/or known boundaries between segments in the training speech signals, the acoustic features of the training speech signals being substantially the same in each segment;

dividing said training speech signals into a succession of frames;

determining, for each frame, acoustic features of the training speech signal during the frame;

defining a succession of segments within the training speech signals, and thus defining the location of trial boundaries between segments, by (i) comparing signals representative of the acoustic features for a current frame with signals representative of the acoustic features of previous frames in the current segment;

(ii) including the current frame in the current segment if said signals differ by less than a threshold value;

(iii) beginning a new segment with the current frame if said signals differ by more than said threshold value; and

(iv) performing steps (i) to (iii) for a plurality of different threshold values;

comparing the trial boundaries between segments defined in said defining step for the different threshold values with the known boundaries of the training speech signals; and

storing the threshold value which gives a desired good correlation between the known boundaries and the trial boundaries defined in said defining step.

28. A speech processing method comprising the steps of:

providing a set of training speech signals having known phonetic boundaries and/or known boundaries between segments in the training speech signals, acoustic features of the training speech signals being substantially the same in each segment;

dividing said training speech signals into a succession of frames;

determining, for each frame, acoustic features of the training speech signal during the frame;

defining a succession of segments within the training speech signals, and thus defining the location of trial boundaries between segments, by (i) comparing signals representative of the acoustic features for a current frame with signal representative of the acoustic features of previous frames in the current segment;

(ii) including the current frame in the current segment if said signals differ by less than a threshold value;

(iii) beginning a new segment with the current frame if said signals differ by more than said threshold value; and

(iv) performing steps (i) to (iii) for a plurality of different threshold values;

comparing the trial boundaries between segments defined in said defining step of the different threshold values with the known boundaries of the training speech signals;

storing the threshold value which gives a good correlation between the known boundaries and the trial boundaries defined in said defining steps;

using the stored threshold value to segment subsequent input speech; and

determining articulatory speech production parameters of the segmented speech.

29. A speech processing method comprising the steps of:

dividing a set of training speech signals representative of utterances from a plurality of different users into a succession of frames for each utterance;

determining acoustic features of the training speech signal during each frame;

defining a succession of segments within each utterance of training speech by (i) comparing signals representative of the acoustic features for a current frame with signals representative of the acoustic features of previous frames in a current segment;

(ii) including the current frame in the current segment if said signals differ by less than a threshold value; and

(iii) beginning a new segment with the current frame if said signals differ by more than said threshold value;

identifying boundaries between acoustic classes representative of acoustic characteristics identified in the set of training speech signals by using segments in all utterances of the training speech;

determining reference data modelling each acoustic class by using segments in all utterances of the training speech;

storing said reference data; and

using the stored reference data to determine articulatory speech production parameters of subsequently input speech.

30. A method according to claim 29, further comprising the steps of:

providing a second set of training speech signals including values of parameters defining the operation of the speech production system that generated the second set of training speech signals;

dividing the second set of training speech signals into a succession of frames;

determining acoustic features the training speech signal during each frame;

defining a succession of segments within the second set of training speech signals by (i) comparing signals representative of the acoustic features for a current frame with signals representative of the acoustic features of previous frames in a current segment;

(ii) including the current frame in the current segment if said signals differ by less than a threshold value; and

(iii) beginning a new segment with the current frame if said signals differ by more than said threshold value;

determining, from said reference data grouping segments in the first set of training speech signals into different acoustic classes, which acoustic class each segment defined in said second set of training speech signals most likely belongs to, by comparing signals representative of each segment defined in said second set of training speech signals with said reference data grouping the segments defined in said first set of training speech signals; and

determining and storing, for each acoustic class found in said first set of training speech, reference data representative of a relationship between said values of parameters and said acoustic features, using the segments defined in the first set of training speech signals in the corresponding acoustic class.

31. A method according to claim 30, further comprising the step of determining from said acoustic features of each frame of training speech in said second set of training speech signals, the probability that a specific parameter has a specific parameter value, given that a segment defined in said second set of training speech signals is in a specific acoustic class, for each parameter and parameter value in each acoustic class.

32. A method according to claim 29, further comprising the step of determining from said values of parameters defining the operation of the speech production system that generated the first set of training speech signals, the probability that a specific parameter parameter of the speech production system has a specific parameter value, for each parameter and each parameter value.

33. A method according to claim 30, further comprising the step of determining from said values of parameters defining the operation of the speech production system that generated the first set of training speech signals, the probability that a specific parameter of the speech production system has a specific parameter value, for each parameter and each parameter value.

35. A method of determining, from an input speech signal, parameters defining the articulatory operation of a speech production system that generated the input speech signal, the method comprising the steps of:

dividing the input speech signal into a succession of frames;

determining acoustic features of the input speech signal during each frame;

defining a succession of segments as a number of frames by (i) comparing signals representative of the acoustic features for a current frame with signals representative of the acoustic features of previous frames in a current segment;

(ii) including the current frame in the current segment if said signals differ by less than a threshold value; and

(iii) beginning a new segment with the current frame if said signals differ by more than said threshold value; and

determining, for each segment, said articulatory speech production parameters using the acoustic features of the input speech signal during the segment and using stored reference data which relates said acoustic features to said articulatory speech production parameters.

36. A method according to claim 35, wherein said difference is determined using a weighted Euclidean distance measure.

37. A method according to claim 35, wherein said defining step comprises the step of generating smoothed signals representative of the acoustic features of each frame, by smoothing variations between the acoustic features of adjacent frames and wherein said defining step performs said comparison using smoothed signals representative of the current frame and smoothed signals representative of the previous frames in the current segment.

38. A method according to claim 35, wherein said defining step is operable to limit the number of frames allowed within each segment.

39. A method according to claim 35, wherein said step of determining acoustic features of the input speech signal during a frame comprises the step of determining formant related information of the input speech signal during each frame.

40. A method according to claim 35, wherein said reference data comprises information that groups a selection of segments of input speech into different acoustic classes representative of acoustic characteristics of speech signals, and wherein said method comprises the step of classifying each defined segment and wherein said step of determining parameters determines said parameters using said classification information.

41. A method according to claim 40, wherein said classifying step determines, for each class, the probability that the current segment belongs to that class.

42. A method according to claim 40, wherein said classifying step determines the most likely class to which the current segment belongs.

43. A method according to claim 40, wherein said information that groups said selection of segments into different classes comprises a plurality of mathematical models, one for each acoustic class, each model modelling the distribution of the acoustic features of the input speech during the selected segments in the class.

44. A method according to claim 43, wherein each said mathematical model is an average.

45. A method according to claim 43 or 44, wherein said classifying step compares signals representative of the current segment with each said mathematical model.

46. A method according to claim 45, wherein said signals representative of the current segment are the average of the acoustic features of the input speech signal during the current segment.

47. A method according to claim 40, wherein each of said parameters has associated therewith a number of discrete parameter values and wherein said reference data comprises a mathematical model for each parameter value in each acoustic class, relating that parameter value to said acoustic features.

48. A method according to claim 47, wherein said mathematical model for each parameter value in each acoustic class is a Gaussian density function.

49. A method according to claim 47, wherein said reference data further comprises the probability that a specific parameter defining the operation of the speech production system has one of said associated discrete parameter values given that a segment is in a specific acoustic class, for each parameter and parameter value in each acoustic class.

50. A method according to claim 49, wherein said reference data further comprises the probability that a specific parameter defining the operation of the speech production system has one of said associated discrete parameter values, for each parameter and each parameter value.

51. A method according to claim 47, wherein said mathematical model for each parameter value in each acoustic class is used to estimate the probability of receiving an input speech signal corresponding to the current frame, given that the current frame is in a segment in the acoustic class for that model, and assuming that the parameter has the parameter value for that model.

52. A method according to claim 35, wherein said step of determining said parameters operates for each frame of input speech.

53. A method according to claim 51, wherein said step of determining said parameters determines the probability of receiving an input speech signal corresponding to the current frame given that a specific parameter defining the operation of the speech production system has one of said associated parameter values, for each parameter and parameter value in dependence upon said reference data.

54. A method according to claim 35, wherein said step for determining said parameters operates once per segment, in dependence upon signals representative of the segment.

55. A method according to claim 39, wherein said classifying step classifies each segment in dependence upon the classification of previous segments.

56. A method according to claim 35, wherein said defining step compares the signals representative of the acoustic features of previous frames in the current segment with the signals representative of the acoustic features for the current frame as soon as the signals representative of the acoustic features for the current frame have been determined.

57. A method according to claim 35, wherein said step of determining said parameters operates as soon as the last frame in the segment has been determined by said defining step.

58. A method according to claim 35, comprising the step of buffering acoustic features corresponding to a plurality of frames, and wherein said defining step defines a plurality of segments within the stored acoustic features corresponding to said plurality of frames, after said acoustic features have been determined.

59. A method according to claim 58, wherein said step of determining said parameters operates as soon as each segment is defined.

60. A method according to claim 53, further comprising the step of deciding the value of each parameter, based on the probabilities determined for that parameter.

61. A method according to claim 60, wherein said decision step decides the value of each parameter in dependence upon past decisions of parameter values.