Method and system for detecting and generating transient conditions in auditory signals

• US 5,884,260 A
• Filed: 12/12/1995
• Issued: 03/16/1999
• Est. Priority Date: 04/22/1993
• Status: Expired due to Term

• Current Assignee: Frank Uldall Leonhard
• Original Assignee: Frank Uldall Leonhard
• Inventors: Leonhard, Frank Uldall
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Storm, Donald L.
• Application Number:
  US08/537,844
• Time in Patent Office: 1,190 Days
• Field of Search: 704/276, 704/243, 704/223, 704/254, 704/249, 704/215, 704/214, 704/211, 704/253, 704/248
• US Class Current: 704/254
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit

Abstract

The shape of energy changes of an auditory signal is used for identifying or representing features which can be perceived by a human ear as representing a distinct sound picture. In order to extract information from the shape of the energy changes, the shape is preferably represented by the shape of a transient pulse of the signal with a rise time of at most 2 ms. It is preferred that an envelope detection is being used in order to obtain the transient signal pulse. The energy change representing the distinct sound picture can be a phoneme or vowel. The invention also relates to a method for identifying the energy changes in the auditory signal by comparing the shape of energy changes of the signal, which can be represented as the shape of the transient pulse, with predetermined energy change shapes representing distinct sound pictures. The invention also relates to a method of speech synthesis wherein a series of transient pulses is generated corresponding to the series of phonemes to be synthesized. The invention further relates to a system for processing an auditory signal in order to reduce the bandwidth of the signal with substantial retention of the information of the signal, the system comprising means for extracting the transient component of the auditory signal, and means for detecting an envelope the transient component.

Claims

1. A method of using a shape of an abrupt energy change in an auditory signal for identifying or representing distinct sound pictures which can be perceived by an animal or human ear, said method comprising the step of:

deriving, from the auditory signal, at most one transient signal including transient pulses representing abrupt energy changes in the auditory signal having a rise time of at most 2 ms.

2. The method according to claim 1, wherein the transient signal is derived from the auditory signal by envelope detection.

3. The method according to claim 1, wherein the distinct sound picture is a phoneme.

4. A method according to claim 1, wherein said method is used in speech recognition.

5. A method according to claim 1, wherein said method is used in speech compression.

6. A method according to claim 1, used for synthesizing human speech, wherein a series of the transient pulses are generated, corresponding to a series of phonemes which constitute the human speech to be synthesized.

7. A method according to claim 6, wherein the series of phonemes is established from a series of letters using rule-based conversion.

8. A method according to claim 1, used in quality-measurement of audio products, the audio products being loudspeakers, hearing aids or telecommunication systems.

9. A method according to claim 1, used in quality-measurement of acoustic conditions in a room or in an open environment.

29. A method according to claim 5, used in speech compression.

10. A method for identifying, in an auditory signal, energy changes which can be perceived by an animal or human ear as representing a distinct sound picture the method comprising the steps of:

deriving, from the auditory signal, at most one transient signal including transient pulses representing the abrupt energy changes in the auditory signal having a rise time of at most 2 ms,

selecting a dominant pulse of the transient pulses in the transient signal, and

comparing a shape of the dominant pulse with predetermined transient signal pulses representing distinct sound pictures.

11. A method according to claim 10, wherein the shapes of the transient pulses are obtained by envelope detection of a transient response of an energy change in the auditory signal.

12. A method according to claim 10, wherein the shape of a leading edge of the transient pulses is identified.

13. A method according to claim 12, wherein the shape of the leading edge is determined by determining a rise time and a slope and/or slope variation of at least part of the leading edge.

14. A method according to claim 13, wherein the rise time and the slope and/or slope variation of at least a top part of the leading edge is determined.

15. A method according to claim 13, wherein the top part is a part beginning substantially at a point where the slope is maximum.

16. A method according to claim 15, wherein the rise time and the slope and/or slope variation of the leading edge is determined based on at least 5 samples.

17. A method according to claim 12, wherein the identification of the shape of the leading edge is performed by comparison with a library of references.

18. A method according claim 17, wherein the library of references with which comparison is made is selected based on the rise time of the leading edge.

19. A method according to claim 10, wherein a duration of the transient pulses is identified.

20. A method according to claim 19, wherein the duration of the transient pulses is determined as a distance from a leading edge to a lagging edge at a predetermined amplitude.

21. A method according to claim 20, wherein the predetermined amplitude is an amplitude of at most 50% of a maximum amplitude of the pulse.

22. A method according to claim 12 wherein the transient pulses which cannot be perceived by the animal ear are discarded from identification.

23. A method according to claim 22, wherein a transient pulse having a leading edge with an amplitude of less than 50% of an amplitude of a preceding pulse and an onset time of less than 3.5 ms, is disregarded.

24. A method according to claim 12, wherein the shape of a lagging edge of the transient pulses is identified.

25. A method according to claim 24, wherein the shape of the lagging edge is determined by determining a fall time and a slope and/or slope variation of at least part of the leading edge.

26. A method according to claim 12, wherein a time period between leading edges of the transient pulses which can be perceived by the animal ear is determined.

27. A method according to claim 26, wherein, prior to transmission of the processed signal, the signal is coded into a digital representation, and the coded signal is decoded in the receiver so as to reestablish transient pulse shapes perceived by the animal ear such as the human ear as representing the distinct sound pictures of the auditory signal.

28. A method according to claim 10, used in speech recognition.

30. A method according to claim 10, used for synthesizing human speech, a series of the transient pulses are generated, corresponding to a series of phonemes which constitute the human speech to be synthesized.

31. A method according to claim 30, wherein the series of phonemes is established from a series of letters using rule-based conversion.

32. A method according to claim 10, used in quality-measurement of audio products, the audio products being loudspeakers, hearing aids or telecommunication systems.

33. A method according to claim 10, used in quality-measurement of acoustic conditions in a room or in an open environment.

34. A system for processing an auditory signal to reduce the bandwidth of the auditory signal with substantial retention of information of the auditory signal, comprising the steps of:

means for extracting the transient component corresponding to an abrupt energy change of the auditory signal, and

means for detecting an envelope of the transient component, said means for detecting the envelope, deriving from the transient component, at most one transient signal including transient pulses representing abrupt energy changes having a rise time of at most 2 ms.

35. A system according to claim 34, further comprising means for identifying or representing the abrupt energy changes based on a shape of the transient pulses.

36. A system according to claim 34, wherein said means for extracting includes a bandpass filter or a highpass filter.

37. A system according to claim 36, wherein a lower cutoff frequency of the bandpass or highpass filter is at least 2 kHz.

38. A system according to claim 36, wherein an upper cutoff frequency of the bandpass filter is in a range between 4.5 and 7 kHz.

39. A system according to claim 34, wherein said means for detecting includes a rectifier and a lowpass filter.

40. A system according to claim 39, wherein the rectifier is a one-way rectifier.

41. A system according to claim 39, wherein a cutoff frequency of the lowpass filter is in a range of 400-1000 Hz.

42. A system according to claim 34, wherein said means for detecting includes a filter bank.

43. A method for processing an auditory signal to reduce a bandwidth of the auditory signal with substantial retention of information of the auditory signal, comprising the steps of:

extracting a transient component corresponding to an abrupt energy change of the auditory signal; and

detecting an envelope of the transient component to obtain, from the transient component, at most one transient signal including transient pulses representing abrupt energy changes having rise times of at most 2 ms.

44. A method according to claim 43, wherein transient pulses of the auditory signal, which can be perceived by an animal or human ear, as representing a distinct sound picture, are identified.

45. A method according to claim 44, wherein the distinct sound picture is a phoneme.

46. A method according to claim 44, wherein a shape of a leading edge of the transient pulses is identified.

47. A method according to claim 46, wherein the shape of the leading edge is determined by determining a rise time and a slope and/or slope variation of at least part of the leading edge.

48. A method according to claim 47, wherein the rise time and the slope and/or slope variation of at least a top part of the leading edge is determined.

49. A method according to claim 48, wherein the top part is a part beginning substantially at a point where the slope is maximum.

50. A method according to claim 47, wherein the rise time and the slope and/or slope variation of the leading edge is determined based on at least 5 samples.

51. A method according to claim 46, wherein the identification of the shape of the leading edge is performed by comparison with a library of references.

52. A method according to claim 51, wherein the library of references with which comparison is made are selected based on the rise time of the leading edge.

53. A method according to claim 46, wherein the transient pulses which cannot be perceived by the animal ear are discarded from identification.

54. A method according to claim 53, wherein a transient pulse having a leading edge with an amplitude of less than 50% of an amplitude of a preceding pulse and an onset time of less than 3.5 ms, is disregarded.

55. A method according to claim 46, wherein the shape of a lagging edge of the transient pulses is identified.

56. A method according to claim 55, wherein the shape of the lagging edge is determined by determining a fall time and a slope and/or slope variation of at least part of the leading edge.

57. A method according to claim 46, wherein a time period between leading edges of the transient pulses, which can be perceived by the animal ear, is determined.

58. A method according to claim 57, wherein the time period between leading edges which have a distance of at least 3 ms from each other, is determined.

59. A method according to claim 44, wherein a duration of the transient pulses is identified.

60. A method according to claim 59, wherein the duration of the transient pulses is determined as a distance from a leading edge to a lagging edge at a predetermined amplitude.

61. A method according to claim 60, wherein the predetermined amplitude is an amplitude of at most 50% of a maximum amplitude of the transient pulses.

62. A method according to claim 43, further comprising the steps of:

transmitting the transient signal, and receiving the transient signal with a receiver, to thereby telecommunicate the auditory signal.

63. A method according to claim 62, wherein, prior to transmission of the transient signal, the transient signal is digitally coded, and the coded signal is decoded in the receiver so as to reestablish the transient pulses perceived by the animal or human ear, as representing distinct sound pictures of the auditory signal.

64. A method according to claim 63, wherein transmission of the digitally coded transient signal is performed at a bandwidth of at most 4000 bits per second.

65. A method according to claim 64, wherein the bandwidth is at most 2000 bits per second.

66. A method according to claim 65, wherein the bandwidth is in an interval of 800-2000 bits per second.

67. A method according to claim 63, wherein the digitally coded transient signal includes information about a leading edge, a lagging edge, and a duration of each transient pulse.

68. A method according to claim 63, wherein a second and subsequent pulses in a sequence of identical pulses are represented by a digital sign indicating repetition.

69. A method according to claim 43, wherein the transient component is extracted by bandpass filtering or highpass filtering.

70. A method according to claim 69, wherein a lower cutoff frequency of the bandpass or highpass filtering is at least 2 kHz.

71. A method according to claim 69, wherein an upper cutoff frequency is in a range between 4.5 and 7 kHz.

72. A method according to claim 43, wherein the envelope is detected by rectification and lowpass filtering.

73. A method according to claim 72, wherein the envelope is detected by one-way rectification.

74. A method according to claim 72, wherein a cutoff frequency of the lowpass filtering is in a range of 400-1000 Hz.

75. A method according to claim 43, wherein the envelope is detected by bandpass filtering, using a bank of filters.

76. A method according to claim 43, used in speech recognition.

77. A method of identifying or representing a phoneme "e" as in "heat", comprising the steps of:

deriving at most one transient signal corresponding to abrupt energy changes of the phoneme; and

identifying or generating a transient pulse in the transient signal with a rise time of a leading edge of at most 0.5 ms and a duration of at most 1.1 ms.

78. A method according to claim 77, wherein the rise time of the leading edge is at most 0.4 ms.

79. A method according to claim 77, wherein a duration is at most 1.0 ms.

80. A method according to claim 77, wherein a fall time of a lagging edge is at most 0.3 ms.

81. A method according to claim 77, wherein said method is used in speech recognition.

82. A method according to claim 77, used in speech compression.

83. A method according to claim 77, used for synthesizing human speech, further comprising the step of:

replicating the transient pulse to provide a series of the transient Pulses corresponding to a series of phonemes wherein the series of the transient pulses are generated corresponding to the series of phonemes which constitute the human speech to be synthesized.

84. A method according to claim 83, wherein the series of phonemes is established from a series of letters using rule-based conversion.

85. A method according to claim 77, used in quality-measurement of audio products, the audio products being loudspeakers, hearing aids or telecommunication systems.

86. A method according to claim 77, used in quality-measurement of acoustic conditions in a room or in an open environment.

87. A method of identifying or representing a phoneme "o" as in an English word "ongaonga" or a Danish word "Ole", comprising the steps of:

deriving at most one transient signal corresponding to abrupt energy changes of the phoneme; and

identifying or generating a transient pulse in the transient signal with a rise time of a leading edge of at most 0.5 ms and a duration of 1.3-1.8 ms.

88. A method according to claim 87, wherein the rise time of the leading edge is at most 0.4 ms.

89. A method of identifying or representing a phoneme "u" as in an English word "who", comprising the steps of:

deriving at most one transient signal corresponding to abrupt energy changes of the phoneme; and

identifying or generating a transient pulse in the transient signal with a sine curve interpolation and a duration of 1.0-2.0 ms.